{"schema_version":"ICEQC_NEWS_ARTICLE_V1","id":"iceqc-news-0d73fc0438195a5f","slug":"2019-08-21-evidence-sufficiency-in-relation-to-artificial-intelligence-in-education-policy","language":"en","publication_status":"READY_FOR_IMPORT","publication_date":"2019-08-21","last_modified_date":"2019-08-21","title":"Evidence sufficiency in relation to artificial intelligence in education policy","summary":"Clarifies the practical meaning of evidence sufficiency in relation to artificial intelligence in education policy and the evidence required to support a reliable assurance.","category":{"code":"STANDARDS_INTERPRETATION","label":"Standards Interpretation"},"article_type":"Standards interpretation","publisher":"International Council for Education Quality Certification (ICEQC)","jurisdictional_scope":"International","historical_reference_basis":"Beijing Consensus adopted in May 2019","reference_authority":"United Nations Educational, Scientific and Cultural Organization","sections":[{"heading":null,"paragraphs":["At the publication date, Beijing Consensus adopted in May 2019 provides the relevant international context for artificial intelligence in education policy. Any consequential application still requires evidence from the affected jurisdiction or institution. The governing expectation for the assurance question should be stated precisely enough to support consistent decisions without displacing applicable law or justified professional judgement. Any indicator used in relation to the issue under review should distinguish description from causal explanation. Interpretation should retain uncertainty, distributional differences and limits on generalisation.","The contemporaneous reference point for the issue under review is Beijing Consensus adopted in May 2019. Its status should be distinguished from the jurisdiction-specific evidence required for implementation. The public-interest assessment of the stated expectation should consider access, learning, fair treatment and the reliability of information on which learners make consequential decisions."]},{"heading":"Applicable scope","paragraphs":["The Beijing Consensus on Artificial Intelligence and Education was adopted in May 2019. It addresses policy planning, management, teaching, learning, skills, lifelong learning, inclusion, gender equality, data and research. The position on artificial intelligence in education policy should be established through proportionate evidence and should remain open to correction when material new information becomes available. Authorities and providers should therefore connect each proposed use to an educational purpose, governance responsibility and evidence of benefit and risk.","In relation to evidence sufficiency in relation to artificial intelligence in education policy, the relevant outcome should be capable of direct and consistent explanation. Responsibility for the assurance question should be identifiable at each consequential decision point. Delegating operational work does not transfer accountability for its effect on learners. Inputs and formal commitments should be distinguished from demonstrated operation and outcome. Implementation evidence should be sufficient to identify unequal consequences and assign corrective responsibility.","The governing expectation for the issue under review should be stated precisely enough to support consistent decisions without displacing applicable law or justified professional judgement. The assurance record for the relevant requirement should permit another competent reviewer to understand the evidence, method, judgement and treatment of material exceptions. Responsibility for the control should be identifiable at each consequential decision point. The responsible authority remains accountable for material learner effects despite operational delegation.","Responsibility for the stated expectation should be visible at the point where consequential decisions are made. Records concerning the stated expectation should remain traceable from source evidence to decision and follow-up. Superseded conclusions should be retained where they informed a material outcome. Incomplete evidence, unmanaged conflict, absent learner groups or material learner impact require a higher level of review.","Reporting on the control should distinguish established fact, analytical judgement and planned action. Material revisions should retain their reason and effective date. Any indicator used in relation to the relevant requirement should distinguish description from causal explanation. A reported result should state how outcomes are distributed and where transfer beyond the observed setting is not supported. In relation to evidence sufficiency in relation to artificial intelligence in education policy, independent records should be reconciled, with disagreement and uncertainty reported alongside the finding."],"bullets":["Prohibit uses for which evidence or authority is insufficient before it informs a consequential decision.","Classify uses by effect on learners.","Notify users of material limitations.","Control personal and confidential information.","Retain accountable human decision-makers."]},{"heading":"Implementation and evidence","paragraphs":["The principal risks associated with artificial intelligence in education policy should be assessed as connected conditions. The record for the stated expectation should identify the responsible function, decision authority and escalation route. Examination of the matter should include the experience of affected learners, particularly where aggregate reporting may conceal exclusion, delay or unequal treatment.","Implementation of the stated expectation can be tested without imposing unnecessary reporting. Reporting on the assurance question should distinguish established fact, analytical judgement and planned action. Expand the sample where an exception, complaint or material unexplained variation indicates that the initial evidence may not be representative. Reporting on the issue under review should distinguish established fact, analytical judgement and planned action.","Assurance concerning the issue under review should be expressed at the level established by the evidence. Review of the relevant requirement should give particular attention to adverse cases, unequal effects and errors that learners may be unable to identify or remedy after the event.","The analysis of the assurance question should remain within the limits of the evidence. Oversight of the stated expectation should reflect the principle that the volume of documentation is not a measure of conformity. Relevance, integrity and coverage are more important than the number of records produced. A decision concerning the control should recognise that a technical capability is not evidence that a use is educationally justified. Data used for the assurance question should be interpreted against stable definitions and an identifiable population. A revision or break in series should not be reported as a change in performance.","Reporting on the stated expectation should distinguish established fact, analytical judgement and planned action. Improvement work on the issue under review should begin with a verified problem, defined baseline and measurable outcome. Completion should depend on evidence of effect rather than completion of planned activity.","Accountability for the issue under review should follow decision-making authority. Risk assessment for the control should consider severity, reach, duration, recurrence and detectability, with escalation where learner impact may be material.","The assurance record for the assurance question should permit another competent reviewer to understand the evidence, method, judgement and treatment of material exceptions. Responsibility for the issue under review should be identifiable at each consequential decision point."]}],"word_count":903,"content_hash":"sha256-4c3c485a3af85040c2de7b5260ff1ea78491491033e8b4396a93954445679152","seo_keywords":["evidence sufficiency in relation to artificial intelligence in education policy","education quality standards","education quality","ICEQC"],"schema_type":"TechArticle"}