Evidence sufficiency in relation to artificial intelligence in education policy

At the publication date, Beijing Consensus adopted in May 2019 provides the relevant international context for artificial intelligence in education policy. Any consequential application still requires evidence from the affected jurisdiction or institution. The applicable expectation should be stated precisely enough to support consistent decisions without displacing applicable law or justified professional judgement. Any indicator used in relation to the matter should distinguish description from causal explanation. Interpretation should retain uncertainty, distributional differences and limits on generalisation.

The contemporaneous reference point for the matter is Beijing Consensus adopted in May 2019. As regards artificial intelligence in education policy, its status should be distinguished from the jurisdiction-specific evidence required for implementation. The public-interest assessment of the applicable expectation should consider access, learning, fair treatment and the reliability of information on which learners make consequential decisions.

Applicable scope

The Beijing Consensus on Artificial Intelligence and Education was adopted in May 2019. It addresses policy planning, management, teaching, learning, skills, lifelong learning, inclusion, gender equality, data and research. The position on artificial intelligence in education policy should be established through proportionate evidence and should remain open to correction when material new information becomes available. Authorities and providers should therefore connect each proposed use to an educational purpose, governance responsibility and evidence of benefit and risk.

For artificial intelligence in education policy, the relevant outcome should be capable of direct and consistent explanation. Responsibility for the assurance conclusion should be identifiable at each consequential decision point. Delegating operational work does not transfer accountability for its effect on learners. Inputs and formal commitments should be distinguished from demonstrated operation and outcome. Implementation evidence should be sufficient to identify unequal consequences and assign corrective responsibility.

The applicable expectation should be stated precisely enough to support consistent decisions without displacing applicable law or justified professional judgement. The assurance record for the applicable requirement should permit another competent reviewer to understand the evidence, method, judgement and treatment of material exceptions. Responsibility for the control should be identifiable at each consequential decision point. The responsible authority remains accountable for material learner effects despite operational delegation.

For decisions concerning artificial intelligence in education policy, responsibility should be identifiable at the point where consequential decisions are made. Records concerning the applicable expectation should remain traceable from source evidence to decision and follow-up. Superseded conclusions should be retained where they informed a material outcome. Incomplete evidence, unmanaged conflict, absent learner groups or material learner impact require a higher level of review.

Reporting on the control should distinguish established fact, analytical judgement and planned action. When examining artificial intelligence in education policy, material revisions should retain their reason and effective date. Any indicator used in relation to the applicable requirement should distinguish description from causal explanation. A reported result should state how outcomes are distributed and where transfer beyond the observed setting is not supported. Independent records should be reconciled, with disagreement and uncertainty reported alongside the finding.

• Prohibit uses for which evidence or authority is insufficient before it informs a consequential decision.
• Classify uses by effect on learners.
• Notify users of material limitations.
• Control personal and confidential information.
• Retain accountable human decision-makers.

Implementation and evidence

The principal risks associated with artificial intelligence in education policy should be assessed as connected conditions. The record for the applicable expectation should identify the responsible function, decision authority and escalation route. Examination of the matter should include the experience of affected learners, particularly where aggregate reporting may conceal exclusion, delay or unequal treatment.

Implementation of the applicable expectation can be tested without imposing unnecessary reporting. Reporting on the assurance conclusion should distinguish established fact, analytical judgement and planned action. In reviewing artificial intelligence in education policy, expand the sample where an exception, complaint or material unexplained variation indicates that the initial evidence may not be representative. Reporting on the matter should distinguish established fact, analytical judgement and planned action.

Assurance concerning the matter should be expressed at the level established by the evidence. Review of the applicable requirement should give particular attention to adverse cases, unequal effects and errors that learners may be unable to identify or remedy after the event.

Within the scope under review, analysis should remain within the limits of the evidence. For artificial intelligence in education policy, the volume of documentation is not a measure of conformity. Relevance, integrity and coverage are more important than the number of records produced. A technical capability is not evidence that a use is educationally justified. Data used to assess artificial intelligence in education policy should be interpreted against stable definitions and an identifiable population. A revision or break in series should not be reported as a change in performance.

Reporting on the applicable expectation should distinguish established fact, analytical judgement and planned action. Improvement work on the matter should begin with a verified problem, defined baseline and measurable outcome. When examining artificial intelligence in education policy, completion should depend on evidence of effect rather than completion of planned activity.

Accountability for the matter should follow decision-making authority. Risk assessment for the control should consider severity, reach, duration, recurrence and detectability, with escalation where learner impact may be material.

The assurance record for artificial intelligence in education policy should permit another competent reviewer to understand the evidence, method, judgement and treatment of material exceptions. Responsibility for the matter should be identifiable at each consequential decision point.