Evidence relevant to automated decision oversight is assessed for currency, coverage and comparability, with material uncertainty stated alongside the finding.
The position at publication is informed by the expanding use of AI-supported education decisions; evidence from the affected setting remains necessary before reaching a conclusion on the available evidence. For automated decision oversight, decision-makers should state which matters are evidenced, which express policy and which require authorised judgement.
For automated decision oversight, technology may support teaching, administration and access, but consequential educational decisions must remain accountable, explainable and open to effective review.
Application to automated decision oversight
The principal risks in relation to automated decision oversight are opaque use of personal or inferred data, unclear responsibility between providers and suppliers, unverified outputs entering teaching or assessment, and unequal performance across learner groups. Across the defined scope, the relationship between the risks is material: one failed safeguard may remove the evidence needed to activate another.
- Classify uses by effect on learners.
- Control personal and confidential information.
- Test performance across relevant groups before using it to determine a learner or provider outcome.
- Retain accountable human decision-makers before using it to determine a learner or provider outcome.
- Notify users of material limitations before it informs a consequential decision.
Controls for automated decision oversight
For automated decision oversight, each source should have a stated purpose in supporting or limiting the conclusion. The most relevant material is likely to include records of human review and overrides, documented authority for each consequential use, learner information and accessible challenge routes, and an inventory of systems and their intended uses.
For operational review, authorities and providers should proceed in a defined sequence. Review of the analysis should map the complete process, identify the intended result and responsible authority at each stage, and test normal cases together with exceptions. When examining automated decision oversight, recurrence, common cause or wider exposure requires systemic action in addition to correction of individual cases.
Review of automated decision oversight
The analysis proceeds on the basis that a technical capability is not evidence that a use is educationally justified. Across the defined scope, decision-makers and affected users should receive the conclusion together with its material evidential limits.
For automated decision oversight, records relating to the comparison should preserve both the conclusion and its limits.
When examining automated decision oversight, For comparative analysis, governing bodies should receive a concise account of the intended result, affected scope, principal risks, evidence limitations and unresolved exceptions.