The reliability of evidence on reporting learner agency is examined together with the limits that apply when findings inform consequential decisions.
Evidence base for reporting learner agency
For learner agency, the public interest is not confined to institutional compliance. For the measure, technology may support teaching, administration and access, but consequential educational decisions must remain accountable, explainable and open to effective review.
- Notify users of material limitations.
- Prohibit uses for which evidence or authority is insufficient.
- Review incidents and supplier changes.
- Retain accountable human decision-makers before it informs a consequential decision.
- Test performance across relevant groups, with responsibility, scope and timing recorded.
Application to reporting learner agency
Its relevance to learner agency should be assessed against the affected jurisdiction, learner population and form of provision.
Analysis should make its decision rule explicit. Data quality comprises accuracy, completeness, timeliness, consistency and traceability. When examining learner agency, strength in one dimension does not compensate automatically for weakness in another, particularly where the information informs a consequential learner decision.
Controls for reporting learner agency
Implementation of learner agency should be organised around a decision that can be tested. For the analysis, where an indicator is used as a proxy, the relationship between the proxy and the underlying educational outcome should be stated and tested.
The principal risks in relation to the available evidence are unclear responsibility between providers and suppliers, unequal performance across learner groups, automation bias in consequential decisions, and loss of meaningful human review. For decisions concerning reporting learner agency, the relationship between the risks is material: one failed safeguard may remove the evidence needed to activate another.
- Are validation rules operating?
- Can reported values be traced to source?
- Who may amend a record?
- What proportion is missing or late?
- Are revisions carried through to public reports?
Review of reporting learner agency
The evidential record for learner agency should permit a reviewer to trace the matter from decision to outcome. This may require learner information and accessible challenge routes, an inventory of systems and their intended uses, pre-deployment and periodic performance testing, and documented authority for each consequential use, supported by records of human review and overrides and supplier change and incident records.
Responsible bodies should trace selected records to source, reconcile totals across systems, quantify missing and late submissions, review manual adjustments and retain a revision history. For learner agency, escalate discrepancies that could alter a published conclusion or individual outcome.
Implications for reporting learner agency
Decision-makers using evidence on learner agency should be told what the data cannot establish as clearly as what it can. The nature of the result and its applicable unit—system, institution, programme or learner group—should be explicit.
Proportionality in relation to the comparison does not mean reduced protection for learners exposed to greater risk. For learner agency, a technical capability is not evidence that a use is educationally justified. Across the defined scope, accuracy measured in one setting may not transfer to another population, language, curriculum or decision context.
For reporting learner agency, decisions concerning the analysis should remain traceable to the information available for the stated reference period.
Evidence concerning the available evidence should be current, attributable and representative of the affected scope. For decisions concerning learner agency, material gaps or contradictions should remain visible in the conclusion.