The article examines AI competency frameworks, separating supported observations from causal claims and identifying where further evidence is required.
In examining AI competency frameworks: evidence for policy decisions, for the measure, the instrument should be used to identify the intended direction, the actors addressed and the implementation measures that remain necessary.
In examining AI competency frameworks: evidence for policy decisions, for AI competency frameworks, international artificial-intelligence competency frameworks for students and teachers were released in September 2024. They organise capability around human-centred understanding, ethics, techniques and application, with teacher responsibilities also covering pedagogy and professional development. Competency frameworks guide curriculum and workforce planning; they do not establish that competence has been achieved without suitable learning and assessment evidence.
For the measure, the public interest is not confined to institutional compliance. In reviewing AI competency frameworks, technology may support teaching, administration and access, but consequential educational decisions must remain accountable, explainable and open to effective review.
Application to AI competency frameworks
In the context of AI competency frameworks, the subject should be examined as a connected system of policy, people, resources, decisions and evidence.
Controls for AI competency frameworks
The principal risks in relation to AI competency frameworks are loss of meaningful human review, unverified outputs entering teaching or assessment, unclear responsibility between providers and suppliers, and automation bias in consequential decisions. Risk assessment should account for dependencies between controls and the possibility that one failure masks the next.
- Review incidents and supplier changes.
- Classify uses by effect on learners.
- Notify users of material limitations, with responsibility, scope and timing recorded.
- Retain accountable human decision-makers.
- Prohibit uses for which evidence or authority is insufficient.
Review of AI competency frameworks
Useful records include data provenance and access controls, supplier change and incident records, records of human review and overrides, learner information and accessible challenge routes, and an inventory of systems and their intended uses. Documents should be reconciled with observed practice and, where relevant, the experience of affected learners.
The review method for the measure should be reproducible. Review of the available evidence should map the complete process, identify the intended result and responsible authority at each stage, and test normal cases together with exceptions. Across the defined scope, working papers should allow another competent reviewer to understand the evidence, judgement and treatment of material exceptions.
Implications for AI competency frameworks
In the context of AI competency frameworks, traceability is necessary for accountable decision-making and fair correction. For comparative analysis, the responsible body should be able to identify the evidence considered, the judgement made, the person or body authorised to make it and the action that followed.
Accountability for the measure should follow decision-making authority. For decisions concerning AI competency frameworks, relevant evidence should reach the body authorised to commit resources, amend policy or accept residual risk, and its judgement should be recorded.
Data used for the comparison should be interpreted against stable definitions and an identifiable population. For AI competency frameworks, a revision or break in series should not be reported as a change in performance.