Assessing the causes of underperformance in AI competency frameworks

The present attention to the causes of underperformance in AI competency frameworks follows the international frameworks for students and teachers released in 2024 and requires a careful distinction between public commitment, institutional practice and demonstrated result. Effective improvement requires ownership, a time-bound intervention and independent confirmation that the intended result has been achieved.

The formal status of the international frameworks for students and teachers released in 2024 should be preserved in any public account. For the intended improvement, the instrument should be used to identify the intended direction, the actors addressed and the implementation measures that remain necessary.

For AI competency frameworks, international artificial-intelligence competency frameworks for students and teachers were released in September 2024. They organise capability around human-centred understanding, ethics, techniques and application, with teacher responsibilities also covering pedagogy and professional development. Competency frameworks guide curriculum and workforce planning; they do not establish that competence has been achieved without suitable learning and assessment evidence.

Implementation of the relevant practice should be organised around a decision that can be tested. In the context of AI competency frameworks, a complete improvement record should define the baseline, affected scope, causal hypothesis, responsible owner, resources, milestones and measures of effectiveness. The implementation record should link purpose, authority, resources, operation and reported result.

Defining the problem

The system and institutional dimensions of the causes of underperformance in AI competency frameworks should be considered together. Technology may support teaching, administration and access, but consequential educational decisions must remain accountable, explainable and open to effective review. Public authorities establish the legal and policy setting; providers remain accountable for the quality and integrity of provision within their control.

For the corrective action, the subject should be examined as a connected system of policy, people, resources, decisions and evidence. Assurance should examine the complete process, including each change in custody, authority or decision ownership. The decision record for AI competency frameworks should distinguish the scope supported by evidence from any scope that remains unresolved.

The principal risks in relation to the matter are opaque use of personal or inferred data, unclear responsibility between providers and suppliers, automation bias in consequential decisions, and loss of meaningful human review. As regards AI competency frameworks, the relationship between the risks is material: one failed safeguard may remove the evidence needed to activate another. Within the scope under review, documents should be tested against the decision process they record and the outcome that followed.

Relevant evidence for the corrective action will normally include supplier change and incident records, an inventory of systems and their intended uses, documented authority for each consequential use, data provenance and access controls, and pre-deployment and periodic performance testing. The record for AI competency frameworks should retain disagreement between sources until its cause and effect are understood.

Improvement method

The review method for the causes of underperformance in AI competency frameworks should be reproducible. For the intended improvement, the reviewer should map the complete process, identify the intended result and responsible authority at each stage, and test normal cases together with exceptions. Recurrence, common cause or wider exposure requires systemic action in addition to correction of individual cases. Working papers should allow another competent reviewer to understand the evidence, judgement and treatment of material exceptions.

Improvement of AI competency frameworks should proceed through controlled tests where risk permits.

Decisions concerning the intended improvement should remain traceable to the information available for the stated reference period. When examining AI competency frameworks, a revision should state whether the change concerns the underlying condition, the evidence, the method or the interpretation. Without this distinction, a reporting change may be mistaken for improvement or deterioration in educational practice.

Measures and review

Care is required in drawing conclusions about the causes of underperformance in AI competency frameworks. For the relevant practice, a technical capability is not evidence that a use is educationally justified. Accuracy measured in one setting may not transfer to another population, language, curriculum or decision context. For the matter, methods should be proportionate to the significance and recurrence of the problem; low-risk local issues and systemic learner-protection failures require different levels of control. A finding should not be separated from limitations capable of changing how it is understood or applied.

Public reporting on AI competency frameworks should distinguish established fact, analytical judgement and planned action. A revised conclusion should distinguish a change in the underlying condition from a change in method, coverage or evidence.

Where evidence concerning AI competency frameworks cannot support assurance, the limitation should be reported and corrective work should remain open.