{"schema_version":"ICEQC_NEWS_ARTICLE_V1","id":"iceqc-news-11f1a13520efe113","slug":"2024-09-24-assessing-the-causes-of-underperformance-in-ai-competency-frameworks","language":"en","publication_status":"READY_FOR_IMPORT","publication_date":"2024-09-24","last_modified_date":"2024-09-24","title":"Assessing the causes of underperformance in AI competency frameworks","summary":"Sets out a controlled approach to assessing the causes of underperformance in AI competency frameworks, linking diagnosis, accountable action, outcome measures and verification.","category":{"code":"QUALITY_IMPROVEMENT_METHODS","label":"Quality Improvement Methods"},"article_type":"Quality improvement method","publisher":"International Council for Education Quality Certification (ICEQC)","jurisdictional_scope":"International","historical_reference_basis":"International frameworks for students and teachers released in 2024","reference_authority":"United Nations Educational, Scientific and Cultural Organization","sections":[{"heading":null,"paragraphs":["The present attention to the causes of underperformance in AI competency frameworks follows the international frameworks for students and teachers released in 2024 and requires a careful distinction between public commitment, institutional practice and demonstrated result. The analysis of the practice proceeds on the basis that effective improvement requires ownership, a time-bound intervention and independent confirmation that the intended result has been achieved.","The formal status of the international frameworks for students and teachers released in 2024 should be preserved in any public account. For the improvement objective, the instrument should be used to identify the intended direction, the actors addressed and the implementation measures that remain necessary.","International artificial-intelligence competency frameworks for students and teachers were released in September 2024. They organise capability around human-centred understanding, ethics, techniques and application, with teacher responsibilities also covering pedagogy and professional development. Competency frameworks guide curriculum and workforce planning; they do not establish that competence has been achieved without suitable learning and assessment evidence.","Implementation of the practice should be organised around a decision that can be tested. Oversight of the corrective programme should reflect the principle that a complete improvement record should define the baseline, affected scope, causal hypothesis, responsible owner, resources, milestones and measures of effectiveness. In relation to assessing the causes of underperformance in AI competency frameworks, the implementation record should link purpose, authority, resources, operation and reported result."]},{"heading":"Defining the problem","paragraphs":["The system and institutional dimensions of the causes of underperformance in AI competency frameworks should be considered together. A decision concerning the practice should recognise that technology may support teaching, administration and access, but consequential educational decisions must remain accountable, explainable and open to effective review. Public authorities establish the legal and policy setting; providers remain accountable for the quality and integrity of provision within their control.","The technical issue within the intervention concerns the basis on which a conclusion is reached. For the intervention, the subject should be examined as a connected system of policy, people, resources, decisions and evidence. Assurance should examine the complete process, including each change in custody, authority or decision ownership. The decision record should distinguish the scope supported by evidence from any scope that remains unresolved.","The principal risks in relation to the issue under review are opaque use of personal or inferred data, unclear responsibility between providers and suppliers, automation bias in consequential decisions, and loss of meaningful human review. In relation to assessing the causes of underperformance in AI competency frameworks, the relationship between the risks is material: one failed safeguard may remove the evidence needed to activate another. Documents should be tested against the decision process they record and the outcome that followed.","Relevant evidence for the intervention will normally include supplier change and incident records, an inventory of systems and their intended uses, documented authority for each consequential use, data provenance and access controls, and pre-deployment and periodic performance testing. The record should retain disagreement between sources until its cause and effect are understood."]},{"heading":"Improvement method","paragraphs":["The review method for the causes of underperformance in AI competency frameworks should be reproducible. For the improvement objective, the reviewer should map the complete process, identify the intended result and responsible authority at each stage, and test normal cases together with exceptions. Recurrence, common cause or wider exposure requires systemic action in addition to correction of individual cases. Working papers should allow another competent reviewer to understand the evidence, judgement and treatment of material exceptions.","Improvement of the practice should proceed through controlled tests where risk permits.","Decisions concerning the improvement objective should remain traceable to the information available for the stated reference period. A revision should state whether the change concerns the underlying condition, the evidence, the method or the interpretation. Without this distinction, a reporting change may be mistaken for improvement or deterioration in educational practice."]},{"heading":"Measures and review","paragraphs":["Care is required in drawing conclusions about the causes of underperformance in AI competency frameworks. For the practice, a technical capability is not evidence that a use is educationally justified. Accuracy measured in one setting may not transfer to another population, language, curriculum or decision context. For the issue under review, methods should be proportionate to the significance and recurrence of the problem; low-risk local issues and systemic learner-protection failures require different levels of control. A finding should not be separated from limitations capable of changing how it is understood or applied.","Public reporting on the issue under review should distinguish established fact, analytical judgement and planned action. A revised conclusion should distinguish a change in the underlying condition from a change in method, coverage or evidence.","The appropriate response to the intervention is therefore one of controlled implementation and review. Where evidence cannot support assurance, the limitation should be reported and corrective work should remain open."]}],"word_count":790,"content_hash":"sha256-55d28d5293c3995ac00e2e4a82663f1bb865e80c84ad05fd487a6740f29186a2","seo_keywords":["assessing the causes of underperformance in AI competency frameworks","education quality improvement","education quality","ICEQC"],"schema_type":"TechArticle"}