{"schema_version":"ICEQC_NEWS_ARTICLE_V1","id":"iceqc-news-b54e9f687e047f38","slug":"2023-09-28-measuring-teacher-ai-capability-across-education-systems","language":"en","publication_status":"READY_FOR_IMPORT","publication_date":"2023-09-28","last_modified_date":"2023-09-28","title":"Measuring teacher AI capability across education systems","summary":"Reviews measuring teacher AI capability across education systems and identifies the conditions required for sound interpretation, responsible comparison and defensible.","category":{"code":"DATA_RESEARCH","label":"Data Research"},"article_type":"Data and research analysis","publisher":"International Council for Education Quality Certification (ICEQC)","jurisdictional_scope":"International","historical_reference_basis":"Generative AI policy and practice","reference_authority":"United Nations Educational, Scientific and Cultural Organization","sections":[{"heading":null,"paragraphs":["The generative AI policy and practice provides the immediate reference point for consideration of teacher AI capability in 2023. In reviewing the analysis, comparable indicators can support public decision-making, but they do not remove the need to examine variation within systems and institutions. The decision should address both public impact and the responsibilities attached to entrusted educational resources. System context should determine the appropriate administrative arrangement within the governing requirements.","The historical reference basis is the generative AI policy and practice. Its relevance to the available evidence should be assessed against the affected jurisdiction, learner population and form of provision.","International guidance on generative artificial intelligence in education and research was released in September 2023. It calls for a human-centred approach, protection of data privacy, age-appropriate use, validation and institutional capacity. Immediate provider controls should address authorised uses, assessment, disclosure, information security, unequal access and human review while evidence on educational benefit and risk continues to develop.","Implementation of the measure should be organised around a decision that can be tested. Oversight of the comparison should reflect the principle that reported averages should be accompanied by sufficient distributional information to identify material differences between learner groups, locations and forms of provision. In relation to measuring teacher AI capability across education systems, resources and activity should be reconciled with the operating evidence and result for which the responsible function is accountable."]},{"heading":"Evidence base for measuring teacher AI capability across education systems","paragraphs":["The quality significance of teacher AI capability follows from a basic distinction between availability and effective provision. In reviewing the issue, technology may support teaching, administration and access, but consequential educational decisions must remain accountable, explainable and open to effective review. Review should cover the stages at which learners receive information, provision, assessment, support and remedy.","The technical issue within the analysis concerns the basis on which a conclusion is reached. In reviewing the issue, the subject should be examined as a connected system of policy, people, resources, decisions and evidence. The assessment should follow authority and information across functional boundaries and verify completion of required action. A conclusion should identify both its evidential basis and the part of the stated scope for which assurance cannot be given.","A narrow control over the available evidence may create false assurance. In the present context, opaque use of personal or inferred data, unverified outputs entering teaching or assessment and loss of meaningful human review may produce acceptable aggregate reporting while individual learners remain exposed to material disadvantage. Adverse cases should form part of the sample wherever they may reveal a material control weakness.","The evidential record for the measure should permit a reviewer to trace the matter from decision to outcome. This may require learner information and accessible challenge routes, an inventory of systems and their intended uses, supplier change and incident records, and pre-deployment and periodic performance testing, supported by documented authority for each consequential use and data provenance and access controls. Sampling remains insufficient where it excludes a material group or cannot resolve contradictory evidence or recurrence."]},{"heading":"Coverage and comparability","paragraphs":["For operational review of teacher AI capability, authorities and providers should proceed in a defined sequence. For the comparison, the reviewer should map the complete process, identify the intended result and responsible authority at each stage, and test normal cases together with exceptions. Findings should state the affected scope and required action; an observation should not be represented as evidence of conformity or effectiveness.","The analytical record for the available evidence should state the research question, data source, unit of analysis, reference period, coverage, exclusions, treatment of missing values and principal limitations.","In relation to measuring teacher AI capability across education systems, the evidential trail should allow an affected decision to be identified, examined and corrected. For the comparison, the responsible body should be able to identify the evidence considered, the judgement made, the person or body authorised to make it and the action that followed. Historical decisions should be assessed against the information then available, with later amendments separately dated and explained."]},{"heading":"Responsible interpretation","paragraphs":["Interpretation of teacher AI capability should avoid two errors: treating a formal commitment as proof of effect, and treating one adverse case as proof that every part of the system has failed. Oversight of the comparison should reflect the principle that a technical capability is not evidence that a use is educationally justified. Accuracy measured in one setting may not transfer to another population, language, curriculum or decision context. In reviewing the comparison, missing or delayed information may be patterned rather than random.","Accountability for the analysis should follow decision-making authority.","A complete conclusion on the analysis requires evidence extending beyond an individual measure or safeguard."]}],"word_count":766,"content_hash":"sha256-8ef4c0f78c786bc9a3838023d5b15174c49345f545d4fbdb7df25463709f31fc","seo_keywords":["measuring teacher AI capability across education systems","international education data research","education quality","ICEQC"],"schema_type":"AnalysisNewsArticle"}