{"schema_version":"ICEQC_NEWS_ARTICLE_V1","id":"iceqc-news-b596823d84b096eb","slug":"2023-01-16-ai-supported-assessment-evidence-coverage-and-limitations","language":"en","publication_status":"READY_FOR_IMPORT","publication_date":"2023-01-16","last_modified_date":"2023-01-16","title":"AI-supported assessment: evidence, coverage and limitations","summary":"Assesses evidence on aI-supported assessment: evidence, coverage and limitations, with particular attention to definitions, population coverage, comparability and uncertainty.","category":{"code":"DATA_RESEARCH","label":"Data Research"},"article_type":"Data and research analysis","publisher":"International Council for Education Quality Certification (ICEQC)","jurisdictional_scope":"International","historical_reference_basis":"Rapid adoption of generative AI tools","reference_authority":"Relevant public authorities and official international sources","sections":[{"heading":null,"paragraphs":["In 2023, consideration of AI-supported assessment must take account of the rapid adoption of generative AI tools and the responsibilities it places before education systems. Oversight of the measure should reflect the principle that comparable indicators can support public decision-making, but they do not remove the need to examine variation within systems and institutions.","The historical reference basis is the rapid adoption of generative AI tools. Its relevance to the available evidence should be assessed against the affected jurisdiction, learner population and form of provision.","Responsibility for the comparison should be visible at the point where consequential decisions are made. A decision concerning the measure should recognise that trend claims require comparable observations over time and a documented account of revisions, breaks in series and changes in coverage. In relation to aI-supported assessment, a decision should not be closed at the operating level where material impact, conflict or a significant evidential gap remains unresolved."]},{"heading":"Evidence and method","paragraphs":["The quality significance of AI-supported assessment follows from a basic distinction between availability and effective provision. A decision concerning the available evidence should recognise that assessment should provide valid and sufficiently consistent evidence that the stated learning outcomes have been achieved by the learner receiving the result.","In practical terms, The comparison should be reviewed against a stated method rather than general assurance. Oversight of the analysis should reflect the principle that data quality comprises accuracy, completeness, timeliness, consistency and traceability. Strength in one dimension does not compensate automatically for weakness in another, particularly where the information informs a consequential learner decision. Decision-makers should receive an intelligible account of how the result was reached and where it should not be applied.","A narrow control over the available evidence may create false assurance. In the present context, tasks that do not assess the stated outcome, uncontrolled changes to assessment and inconsistent judgement between markers or locations may produce acceptable aggregate reporting while individual learners remain exposed to material disadvantage. Adverse cases should form part of the sample wherever they may reveal a material control weakness.","The evidential record for the comparison should permit a reviewer to trace the matter from decision to outcome. This may require approval and change-control records, authorship and identity controls proportionate to risk, marking criteria and calibrated judgement, and appeal and correction records, supported by assessment maps to learning outcomes and analysis of results and differential outcomes. Sampling remains insufficient where it excludes a material group or cannot resolve contradictory evidence or recurrence."]},{"heading":"Patterns requiring examination","paragraphs":["The review method for AI-supported assessment should be reproducible. Review of the issue should trace selected records to source, reconcile totals across systems, quantify missing and late submissions, review manual adjustments and retain a revision history. Escalate discrepancies that could alter a published conclusion or individual outcome. The retained analysis should be reproducible from the selected evidence, decision rule and recorded reasons for accepted exceptions.","Publication of findings on the issue should distinguish observed values, estimates and interpretation.","Decisions concerning the analysis should remain traceable to the information available for the stated reference period. In relation to aI-supported assessment, changes in condition, evidence, method and interpretation should be recorded separately when a conclusion is revised. Without this distinction, a reporting change may be mistaken for improvement or deterioration in educational practice."]},{"heading":"Implications for decision-makers","paragraphs":["The analysis of AI-supported assessment should remain within the limits of the evidence. In reviewing the issue, a single indicator rarely provides an adequate account of quality. Quantitative evidence should be considered with implementation records and the experience of affected learners. For The comparison, reliability without validity produces consistent but potentially irrelevant results. Validity without adequate consistency may expose learners to unequal judgement. If uncertainty could change a consequential decision, additional evidence or a narrower conclusion is required.","Where The comparison involves partners, suppliers or several public bodies, responsibility should be mapped across the complete service. Contractual or inter-agency arrangements should identify who holds records, informs learners and acts on incidents. Multiple delivery partners do not justify fragmented accountability or remedy.","Assurance concerning the available evidence requires corroborating evidence across the material scope."]}],"word_count":674,"content_hash":"sha256-010345ebab6f4f9440cac924b63710089cf132549ffb9dd19d0ff18bf35502de","seo_keywords":["aI-supported assessment","international education data research","education quality","ICEQC"],"schema_type":"AnalysisNewsArticle"}