{"schema_version":"ICEQC_NEWS_ARTICLE_V1","id":"iceqc-news-4341f4421ac84290","slug":"2023-03-20-ai-supported-assessment-an-evidence-led-improvement-method","language":"en","publication_status":"READY_FOR_IMPORT","publication_date":"2023-03-20","last_modified_date":"2023-03-20","title":"AI-supported assessment: an evidence-led improvement method","summary":"Describes how improvement in aI-supported assessment: an evidence-led improvement method can be planned, measured and maintained through clear responsibility and follow-up.","category":{"code":"QUALITY_IMPROVEMENT_METHODS","label":"Quality Improvement Methods"},"article_type":"Quality improvement method","publisher":"International Council for Education Quality Certification (ICEQC)","jurisdictional_scope":"International","historical_reference_basis":"Rapid adoption of generative AI tools","reference_authority":"Relevant public authorities and official international sources","sections":[{"heading":null,"paragraphs":["Current consideration of AI-supported assessment is informed by the rapid adoption of generative AI tools, with consequences for governance, evidence and the treatment of affected learners. In reviewing The corrective programme, improvement should begin with a defined problem, a credible account of its causes and a measure capable of showing whether the response has worked. The unit of review should correspond to the full reach of the decision, including significant differences in provision and population."]},{"heading":"Improvement objective and baseline","paragraphs":["The stated reference—the rapid adoption of generative AI tools—establishes the contemporaneous context. Any conclusion about AI-supported assessment still requires evidence from the setting concerned. The decision basis should identify what is evidenced, what reflects policy and what depends on authorised discretion. In relation to aI-supported assessment, later review should not obscure whether the earlier position rested on fact, policy or judgement.","The governing expectation for the practice should be capable of consistent application. In reviewing the improvement objective, follow-up should determine whether the change is embedded in ordinary operations and whether it has created new risks or unequal effects. Criteria affecting learners should not permit materially different interpretation without an evidenced reason.","The principal risks in relation to the corrective programme are reasonable adjustment altering the assessed outcome, tasks that do not assess the stated outcome, uncontrolled changes to assessment, and results used beyond the evidence they support. The risks are interdependent; failure of one control may conceal or disable another.","The analysis of the intervention should make its decision rule explicit. In reviewing the issue under review, the subject should be examined as a connected system of policy, people, resources, decisions and evidence. Gaps may emerge when authority, records or action pass between responsible bodies. A stated decision rule enables comparable examination and limits retrospective explanations of adverse evidence.","Evidence should be selected against a clearly defined question. For The corrective programme, the most relevant material is likely to include approval and change-control records, moderation and exception records, marking criteria and calibrated judgement, and assessment maps to learning outcomes. In relation to aI-supported assessment, independent records should be reconciled, with disagreement and uncertainty reported alongside the finding."]},{"heading":"Controls and accountable action","paragraphs":["The analysis of AI-supported assessment should remain within the limits of the evidence. Oversight of aI-supported assessment should reflect the principle that a short-term increase in activity may not represent sustained improvement. Measures should remain in place long enough to detect recurrence and unintended effects. For The corrective programme, reliability without validity produces consistent but potentially irrelevant results. Validity without adequate consistency may expose learners to unequal judgement. Material uncertainty should result in further enquiry or an expressly limited finding.","The evidential trail should allow an affected decision to be identified, examined and corrected. For the issue under review, the responsible body should be able to identify the evidence considered, the judgement made, the person or body authorised to make it and the action that followed. Historical decisions should be assessed against the information then available, with later amendments separately dated and explained."],"bullets":["Align tasks and criteria with learning outcomes.","Calibrate assessors.","Control changes, with responsibility, scope and timing recorded.","Retain evidence sufficient for review.","Moderate material variation."]},{"heading":"Evidence of effect","paragraphs":["A proportionate method is available for AI-supported assessment. The method for the issue under review is to map the complete process, identify the intended result and responsible authority at each stage, and test normal cases together with exceptions. Contrary evidence should not be removed merely because aggregate performance appears acceptable.","A decision to close improvement work on the intervention should be made by a person with authority and sufficient independence from implementation."],"bullets":["Who controls each stage?","What outcome is intended?","Where do exceptions occur?","What action is required by the finding?","Which evidence establishes operation?"]},{"heading":"Sustaining improvement","paragraphs":["Public reporting on AI-supported assessment should distinguish established fact, analytical judgement and planned action. Material revisions should be traceable to their reason and effective date. Changes to definitions or evidence should be recorded separately from changes in educational performance.","The central objective should not be obscured by the form of the administrative response. In reviewing the improvement objective, assessment should provide valid and sufficiently consistent evidence that the stated learning outcomes have been achieved by the learner receiving the result. Inputs and formal commitments should be distinguished from demonstrated operation and outcome.","Authorities and providers should use the current development to test whether the improvement objective connects public commitment with effective operation and evidence of result. Improvement should be supported by evidence and an accountable decision record capable of public scrutiny."]}],"word_count":749,"content_hash":"sha256-30fdec83428dca0a7d3726a15c1702058bcbd4cabefc802bd21042f7f34faef5","seo_keywords":["aI-supported assessment","education quality improvement","education quality","ICEQC"],"schema_type":"TechArticle"}