{"schema_version":"ICEQC_NEWS_ARTICLE_V1","id":"iceqc-news-afa92bfdc21a4d4b","slug":"2006-10-23-using-root-cause-analysis-to-improve-multilingual-literacy-assessment","language":"en","publication_status":"READY_FOR_IMPORT","publication_date":"2006-10-23","last_modified_date":"2006-10-23","title":"Using root-cause analysis to improve multilingual literacy assessment","summary":"Sets out a controlled approach to using root-cause analysis to improve multilingual literacy assessment, linking diagnosis, accountable action, outcome measures.","category":{"code":"QUALITY_IMPROVEMENT_METHODS","label":"Quality Improvement Methods"},"article_type":"Quality improvement method","publisher":"International Council for Education Quality Certification (ICEQC)","jurisdictional_scope":"International","historical_reference_basis":"International literacy measurement guidance available in 2006","reference_authority":"Relevant public authorities and official international sources","sections":[{"heading":null,"paragraphs":["The policy and evidence context for root-cause analysis to improve multilingual literacy assessment has been materially shaped by the international literacy measurement guidance available in 2006. For the intervention, improvement should begin with a defined problem, a credible account of its causes and a measure capable of showing whether the response has worked.","The stated reference is the international literacy measurement guidance available in 2006. Application to the corrective programme depends on evidence from the relevant jurisdiction or institution. Verified fact, policy expectation and discretionary institutional choice should remain distinct in the record."]},{"heading":"Defining the problem","paragraphs":["In relation to using root-cause analysis to improve multilingual literacy assessment, the relevant outcome should be capable of direct and consistent explanation. Oversight of root-cause analysis to improve multilingual literacy assessment should reflect the principle that assessment should provide valid and sufficiently consistent evidence that the stated learning outcomes have been achieved by the learner receiving the result. Formal adoption, expenditure and activity do not in themselves establish the intended result. Implementation evidence should be sufficient to identify unequal consequences and assign corrective responsibility.","The technical issue within the corrective programme concerns the basis on which a conclusion is reached. A decision concerning the corrective programme should recognise that cause analysis should explain why the condition occurred and why existing controls did not prevent or detect it. Restating the event, assigning blame or identifying the last person involved does not establish a root cause. The decision record should distinguish the scope supported by evidence from any scope that remains unresolved.","The principal risks in relation to the corrective programme are inconsistent judgement between markers or locations, results used beyond the evidence they support, tasks that do not assess the stated outcome, and weak assurance of authorship or performance. Risk assessment should account for dependencies between controls and the possibility that one failure masks the next."]},{"heading":"Improvement method","paragraphs":["The evidential record for root-cause analysis to improve multilingual literacy assessment should permit a reviewer to trace the matter from decision to outcome. This may require analysis of results and differential outcomes, authorship and identity controls proportionate to risk, moderation and exception records, and appeal and correction records, supported by marking criteria and calibrated judgement and assessment maps to learning outcomes. Conflicting records, absent populations and uncertain follow-through require additional testing.","The assurance record for the intervention should retain the date of the evidence, the source responsible for it, the scope examined and the version of any instrument or definition applied. In relation to using root-cause analysis to improve multilingual literacy assessment, the evidential history should preserve conclusions that were operative when a material decision was made."],"bullets":["Moderate material variation.","Retain evidence sufficient for review.","Define the decision each assessment must support.","Calibrate assessors before using it to determine a learner or provider outcome.","Align tasks and criteria with learning outcomes."]},{"heading":"Measures and review","paragraphs":["Responsibility for root-cause analysis to improve multilingual literacy assessment should be visible at the point where consequential decisions are made. The analysis of the practice proceeds on the basis that the intervention should be tested on a scale proportionate to the risk before wider implementation, unless immediate system-wide action is necessary to protect learners. Incomplete evidence, unmanaged conflict, absent learner groups or material learner impact require a higher level of review.","For operational review of the corrective programme, authorities and providers should proceed in a defined sequence. Review of the intervention should reconstruct the sequence, test contributing conditions, compare affected and unaffected cases, and identify the control failure that can be changed. Confirm the explanation against evidence before selecting corrective action. Findings should state the affected scope and required action; an observation should not be represented as evidence of conformity or effectiveness.","The improvement record for the corrective programme should contain the verified problem, affected scope, immediate containment, causal analysis, selected intervention, accountable owner, resources, milestones and effectiveness measure. The action record should separate administrative completion from verification of the intended change. Closure reporting should not obscure unresolved action or risk retained by the responsible authority.","For the corrective programme, governing bodies should receive a concise account of the intended result, affected scope, principal risks, evidence limitations and unresolved exceptions. The action record should identify who is responsible and when implementation is due."]},{"heading":"Residual risk and follow-up","paragraphs":["The analysis of root-cause analysis to improve multilingual literacy assessment should remain within the limits of the evidence. The analysis of the improvement objective proceeds on the basis that a short-term increase in activity may not represent sustained improvement. Measures should remain in place long enough to detect recurrence and unintended effects. In reviewing the practice, reliability without validity produces consistent but potentially irrelevant results. Validity without adequate consistency may expose learners to unequal judgement.","The measure of progress on the intervention is not the amount of policy or documentation produced."]}],"word_count":793,"content_hash":"sha256-92a9bfd0f3d8a952e63f32dbd35356bcb71ba1ef0a20c40a3126c092434be753","seo_keywords":["using root-cause analysis to improve multilingual literacy assessment","education quality improvement","education quality","ICEQC"],"schema_type":"TechArticle"}