Sets out a method for using root-cause analysis to improve multilingual literacy assessment, covering diagnosis, responsible action, outcome evidence.
The policy and evidence context for root-cause analysis to improve multilingual literacy assessment has been materially shaped by the international literacy measurement guidance available in 2006. For the corrective action, improvement should begin with a defined problem, a credible account of its causes and a measure capable of showing whether the response has worked.
The stated reference is the international literacy measurement guidance available in 2006. Application to corrective action depends on evidence from the relevant jurisdiction or institution. Verified fact, policy expectation and discretionary institutional choice should remain distinct in the record.
Defining the problem
For multilingual literacy assessment, the relevant outcome should be capable of direct and consistent explanation. Assessment should provide valid and sufficiently consistent evidence that the stated learning outcomes have been achieved by the learner receiving the result. Formal adoption, expenditure and activity do not in themselves establish the intended result. Implementation evidence should be sufficient to identify unequal consequences and assign corrective responsibility.
Cause analysis should explain why the condition occurred and why existing controls did not prevent or detect it. Restating the event, assigning blame or identifying the last person involved does not establish a root cause. The decision record for multilingual literacy assessment should distinguish the scope supported by evidence from any scope that remains unresolved.
The principal risks in relation to corrective action are inconsistent judgement between markers or locations, results used beyond the evidence they support, tasks that do not assess the stated outcome, and weak assurance of authorship or performance. Risk assessment should account for dependencies between controls and the possibility that one failure masks the next.
Improvement method
The evidential record for root-cause analysis to improve multilingual literacy assessment should permit a reviewer to trace the matter from decision to outcome. This may require analysis of results and differential outcomes, authorship and identity controls proportionate to risk, moderation and exception records, and appeal and correction records, supported by marking criteria and calibrated judgement and assessment maps to learning outcomes. Conflicting records, absent populations and uncertain follow-through require additional testing.
The assurance record for the corrective action should retain the date of the evidence, the source responsible for it, the scope examined and the version of any instrument or definition applied. As regards multilingual literacy assessment, the evidential history should preserve conclusions that were operative when a material decision was made.
- Moderate material variation.
- Retain evidence sufficient for review.
- Define the decision each assessment must support.
- Calibrate assessors before using it to determine a learner or provider outcome.
- Align tasks and criteria with learning outcomes.
Measures and review
Within the scope under review, responsibility should be identifiable at the point where consequential decisions are made. For multilingual literacy assessment, the corrective action should be tested on a scale proportionate to the risk before wider implementation, unless immediate system-wide action is necessary to protect learners. Incomplete evidence, unmanaged conflict, absent learner groups or material learner impact require a higher level of review.
Authorities and providers reviewing corrective action should proceed in a defined sequence. Review of the corrective action should reconstruct the sequence, test contributing conditions, compare affected and unaffected cases, and identify the control failure that can be changed. Confirm the explanation against evidence before selecting corrective action. When examining multilingual literacy assessment, findings should state the affected scope and required action; an observation should not be represented as evidence of conformity or effectiveness.
In the context of multilingual literacy assessment, the improvement record for corrective action should contain the verified problem, affected scope, immediate containment, causal analysis, selected intervention, accountable owner, resources, milestones and effectiveness measure. The action record should separate administrative completion from verification of the intended change. Closure reporting should not obscure unresolved action or risk retained by the responsible authority.
When examining multilingual literacy assessment, for corrective action, governing bodies should receive a concise account of the intended result, affected scope, principal risks, evidence limitations and unresolved exceptions. The action record should identify who is responsible and when implementation is due.
Residual risk and follow-up
The analysis of root-cause analysis to improve multilingual literacy assessment should remain within the limits of the evidence. A short-term increase in activity may not represent sustained improvement. Measures should remain in place long enough to detect recurrence and unintended effects. Reliability without validity produces consistent but potentially irrelevant results. Validity without adequate consistency may expose learners to unequal judgement.
As regards multilingual literacy assessment, progress should not be assessed by the amount of policy or documentation produced.