{"schema_version":"ICEQC_NEWS_ARTICLE_V1","id":"iceqc-news-c83846ebf934b0b4","slug":"2010-11-15-quality-controls-for-learning-outcome-measurement","language":"en","publication_status":"READY_FOR_IMPORT","publication_date":"2010-11-15","last_modified_date":"2010-11-15","title":"Quality controls for learning-outcome measurement","summary":"Sets out a controlled approach to quality controls for learning-outcome measurement, linking diagnosis, accountable action, outcome measures and verification of sustained.","category":{"code":"QUALITY_IMPROVEMENT_METHODS","label":"Quality Improvement Methods"},"article_type":"Quality improvement method","publisher":"International Council for Education Quality Certification (ICEQC)","jurisdictional_scope":"International","historical_reference_basis":"International assessment and indicator development","reference_authority":"Relevant public authorities and official international sources","sections":[{"heading":null,"paragraphs":["In 2010, consideration of learning-outcome measurement must take account of the international assessment and indicator development and the responsibilities it places before education systems. Oversight of the intervention should reflect the principle that the method set out here treats improvement as a controlled cycle of diagnosis, action, measurement and review. The control response should be sufficient to protect learners while avoiding burdens not justified by the evidence.","The historical reference basis is the international assessment and indicator development. Its relevance to the corrective programme should be assessed against the affected jurisdiction, learner population and form of provision.","Responsibility for the improvement objective should be visible at the point where consequential decisions are made. A decision concerning the corrective programme should recognise that the intervention should be tested on a scale proportionate to the risk before wider implementation, unless immediate system-wide action is necessary to protect learners. In relation to quality controls for learning-outcome measurement, a decision should not be closed at the operating level where material impact, conflict or a significant evidential gap remains unresolved."]},{"heading":"Defining the problem","paragraphs":["The system and institutional dimensions of learning-outcome measurement should be considered together. Oversight of the practice should reflect the principle that education indicators should support decisions by describing outcomes and variation with definitions and limitations that permit responsible interpretation. The regulatory setting is determined by public authorities, but responsibility for controlled provision remains with the provider.","In practical terms, the corrective programme should be reviewed against a stated method rather than general assurance. Oversight of the corrective programme should reflect the principle that effectiveness is the demonstrated change in the condition the action was intended to address. Completion of training, publication of guidance or installation of a system is an output and should not be reported as an outcome without further evidence.","Failure in relation to the improvement objective may arise even where the stated policy is reasonable. Material concerns include data revisions not carried through to published conclusions, small differences overstated, averages concealing distribution, and incomplete coverage. An exception should be assessed by effect, duration, recurrence and reach, including possible exposure beyond the initial sample.","The evidential record for the intervention should permit a reviewer to trace the matter from decision to outcome. This may require disaggregated results, coverage and missingness analysis, revision and comparability records, and population and sampling information, supported by indicator definitions and metadata and uncertainty estimates where relevant. Conflicting records, absent populations and uncertain follow-through require additional testing."]},{"heading":"Improvement method","paragraphs":["For operational review of learning-outcome measurement, authorities and providers should proceed in a defined sequence. The method for the improvement objective is to set a baseline and success measure before intervention, define the review period, compare the result with the intended outcome and examine adverse or unequal effects. In relation to quality controls for learning-outcome measurement, continue monitoring long enough to determine whether the improvement is sustained. The record should distinguish a finding that requires action from an observation that supports no formal conclusion.","Improvement of the improvement objective should proceed through controlled tests where risk permits.","Records relating to the issue under review should preserve both the conclusion and its limits. New evidence should trigger a traceable correction and review of decisions materially affected by the earlier conclusion."]},{"heading":"Measures and review","paragraphs":["The analysis of learning-outcome measurement should remain within the limits of the evidence. Oversight of the corrective programme should reflect the principle that correcting an individual record does not establish that the process which produced the error has been corrected. In reviewing the practice, measurement can reveal where outcomes differ; it does not by itself establish why they differ or which intervention will work.","Where the issue under review involves partners, suppliers or several public bodies, responsibility should be mapped across the complete service. The division of responsibilities should cover records, communication, escalation and the power to require correction. Multiple delivery partners do not justify fragmented accountability or remedy.","Assessment of the corrective programme should reconcile more than one source of evidence and control. The final judgement should connect the applicable expectation to implementation and outcomes while identifying unresolved risk.","Effectiveness should be reassessed after the defined review period and against the position recorded under International assessment and indicator development. Later improvement does not retrospectively remove the original deficiency or the need for a traceable corrective record. The assessment should show whether benefit is sustained and whether material adverse effects remain."]}],"word_count":728,"content_hash":"sha256-a2aa64dc82c39378d143ca795520cb8731a7fc6a3ff0d10b2bbc231b8b65bf82","seo_keywords":["quality controls for learning-outcome measurement","education quality improvement","education quality","ICEQC"],"schema_type":"TechArticle"}