{"schema_version":"ICEQC_NEWS_ARTICLE_V1","id":"iceqc-news-1c0a69e8bbcdad81","slug":"2008-04-21-completion-and-retention-measurement-and-statistical-uncertainty","language":"en","publication_status":"READY_FOR_IMPORT","publication_date":"2008-04-21","last_modified_date":"2008-04-21","title":"Completion and retention: measurement and statistical uncertainty","summary":"Reviews completion and retention: measurement and statistical uncertainty and identifies the conditions required for sound interpretation, responsible comparison and defensible.","category":{"code":"DATA_RESEARCH","label":"Data Research"},"article_type":"Data and research analysis","publisher":"International Council for Education Quality Certification (ICEQC)","jurisdictional_scope":"International","historical_reference_basis":"2015 education target monitoring","reference_authority":"Relevant public authorities and official international sources","sections":[{"heading":null,"paragraphs":["The 2015 education target monitoring provides the immediate reference point for consideration of completion and retention in 2008. In reviewing the measure, the value of the present data lies in the questions it can answer reliably and in the limits it makes visible. The proportionality test should consider both the identified risk and the consequences of the control for affected learners."]},{"heading":"Analytical scope","paragraphs":["The historical reference basis is the 2015 education target monitoring. Its relevance to completion and retention should be assessed against the affected jurisdiction, learner population and form of provision.","A proper review of the analysis should establish the intended outcome before selecting controls or indicators. In reviewing the comparison, where an indicator is used as a proxy, the relationship between the proxy and the underlying educational outcome should be stated and tested. Suitability, authorised variation and the date for reconsideration should be established when the arrangement is approved.","Risk assessment of the analysis should give particular attention to aggregate gains concealing excluded groups, admission without adequate support, and attrition detected too late. A provider should also consider completion measures detached from learning and support allocated without evidence of need. Stronger controls are required where learners may not detect an error or where later correction cannot restore the lost opportunity.","The analysis of the comparison should make its decision rule explicit. Oversight of the measure should reflect the principle that an estimate should be reported with the uncertainty arising from sampling, measurement, missing information and model choice. A precise number may still support only a cautious conclusion if the underlying assumptions are weak. The method should prevent an unfavourable result from being dismissed through an unrecorded change in interpretation.","In relation to completion and retention, each source should have a stated purpose in supporting or limiting the conclusion. For the available evidence, the most relevant material is likely to include evaluation of interventions, clearly defined access and completion indicators, resource allocation by need, and progression and early-warning information. In relation to completion and retention, confidence is strengthened by corroboration, not by the volume of records drawn from the same underlying source."]},{"heading":"Definitions and data coverage","paragraphs":["The analysis of completion and retention should remain within the limits of the evidence. The analysis of the issue proceeds on the basis that a single indicator rarely provides an adequate account of quality. Quantitative evidence should be considered with implementation records and the experience of affected learners. Oversight of the issue should reflect the principle that equal treatment does not necessarily produce equitable access or outcomes. Different support may be justified where it addresses a documented barrier without changing the required educational outcome. If uncertainty could change a consequential decision, additional evidence or a narrower conclusion is required.","Records relating to the available evidence should preserve both the conclusion and its limits. The correction record should state what the new evidence changes and which earlier conclusions or decisions require review. This is material where learners, authorities or institutions relied on information that cannot be corrected by replacing the current text alone."],"bullets":["Analyse barriers across the learner journey.","Report limitations in available data before it informs a consequential decision.","Evaluate differential outcomes.","Monitor progression at useful intervals, identifying the accountable function and affected scope.","Identify groups not represented in aggregate results."]},{"heading":"Use of the findings","paragraphs":["For operational review of completion and retention, authorities and providers should proceed in a defined sequence. For the available evidence, the reviewer should report intervals or sensitivity ranges where appropriate, test whether the conclusion changes under plausible assumptions, and distinguish statistical detectability from educational significance. Record unresolved uncertainty beside the principal finding.","Decision-makers using evidence on the analysis should be told what the data cannot establish as clearly as what it can. Users should be able to distinguish descriptive, comparative and evaluative findings and understand their proper level of application. Application in another setting depends on a separate examination of context and comparability."],"bullets":["What are the principal sources of uncertainty?","Would a reasonable alternative assumption change the conclusion?","Are small groups adequately represented?","How should uncertainty affect the decision?","Is the apparent difference larger than the measurement error?"]},{"heading":"Uncertainty and safeguards","paragraphs":["Public reporting on completion and retention should distinguish established fact, analytical judgement and planned action. If definitions, coverage or evidence alter an earlier conclusion, the reason should be stated so that revision is not mistaken for changed performance.","The quality significance of the available evidence follows from a basic distinction between availability and effective provision. Oversight of the comparison should reflect the principle that education systems should examine not only who enters education, but who can participate effectively, progress and complete with the intended learning outcomes.","A complete conclusion on the issue requires evidence extending beyond an individual measure or safeguard. The final judgement should connect the applicable expectation to implementation and outcomes while identifying unresolved risk."]}],"word_count":797,"content_hash":"sha256-8c247566aee1c552e6a8b3a2471edd9ee58d5d1e6cf40e79f841926020b4b9f7","seo_keywords":["completion and retention","international education data research","education quality","ICEQC"],"schema_type":"AnalysisNewsArticle"}