{"schema_version":"ICEQC_NEWS_ARTICLE_V1","id":"iceqc-news-a7ff39a483631275","slug":"2018-02-02-equity-analysis-of-learning-outcome-shortfalls","language":"en","publication_status":"READY_FOR_IMPORT","publication_date":"2018-02-02","last_modified_date":"2018-02-02","title":"Equity analysis of learning outcome shortfalls","summary":"Reviews equity analysis of learning outcome shortfalls and identifies the conditions required for sound interpretation, responsible comparison and defensible conclusions.","category":{"code":"DATA_RESEARCH","label":"Data Research"},"article_type":"Data and research analysis","publisher":"International Council for Education Quality Certification (ICEQC)","jurisdictional_scope":"International","historical_reference_basis":"World Development Report 2018","reference_authority":"World Bank","sections":[{"heading":null,"paragraphs":["Current consideration of learning outcome shortfalls is informed by the World Development Report 2018, with consequences for governance, evidence and the treatment of affected learners. In this case, the principal analytical task is to separate an observed difference from a conclusion about its cause.","The reference basis—the World Development Report 2018—is evidential rather than self-executing. The material may reveal patterns or evidential gaps, but it neither directs a legal outcome nor establishes causation. In applying it to the measure, users should review the source definitions, population coverage, reference period and stated limitations before transferring a system-level finding to an individual provider or learner group.","The World Development Report 2018, Learning to Realize Education’s Promise, distinguishes schooling from learning and describes poor outcomes, immediate barriers within teaching and learning, and deeper system-level causes. It proposes three connected directions: assess learning, act on evidence and align actors around learning. Attendance and completion remain important, but they do not provide direct evidence that learners have acquired the intended knowledge and skills.","For the measure, the public interest is not confined to institutional compliance. Oversight of the analysis should reflect the principle that assessment should provide valid and sufficiently consistent evidence that the stated learning outcomes have been achieved by the learner receiving the result. In relation to equity analysis of learning outcome shortfalls, where learners rely on published information or support decisions, errors should be identifiable and capable of prompt, fair correction."]},{"heading":"Analytical scope","paragraphs":["The analysis of learning outcome shortfalls should make its decision rule explicit. The analysis of the available evidence proceeds on the basis that an average may improve while a material group experiences no improvement or a worse outcome. Disaggregation should follow a defined public-interest question and should protect confidentiality where small numbers could identify individuals. This supports consistent review and reduces the risk of redefining the basis of judgement after an adverse result appears.","The governing expectation for the analysis should be capable of consistent application. For the analysis, trend claims require comparable observations over time and a documented account of revisions, breaks in series and changes in coverage. Criteria affecting learners should not permit materially different interpretation without an evidenced reason."]},{"heading":"Definitions and data coverage","paragraphs":["The principal risks in relation to learning outcome shortfalls are tasks that do not assess the stated outcome, uncontrolled changes to assessment, reasonable adjustment altering the assessed outcome, and results used beyond the evidence they support. The risks are connected, and failure of one safeguard may disable or conceal another."],"bullets":["Moderate material variation before it informs a consequential decision.","Calibrate assessors.","Define the decision each assessment must support.","Control changes.","Review differential and anomalous results."]},{"heading":"Use of the findings","paragraphs":["Relevant evidence for learning outcome shortfalls will normally include analysis of results and differential outcomes, authorship and identity controls proportionate to risk, moderation and exception records, marking criteria and calibrated judgement, and appeal and correction records. In relation to equity analysis of learning outcome shortfalls, the conclusion should rely on evidence whose date, source and coverage are sufficient for the decision. Conflicting records require reconciliation before a complete assurance conclusion is reached.","Implementation of the measure can be tested without imposing unnecessary reporting. For the analysis, the reviewer should examine results by relevant learner, programme, location and delivery characteristics; compare both levels and rates of change; and test whether observed gaps persist after differences in coverage and prior conditions are considered. Existing records may be used if reliable and relevant, but data collected for another purpose may not answer the assurance question.","Publication of findings on the available evidence should distinguish observed values, estimates and interpretation."]},{"heading":"Uncertainty and safeguards","paragraphs":["Interpretation of learning outcome shortfalls should avoid two errors: treating a formal commitment as proof of effect, and treating one adverse case as proof that every part of the system has failed. In reviewing the measure, reliability without validity produces consistent but potentially irrelevant results. Validity without adequate consistency may expose learners to unequal judgement. A decision concerning the measure should recognise that a single indicator rarely provides an adequate account of quality. Quantitative evidence should be considered with implementation records and the experience of affected learners.","Accountability and effective correction both depend on a record that can be followed from evidence to decision. For the analysis, the responsible body should be able to identify the evidence considered, the judgement made, the person or body authorised to make it and the action that followed. Material changes require a traceable effective date and explanation so that prior reliance can be reviewed fairly.","For the measure, governing bodies should receive a concise account of the intended result, affected scope, principal risks, evidence limitations and unresolved exceptions. Material action requires a named responsible function and a defined completion point. Evidence of outcome, rather than completion of tasks, should determine whether corrective work can close.","The measure of progress on the comparison is not the amount of policy or documentation produced."]}],"word_count":811,"content_hash":"sha256-7daf5fb255453d43fbb2a3a93dd06197d9c840b98363b25ca4c708bb305a0dec","seo_keywords":["equity analysis of learning outcome shortfalls","international education data research","education quality","ICEQC"],"schema_type":"AnalysisNewsArticle"}