{"schema_version":"ICEQC_NEWS_ARTICLE_V1","id":"iceqc-news-fe89b79ec117fa12","slug":"2019-12-06-pisa-2018-performance-equity-and-responsible-international-comparison","language":"en","publication_status":"READY_FOR_IMPORT","publication_date":"2019-12-06","last_modified_date":"2019-12-06","title":"PISA 2018: performance, equity and responsible international comparison","summary":"Reviews pISA 2018: performance, equity and responsible international comparison and identifies the conditions required for sound interpretation, responsible comparison.","category":{"code":"DATA_RESEARCH","label":"Data Research"},"article_type":"Data and research analysis","publisher":"International Council for Education Quality Certification (ICEQC)","jurisdictional_scope":"International","historical_reference_basis":"PISA 2018 results released on 3 December 2019","reference_authority":"Organisation for Economic Co-operation and Development","sections":[{"heading":null,"paragraphs":["At the publication date, PISA 2018 results released on 3 December 2019 provides the relevant international context for the measure. Any consequential application still requires evidence from the affected jurisdiction or institution. Oversight of the measure should reflect the principle that evidence should inform action without implying a level of precision, coverage or causal certainty that the underlying data cannot support. Data used for PISA 2018 should be interpreted against stable definitions and an identifiable population. A revision or break in series should not be reported as a change in performance.","The contemporaneous reference point for the analysis is PISA 2018 results released on 3 December 2019. Its status should be distinguished from the jurisdiction-specific evidence required for implementation. The level of inference should not extend beyond the level at which the source data are valid. The evidential basis for the measure should identify source, period, coverage and material limitations. Corroboration is required where a single record cannot support the decision. Data used for the measure should be interpreted against stable definitions and an identifiable population. Reporting should identify a break in comparability before describing movement over time."]},{"heading":"Evidence base for pISA 2018","paragraphs":["Consideration of PISA 2018 should retain the date and status of PISA 2018 results released on 3 December 2019. Later developments should not be read into the position available at publication. Analysis of the available evidence should state the unit of analysis, reference period, coverage, exclusions and treatment of missing information. Material differences in population, setting or method should remain explicit in any comparison. Any indicator used in relation to the available evidence should distinguish description from causal explanation. The result should be accompanied by distributional information, uncertainty and limits on transfer to another setting.","Analysis of the measure should state the unit of analysis, reference period, coverage, exclusions and treatment of missing information. Comparative findings should not conceal differences capable of changing their meaning. In relation to pISA 2018, review should cover the stages at which learners receive information, provision, assessment, support and remedy.","Assurance should not overlook failures arising at the boundary between otherwise adequate controls. A formally complete record is not reliable if its scope or measure does not correspond to the decision being made.","Responsibility for the issue should be visible at the point where consequential decisions are made. Data used for the available evidence should be interpreted against stable definitions and an identifiable population. Apparent movement caused by revision should not be attributed to educational performance.","Assurance of the analysis should draw on more than one form of evidence. A comparison is reliable only if material differences remain visible. A positive example may illustrate operation, but it cannot demonstrate coverage or consistency."],"bullets":["Review policies that create avoidable exclusion, identifying the accountable function and affected scope.","Analyse barriers across the learner journey.","Target support transparently before it informs a consequential decision.","Report limitations in available data.","Evaluate differential outcomes."]},{"heading":"Coverage and comparability","paragraphs":["Risk assessment of PISA 2018 should give particular attention to financial or geographic barriers, completion measures detached from learning, and admission without adequate support. The position on the analysis should be established through proportionate evidence and should remain open to correction when material new information becomes available.","The review method for the available evidence should be reproducible. Review of the analysis should give particular attention to adverse cases, unequal effects and errors that learners may be unable to identify or remedy after the event. Repeated or connected failures should be addressed through the relevant system control, not only through individual remedies. Risk assessment for the available evidence should consider severity, reach, duration, recurrence and detectability, with escalation where learner impact may be material.","Decision-makers using evidence on the comparison should be told what the data cannot establish as clearly as what it can. The conclusion should state whether the result is descriptive, comparative or evaluative, the issue whether it applies to a system, institution, programme or learner group. Analysis of the analysis should state the unit of analysis, reference period, coverage, exclusions and treatment of missing information.","Proportionality in relation to the issue does not mean reduced protection for learners exposed to greater risk. In reviewing the measure, equal treatment does not necessarily produce equitable access or outcomes. Different support may be justified where it addresses a documented barrier without changing the required educational outcome. Review of the measure should give particular attention to adverse cases, unequal effects and errors that learners may be unable to identify or remedy after the event.","Decisions concerning the analysis should remain traceable to the information available for the stated reference period. In relation to pISA 2018, the reason for revision should be explicit, including whether it arises from new evidence, a methodological change or a different interpretation.","The principal risks associated with the analysis should be assessed as connected conditions. Governance of the available evidence requires a clear allocation of authority, information and follow-through. Material matters should be referred to the body authorised to act or accept residual risk. Closure requires evidence that the condition has changed; completion of planned activity is not sufficient.","Data used for the analysis should be interpreted against stable definitions and an identifiable population."]}],"word_count":857,"content_hash":"sha256-a6d915eff773a53bcdcc9a1d950b9b12ce679123c062d4913bf3f45dda0b6c11","seo_keywords":["pISA 2018","international education data research","education quality","ICEQC"],"schema_type":"AnalysisNewsArticle"}