Review of PISA 2012 addresses the unit of analysis, source definitions, missing data and transfer beyond the reported setting.
The contemporaneous reference point for PISA 2012 is PISA 2012 results released on 3 December 2013. Any consequential application still requires evidence from the affected jurisdiction or institution. A national or international pattern may justify closer review of the comparison, but provider-level action requires evidence relating to the affected provision.
The review should include the experience of affected learners, particularly where aggregate reporting may conceal exclusion, delay or unequal treatment. The data permit examination of performance distributions and differences within and between participating systems. Results on engagement or self-belief should not be treated as interchangeable with demonstrated attainment; each construct has its own measures and limitations.
Application to pISA 2012
For PISA 2012, the public interest is not confined to institutional compliance. For the measure, education systems should examine not only who enters education, but who can participate effectively, progress and complete with the intended learning outcomes.
In the context of pISA 2012, the subject should be examined as a connected system of policy, people, resources, decisions and evidence.
A narrow control over the analysis may create false assurance. In the present context, financial or geographic barriers, admission without adequate support and attrition detected too late may produce acceptable aggregate reporting while individual learners remain exposed to material disadvantage. When examining pISA 2012, a sample confined to compliant cases cannot establish the reliability of the control.
Controls for pISA 2012
Assurance of PISA 2012 should draw on more than one form of evidence. Useful records include progression and early-warning information, records of barriers and support, disaggregated participation and outcome data, resource allocation by need, and clearly defined access and completion indicators.
The assurance record for the available evidence should retain the date of the evidence, the source responsible for it, the scope examined and the version of any instrument or definition applied. For decisions concerning pISA 2012, revision should not remove an earlier conclusion from the record where reliance has occurred.
- Identify groups not represented in aggregate results.
- Analyse barriers across the learner journey.
- Monitor progression at useful intervals.
- Report limitations in available data.
- Review policies that create avoidable exclusion before it informs a consequential decision.
Review of pISA 2012
For operational review, authorities and providers should proceed in a defined sequence. Across the defined scope, responsible bodies should map the complete process, identify the intended result and responsible authority at each stage, and test normal cases together with exceptions. For pISA 2012, an isolated incident and a recurring or systemic condition require different findings and responses.
The analytical record for pISA 2012 should state the research question, data source, unit of analysis, reference period, coverage, exclusions, treatment of missing values and principal limitations.
Public reporting on pISA 2012 should distinguish established fact, analytical judgement and planned action.
Implications for pISA 2012
The analysis of PISA 2012 should remain within the limits of the evidence. For the available evidence, equal treatment does not necessarily produce equitable access or outcomes.