The reliability of evidence on responsible education data use is examined together with the limits that apply when findings inform consequential decisions.
The expanding public reporting and comparative indicators provide the immediate reference point for consideration of responsible education data use in 2017. In this case, the available evidence should be interpreted with close attention to definitions, population coverage, collection methods and the limits of comparison.
Evidence base for responsible education data use
For responsible education data use, the required public outcome should be stated in operational terms.
- Define the decision the indicator will inform.
- Avoid causal claims unsupported by the design.
- Analyse missing information.
- Test comparability.
- Document numerator and denominator.
Controls for responsible education data use
For decisions concerning responsible education data use, the reference basis—the expanding public reporting and comparative indicators—is evidential rather than self-executing. The material may reveal patterns or evidential gaps, but it neither directs a legal outcome nor establishes causation.
Review of responsible education data use
The principal risks in relation to the available evidence are changes in definition presented as changes in performance, proxy measures treated as direct outcomes, incomplete coverage, and small differences overstated. In the context of responsible education data use, the control environment should be assessed as a connected system rather than as unrelated individual risks. Across the defined scope, review should follow the sequence of decisions and records rather than assess documents in isolation.
- What action is required by the finding?
- Where do exceptions occur?
- What outcome is intended?
- Which evidence establishes operation?
- Who controls each stage?
Implications for responsible education data use
This may require uncertainty estimates where relevant, indicator definitions and metadata, revision and comparability records, and disaggregated results, supported by coverage and missingness analysis and population and sampling information. Conflicting records, absent populations and uncertain follow-through require additional testing.
For the analysis, the reviewer should map the complete process, identify the intended result and responsible authority at each stage, and test normal cases together with exceptions. For responsible education data use, findings should establish whether the matter is isolated or indicates a condition requiring systemic response.
Evidence relevant to responsible education data use
The basis and limits of any conclusion concerning the available evidence should be explicit. For the analysis, measurement can reveal where outcomes differ; it does not by itself establish why they differ or which intervention will work. In the context of responsible education data use, international comparison can identify variation, but institutional and policy context remains necessary before a practice is transferred from one setting to another.
For decisions concerning responsible education data use, the evidential trail should allow an affected decision to be identified, examined and corrected. For the analysis, the responsible body should be able to identify the evidence considered, the judgement made, the person or body authorised to make it and the action that followed.
In examining responsible education data use: methodological considerations, across the defined scope, division of delivery responsibilities must not create gaps in learner protection.
Assessment of the measure should reconcile more than one source of evidence and control.