Data quality in reporting equitable higher education access

Consideration of equitable higher education access should retain the date and status of 2009 World Conference on Higher Education. Later developments should not be read into the position available at publication. Analysis of the analytical question should state the unit of analysis, reference period, coverage, exclusions and treatment of missing information. A comparison is reliable only if material differences remain visible. Consequential decisions should be considered in light of learner impact, institutional duty and stewardship of educational resources. The appropriate administrative form will depend on the jurisdiction and the allocation of lawful responsibility.

Scope of this analysis

The historical reference basis is the 2009 World Conference on Higher Education. Data used for equitable higher education access should be interpreted against stable definitions and an identifiable population. Changes in method, definition or series should remain separate from changes in the underlying result. Any consequential application should rest on evidence suited to the affected scope, not on the existence of an international development alone.

The system and institutional dimensions of the matter examined should be considered together. For the comparison, education indicators should support decisions by describing outcomes and variation with definitions and limitations that permit responsible interpretation. Authorities and providers hold different responsibilities, both of which must be discharged for the arrangement to operate reliably. Each level should be able to demonstrate the decisions and controls for which it is accountable.

A focused examination of the evidence under review requires a clear analytical discipline. Analysis of the matter examined should state the unit of analysis, reference period, coverage, exclusions and treatment of missing information. Any indicator used in relation to the matter examined should distinguish description from causal explanation. Material variation and uncertainty should be reported together with any restriction on wider application. The decision question, affected scope and measure should align; otherwise the conclusion may be unsupported despite substantial documentation.

The governing expectation for the reported measure should be capable of consistent application. Analysis of the comparison should state the unit of analysis, reference period, coverage, exclusions and treatment of missing information. The result should be accompanied by distributional information, uncertainty and limits on transfer to another setting. Definitions should provide a stable basis for decisions while allowing relevant differences to be identified and justified.

Any indicator used in relation to the comparison should distinguish description from causal explanation. A provider should also consider small differences overstated and proxy measures treated as direct outcomes. Stronger controls are required where learners may not detect an error or where later correction cannot restore the lost opportunity.

Assurance of the evidence under review should draw on more than one form of evidence. Useful records include revision and comparability records, uncertainty estimates where relevant, indicator definitions and metadata, disaggregated results, and triangulation with administrative and qualitative evidence. Documents should be reconciled with observed practice and, where relevant, the experience of affected learners. A positive example may illustrate operation, but it cannot demonstrate coverage or consistency.

Application in practice

Any indicator used in relation to equitable higher education access should distinguish description from causal explanation. Review of the evidence under review should trace selected records to source, reconcile totals across systems, quantify missing and late submissions, review manual adjustments and retain a revision history. Escalate discrepancies that could alter a published conclusion or individual outcome. Observations may inform further enquiry, but only supported findings should determine conformity or effectiveness.

The analytical record for the reported measure should state the research question, data source, unit of analysis, reference period, coverage, exclusions, treatment of missing values and principal limitations. Results should be reproducible from the retained data and method. Any causal explanation should be identified separately from descriptive findings and supported by an appropriate design.

Any indicator used in relation to the analytical question should distinguish description from causal explanation. When reviewing the analytical question, international comparison can identify variation, but institutional and policy context remains necessary before a practice is transferred from one setting to another. An exception is to remain time-limited, approved and subject to a stated review point.

Traceability is necessary for accountable decision-making and fair correction. For the evidence under review, the responsible body should be able to identify the evidence considered, the judgement made, the person or body authorised to make it and the action that followed. Material changes require a traceable effective date and explanation so that prior reliance can be reviewed fairly.

Accountability for the comparison should follow decision-making authority. Evidence of material risk should be placed before the body with authority to act, together with a traceable decision. Operational tasks may be delegated, but accountability for material effects on learners must remain identifiable.

Any indicator used in relation to the reported measure should distinguish description from causal explanation. Interpretation should retain uncertainty, distributional differences and limits on generalisation.

A revision or break in series should not be reported as a change in performance. The evidential basis for the matter under review should identify source, period, coverage and material limitations.