Examines open educational resources, addressing cross-system comparability and the evidential limits relevant to responsible interpretation and decision-making.
The international recommendation adopted in November 2019 provides the immediate context for open educational resources. For comparative analysis, the value of the present data lies in the questions it can answer reliably and in the limits it makes visible.
The principal risks in relation to the comparison are outdated or inaccurate material, content altered without adequate review, inaccessible resources, and confusion between participation and recognised achievement. For open educational resources, a weakness in one part of the control environment may obscure a related failure elsewhere.
Evidence base for cross-system comparability in open educational resources
A proper review of open educational resources should establish the intended outcome before selecting controls or indicators. For comparative analysis, reported averages should be accompanied by sufficient distributional information to identify material differences between learner groups, locations and forms of provision. The record for open educational resources should explain why the approach suits the affected context, how material departures are authorised and when review will occur.
The formal status of the international recommendation adopted in November 2019 should be preserved in any public account. In this case, the instrument should be used to identify the intended direction, the actors addressed and the implementation measures that remain necessary.
Comparison requires more than the use of a common label. For decisions concerning open educational resources, definitions, reference periods, population coverage, institutional boundaries and collection practices must be sufficiently aligned for the observed difference to have a stable meaning. The judgement should state its supporting evidence and any condition limiting application to the declared scope.
- Identify authorship and version.
- Verify rights before publication.
- Monitor use without overstating learning.
- Review content and accessibility.
- State whether assessment or credit is available.
Coverage and comparability
Proportionality in relation to open educational resources does not mean reduced protection for learners exposed to greater risk. Availability is not equivalent to accessibility, participation or learning. Each outcome requires separate evidence. International comparison can identify variation, but institutional and policy context remains necessary before a practice is transferred from one setting to another. No exception should continue without a documented basis, accountable approval and scheduled review.
Relevant evidence for the issue will normally include licence and provenance records, clear statements on support and assessment, accessibility checks, academic and technical review, and version and maintenance information. As regards open educational resources, contradictory evidence should be investigated and resolved, not omitted from the record.
Within the scope under review, the evidential trail should allow an affected decision to be identified, examined and corrected. For the measure, the responsible body should be able to identify the evidence considered, the judgement made, the person or body authorised to make it and the action that followed. Historical decisions concerning open educational resources should be assessed against the information then available, with later amendments separately dated and explained.
Responsible interpretation
A competent review of the measure should prepare a comparability table before analysing results. In reviewing open educational resources, record common elements, material differences, breaks in series and the direction in which each limitation may affect the conclusion; do not rank systems where those limitations remain material. Adverse cases and unresolved contradictions should be retained because they may reveal limitations concealed by an average result.
In the context of open educational resources, decision-makers using evidence on the available evidence should be told what the data cannot establish as clearly as what it can. Users should be able to distinguish descriptive, comparative and evaluative findings and understand their proper level of application. Use in a different context requires an independent judgement that the settings are materially comparable.
- Are exclusions and missing records comparable?
- Do the reference periods align?
- Has a classification changed?
- Is the remaining difference educationally material?
- Are the populations defined on the same basis?
Limitations and reporting
Open access can widen participation, but users require clear information about authorship, licence, quality, accessibility, maintenance and the status of any learning or assessment. When examining open educational resources, review should cover the stages at which learners receive information, provision, assessment, support and remedy.
Public reporting on the comparison should distinguish established fact, analytical judgement and planned action. For decisions concerning open educational resources, material revisions should be traceable to their reason and effective date. Changes to definitions or evidence should be recorded separately from changes in educational performance.
The decision record for open educational resources should connect the stated objective to suitable evidence and the position of those affected. An evidential gap in relation to open educational resources should lead to a qualified conclusion and continued action, not administrative closure.