Measuring open educational resources across education systems

In examining measuring open educational resources across education systems, the analysis proceeds on the basis that the value of the present data lies in the questions it can answer reliably and in the limits it makes visible.

Application of the evidence to measuring open educational resources across education systems

The international recommendation adopted in November 2019 provides a policy reference for open educational resources.

Controls relevant to measuring open educational resources across education systems

Review of open educational resources should address both system-level conditions and institutional practice. Open access can widen participation, but users require clear information about authorship, licence, quality, accessibility, maintenance and the status of any learning or assessment.

When examining open educational resources, the subject should be examined as a connected system of policy, people, resources, decisions and evidence. The decision record for open educational resources should distinguish the scope supported by evidence from any scope that remains unresolved.

• Who controls each stage?
• Which evidence establishes operation?
• What outcome is intended?
• What action is required by the finding?
• Where do exceptions occur?

Review criteria for measuring open educational resources across education systems

In examining measuring open educational resources across education systems, for comparative analysis, trend claims require comparable observations over time and a documented account of revisions, breaks in series and changes in coverage.

The principal risks in relation to the issue are content altered without adequate review, unclear permission to reuse or adapt, inaccessible resources, and uncertain responsibility for learner support. Across the defined scope, a weakness in one part of the control environment may obscure a related failure elsewhere. For open educational resources, documents should be tested against the decision process they record and the outcome that followed.

Implications for measuring open educational resources across education systems

For open educational resources, the evidential record should be limited to material that can answer the question under review. The most relevant material is likely to include use and completion data interpreted cautiously, accessibility checks, academic and technical review, and clear statements on support and assessment.

Review of the measure should map the complete process, identify the intended result and responsible authority at each stage, and test normal cases together with exceptions. When examining open educational resources, the review record should preserve exceptions capable of showing a weakness in design, implementation or coverage.

Publication of findings on open educational resources should distinguish observed values, estimates and interpretation.

Evidence considered for measuring open educational resources across education systems

Interpretation of open educational resources should avoid two errors: treating a formal commitment as proof of effect, and treating one adverse case as proof that every part of the system has failed. Availability is not equivalent to accessibility, participation or learning. Each outcome requires separate evidence.

For open educational resources, decisions concerning the measure should remain traceable to the information available for the stated reference period. Across the defined scope, the reason for revision should be explicit, including whether it arises from new evidence, a methodological change or a different interpretation.

Public reporting on the issue should distinguish established fact, analytical judgement and planned action. For open educational resources, revision history should remain available where users have relied on the earlier conclusion.

The present development should inform review of the available evidence, with attention to the relationship between commitment, implementation and demonstrated outcome.