Interpretation of open science identifies scope, evidence, decision authority, material exceptions and continuing review.
Evidence relevant to open science
For open science, the instrument should be used to identify the intended direction, the actors addressed and the implementation measures that remain necessary.
In the context of open science, the Recommendation on the Ethics of Artificial Intelligence was adopted in November 2021. It establishes a global ethical framework addressing human rights, fairness, transparency, accountability, privacy, data governance, human oversight and environmental and social effects. In education, these principles require use-specific assessment: a system supporting routine administration does not carry the same risk as one influencing admission, assessment, progression or learner welfare.
In reviewing open science, implementation should be organised around a decision that can be tested.
The principal risks in relation to the control are reasonable adjustment altering the assessed outcome, inconsistent judgement between markers or locations, results used beyond the evidence they support, and tasks that do not assess the stated outcome. For open science, the relationship between the risks is material: one failed safeguard may remove the evidence needed to activate another.
Across the defined scope, the subject should be examined as a connected system of policy, people, resources, decisions and evidence. For open science, an imprecise scope or measure may produce a credible-looking record that does not answer the relevant decision question.
Assurance of the control should draw on more than one form of evidence. Useful records include analysis of results and differential outcomes, appeal and correction records, authorship and identity controls proportionate to risk, approval and change-control records, and marking criteria and calibrated judgement.
Application to open science
For the applicable requirement, the volume of documentation is not a measure of conformity.
When examining open science, traceability is necessary for accountable decision-making and fair correction. For the control, the responsible body should be able to identify the evidence considered, the judgement made, the person or body authorised to make it and the action that followed.
- Align tasks and criteria with learning outcomes.
- Calibrate assessors.
- Control changes.
- Moderate material variation.
- Retain evidence sufficient for review before it is relied on for a decision with material effect.
Controls for open science
Review of the control should map the complete process, identify the intended result and responsible authority at each stage, and test normal cases together with exceptions.
The final record on the applicable expectation should identify the applicable expectation, the relevant scope, the evidence examined, the sampling basis, material exceptions and the reason for the conclusion. For open science, equivalent methods should be assessed by demonstrated result, with the basis for acceptance retained.
- What action is required by the finding?
- Which evidence establishes operation?
- Where do exceptions occur?
- What outcome is intended?
- Who controls each stage?
Review of open science
In examining open science: implications for research governance, access and integrity, across the defined scope, division of delivery responsibilities must not create gaps in learner protection.
Review of the applicable expectation should address both system-level conditions and institutional practice. Assessment of open science should provide valid and sufficiently consistent evidence that the stated learning outcomes have been achieved by the learner receiving the result.
In the context of open science, progress should not be assessed by the amount of policy or documentation produced.