Consistency in the assessment of research integrity

Evidence relevant to consistency in the assessment of research integrity

The revised recommendation on science and scientific researchers provides a policy reference for consistency in the assessment of research integrity.

For the conclusion, the public interest is not confined to institutional compliance. Assessment of research integrity should provide valid and sufficiently consistent evidence that the stated learning outcomes have been achieved by the learner receiving the result.

• Moderate material variation before using it to determine a learner or provider outcome.
• Retain evidence sufficient for review before using it to determine a learner or provider outcome.
• Calibrate assessors.
• Align tasks and criteria with learning outcomes before it is relied on for a decision with material effect.
• Review differential and anomalous results, with responsibility, scope and timing recorded.

Application to consistency in the assessment of research integrity

In examining consistency in the assessment of research integrity, consistency does not require identical decisions regardless of context. It requires comparable matters to be treated on the same principles, with material differences explained by relevant evidence and recorded criteria.

The principal risks in relation to the matter are weak assurance of authorship or performance, tasks that do not assess the stated outcome, results used beyond the evidence they support, and reasonable adjustment altering the assessed outcome. Separate treatment would overlook how a failed control may prevent detection or operation of another safeguard. Across the defined scope, review should follow the sequence of decisions and records rather than assess documents in isolation.

The evidential record for the applicable requirement should permit a reviewer to trace the matter from decision to outcome. This may require approval and change-control records, analysis of results and differential outcomes, authorship and identity controls proportionate to risk, and marking criteria and calibrated judgement, supported by appeal and correction records and moderation and exception records. For research integrity, further cases should be examined when the initial sample does not represent the affected scope or confirm sustained correction.

• Where are outcomes materially different?
• Is the reason relevant and documented?
• Have decision-makers been calibrated?
• Does review correct inconsistent treatment?
• Are common criteria in use?

Controls for consistency in the assessment of research integrity

In examining consistency in the assessment of research integrity, responsible bodies should use common definitions and decision criteria, calibrate responsible staff, review outliers and compare outcomes across locations and groups. Where variation is justified, retain the reason and verify that it is applied without arbitrary disadvantage.

The final record on the applicable expectation should identify the applicable expectation, the relevant scope, the evidence examined, the sampling basis, material exceptions and the reason for the conclusion. The approving record should explain how an alternative approach satisfies the governing requirement. For research integrity, a limitation preventing a complete conclusion should remain visible and unresolved until suitable evidence is obtained.

Proportionality in relation to the applicable requirement does not mean reduced protection for learners exposed to greater risk. In reviewing research integrity, reliability without validity produces consistent but potentially irrelevant results.

The assurance record for research integrity should retain the date of the evidence, the source responsible for it, the scope examined and the version of any instrument or definition applied. A later reviewer should be able to identify whether the condition changed or the evidential record was corrected.

In examining consistency in the assessment of research integrity, accountability for research integrity should follow decision-making authority.

The decision record for research integrity should connect the stated objective to suitable evidence and the position of those affected. Where evidence concerning research integrity cannot support assurance, the limitation should be reported and corrective work should remain open.