Research integrity: responsibilities, evidence and follow-up

The present attention to research integrity follows the revised recommendation on science and scientific researchers and requires a careful distinction between public commitment, institutional practice and demonstrated result. The method set out here treats improvement as a controlled cycle of diagnosis, action, measurement and review. The decision should address both public impact and the responsibilities attached to entrusted educational resources. Systems may organise responsibility differently while remaining accountable for comparable public results.

Scope of the improvement

For research integrity, the formal status of the revised recommendation on science and scientific researchers should be preserved in any public account. The instrument should be used to identify the intended direction, the actors addressed and the implementation measures that remain necessary.

A proper examination of the matter should establish the intended outcome before selecting controls or indicators. In the context of research integrity, the corrective action should be tested on a scale proportionate to the risk before wider implementation, unless immediate system-wide action is necessary to protect learners. Suitability, authorised variation and the date for reconsideration should be established when the arrangement is approved.

A narrow control over the intended improvement may create false assurance. In the present context, inconsistent judgement between markers or locations, uncontrolled changes to assessment and weak assurance of authorship or performance may produce acceptable aggregate reporting while individual learners remain exposed to material disadvantage. For decisions concerning research integrity, adverse cases should form part of the sample wherever they may reveal a material control weakness.

Analysis should make its decision rule explicit. For the relevant practice, effectiveness is the demonstrated change in the condition the action was intended to address. In work concerning research integrity, completion of training, publication of guidance or installation of a system is an output and should not be reported as an outcome without further evidence. A stated decision rule enables comparable examination and limits retrospective explanations of adverse evidence.

Assurance of the corrective action should draw on more than one form of evidence. Useful records include assessment maps to learning outcomes, marking criteria and calibrated judgement, appeal and correction records, moderation and exception records, and analysis of results and differential outcomes. In work concerning research integrity, documents should be reconciled with observed practice and, where relevant, the experience of affected learners.

Implementation responsibilities

The analysis of research integrity should remain within the limits of the evidence. For the intended improvement, correcting an individual record does not establish that the process which produced the error has been corrected. Reliability without validity produces consistent but potentially irrelevant results. Within the scope under review, validity without adequate consistency may expose learners to unequal judgement. Material uncertainty should result in further enquiry or an expressly limited finding.

For research integrity, records relating to the matter should preserve both the conclusion and its limits. New evidence should trigger a traceable correction and review of decisions materially affected by the earlier conclusion. Where reliance has occurred, correction may require review of affected decisions as well as amendment of published information.

• Define the decision each assessment must support.
• Calibrate assessors.
• Moderate material variation, with responsibility, scope and timing recorded.
• Review differential and anomalous results.
• Retain evidence sufficient for review.

Testing effectiveness

A competent review of the relevant practice should set a baseline and success measure before intervention, define the review period, compare the result with the intended outcome and examine adverse or unequal effects. For decisions concerning research integrity, continue monitoring long enough to determine whether the improvement is sustained. Averages should be tested against adverse cases that may indicate unequal effect or incomplete operation.

A decision to close improvement work on research integrity should be made by a person with authority and sufficient independence from implementation.

• Did the effect reach the intended group?
• What was the baseline?
• Has the improvement been sustained?
• When should an effect be visible?
• What condition should change?

Maintaining the result

For research integrity, governing bodies should receive a concise account of the intended result, affected scope, principal risks, evidence limitations and unresolved exceptions. Within the scope under review, material action requires a named responsible function and a defined completion point. Evidence of outcome, rather than completion of tasks, should determine whether corrective work can close.

The system and institutional dimensions of the corrective action should be considered together. In work concerning research integrity, assessment should provide valid and sufficiently consistent evidence that the stated learning outcomes have been achieved by the learner receiving the result. Authorities and providers hold different responsibilities, both of which must be discharged for the arrangement to operate reliably.

Progress on research integrity under review is not the amount of policy or documentation produced.