Explains competence requirements in relation to learning-outcome measurement, covering scope, evidence, decision authority, material exceptions and continuing assurance.
The policy and evidence context for learning-outcome measurement has been materially shaped by the international assessment and indicator development. For the assurance conclusion, the requirement should be read as an assurance obligation: the provider must be able to explain the control, show its operation and account for material exceptions.
This analysis is informed by international assessment and indicator development. Its relevance to the applicable expectation should be assessed against the affected jurisdiction, learner population and form of provision. For learning-outcome measurement, any consequential application should rest on evidence suited to the affected scope, not on the existence of an international development alone.
For the control, assessment should provide valid and sufficiently consistent evidence that the stated learning outcomes have been achieved by the learner receiving the result.
Meaning in practice
The analysis of learning-outcome measurement should make its decision rule explicit. Effectiveness is the demonstrated change in the condition the action was intended to address. Completion of training, publication of guidance or installation of a system is an output and should not be reported as an outcome without further evidence. Comparable evidence should be assessed against criteria settled before the result is known.
In work concerning learning-outcome measurement, the applicable expectation should be capable of consistent application. For the assurance conclusion, evidence is sufficient when it is current, attributable, representative of the relevant scope and capable of being reconciled with other available records. Criteria affecting learners should not permit materially different interpretation without an evidenced reason.
Responsibilities and material risks
Risk assessment of learning-outcome measurement should give particular attention to tasks that do not assess the stated outcome, uncontrolled changes to assessment, and weak assurance of authorship or performance. A provider should also consider inconsistent judgement between markers or locations and reasonable adjustment altering the assessed outcome.
- Review differential and anomalous results.
- Calibrate assessors.
- Control changes, identifying the accountable function and affected scope.
- Retain evidence sufficient for review.
- Define the decision each assessment must support.
Basis for a reliable conclusion
For learning-outcome measurement, readily available material should not define the enquiry if it cannot answer the relevant decision question. The most relevant material is likely to include approval and change-control records, authorship and identity controls proportionate to risk, appeal and correction records, and assessment maps to learning outcomes.
A competent review of the applicable expectation should set a baseline and success measure before intervention, define the review period, compare the result with the intended outcome and examine adverse or unequal effects. Within the scope under review, continue monitoring long enough to determine whether the improvement is sustained. For learning-outcome measurement, averages should be tested against adverse cases that may indicate unequal effect or incomplete operation.
Interpretation of learning-outcome measurement should produce a test that another competent reviewer can apply to comparable evidence.
Maintaining effective oversight
Interpretation of learning-outcome measurement should avoid two errors: treating a formal commitment as proof of effect, and treating one adverse case as proof that every part of the system has failed. Reliability without validity produces consistent but potentially irrelevant results. Validity without adequate consistency may expose learners to unequal judgement. An isolated example cannot establish consistent operation, and an isolated failure should be evaluated for materiality, recurrence and systemic effect.
For decisions concerning learning-outcome measurement, a traceable record enables responsibility to be established and errors to be corrected fairly. For the applicable expectation, the responsible body should be able to identify the evidence considered, the judgement made, the person or body authorised to make it and the action that followed. A material amendment should record its reason and effective date, preserving the information basis of earlier decisions.
Accountability for learning-outcome measurement should follow decision-making authority. Within the scope under review, delegation of delivery does not remove the need for a named authority to oversee material learner impact.
For learning-outcome measurement, progress should not be assessed by the amount of policy or documentation produced.