Quality evidence for learning-outcomes-based qualifications

The revised qualifications framework recommendation provides the immediate reference point for consideration of learning-outcomes-based qualifications in 2017. For the relevant requirement, a standard is effective only when its terms lead to consistent decisions without displacing professional judgement or applicable law. The unit of review should correspond to the full reach of the decision, including significant differences in provision and population. Evidence of formal policy should not be treated as evidence of uniform implementation.

The revised 2017 Recommendation on the European Qualifications Framework retains eight learning-outcome-based reference levels and strengthens attention to transparency, quality assurance and comparison. It supports the presentation of level information but does not replace national qualifications or recognition decisions. Institutions should ensure that published references are authorised, current and connected to the complete qualification record.

The governing expectation for the stated expectation should be capable of consistent application. Oversight of the relevant requirement should reflect the principle that conformity should not be inferred from a policy document alone; operating records and outcomes should show that the stated arrangements are in use. Operational definitions should be precise enough to support consistent consequential decisions and explain justified variation.

Purpose and present context

The formal status of the revised qualifications framework recommendation should be preserved in any public account. Adoption records an agreed instrument or policy position; it does not necessarily make every provision directly enforceable in every jurisdiction. For learning-outcomes-based qualifications, the instrument should be used to identify the intended direction, the actors addressed and the implementation measures that remain necessary. Domestic law and authorised guidance continue to determine specific legal duties.

The intended substantive result should remain the starting point for review. A decision concerning the assurance matter should recognise that learners should receive accurate information about the status, level, content and recognition of learning before committing time or money across jurisdictions. The existence of an approved measure or completed activity is not evidence of educational effect. The operating record should enable responsible bodies to detect unintended effects and act where outcomes are unequal.

• State the legal and academic status of the offer and retain evidence sufficient for independent review.
• Preserve verifiable records within a defined period and review the result.
• Provide support suited to mobile learners, including material exceptions and unequal effects.
• Monitor partner and jurisdictional risks, including material exceptions and unequal effects.
• Identify the authority responsible for each decision and retain evidence sufficient for independent review.

Application in practice

The technical issue within learning-outcomes-based qualifications concerns the basis on which a conclusion is reached. For the control, the subject should be examined as a connected system of policy, people, resources, decisions and evidence. Transfer points should be tested because responsibility and information may be lost between otherwise sound functions. Any condition preventing complete assurance should appear with the evidence on which the judgement relies.

A narrow control applied to the relevant process may create false assurance. In the present context, jurisdictional uncertainty in complaints, unclear awarding responsibility and claims that overstate recognition or transferability may produce acceptable aggregate reporting while individual learners remain exposed to material disadvantage. A sample confined to compliant cases cannot establish the reliability of the control.

Assurance of the control should draw on more than one form of evidence. Useful records include cross-border agreements and responsibility maps, documented credit and recognition decisions, outcomes for mobile and non-mobile learners, clear identification of providers and awarding bodies, and complaint and appeal routes. Policy and records should be tested against actual practice, including evidence from learners where appropriate. A positive example may illustrate operation, but it cannot demonstrate coverage or consistency.

• Which evidence establishes operation?
• Where do exceptions occur?
• What outcome is intended?
• Who controls each stage?
• What action is required by the finding?

Evidence and assurance

A proportionate method is available for learning-outcomes-based qualifications. For the matter under review, the reviewer should map the complete process, identify the intended result and responsible authority at each stage, and test normal cases together with exceptions. The conclusion should differentiate an individual error from evidence of a wider control weakness. The review record should preserve exceptions capable of showing a weakness in design, implementation or coverage.

Assurance concerning the stated expectation should be expressed at the level established by the evidence. A sample may support a conclusion about the sampled process, but not automatically about every location or programme. Where reliance is placed on central controls, testing should confirm that local operation and exceptions are reported accurately to the centre.

Proportionality in relation to the relevant requirement does not mean reduced protection for learners exposed to greater risk. A decision concerning the stated expectation should recognise that transparency supports fair decision-making but does not make qualifications automatically equivalent. The basis and intended use of recognition should be explicit in each consequential decision. The analysis of the relevant requirement proceeds on the basis that the volume of documentation is not a measure of conformity. Relevance, integrity and coverage are more important than the number of records produced. Each exception should record its basis, authorisation, duration and review date.

Records relating to the matter under review should preserve both the conclusion and its limits. A changed evidential position should be applied to the affected scope, including prior decisions that may no longer be reliable. Where reliance has occurred, correction may require review of affected decisions as well as amendment of published information.

Public reporting on the control should distinguish established fact, analytical judgement and planned action. A material change should not remove the earlier position from the evidential trail. Users should be told when apparent movement results from revision rather than substantive improvement or deterioration.

Neither one indicator nor one control can establish the complete position on the control. A conclusion should be revised when stronger evidence materially changes the assessment of implementation, outcome or risk.