Clarifies the scope, evidence and assurance considerations relevant to large-scale open online learning.
The present attention to large-scale open online learning follows the rapid expansion of open online courses in 2012 and requires a careful distinction between public commitment, institutional practice and demonstrated result. For the stated expectation, interpretation should begin with the intended outcome, then identify the controls and evidence needed to show that the outcome is achieved across the declared scope. A reliable review extends beyond the central process to material variation across programmes, sites, delivery arrangements and learner groups. Central policy alone does not establish consistent operation across the declared scope.
The governing expectation for the stated expectation should be capable of consistent application. In reviewing the relevant requirement, a provider should be able to trace the expectation from approved policy through implementation, monitoring, identified exceptions and corrective action. Criteria affecting learners should not permit materially different interpretation without an evidenced reason.
The present position
The historical reference basis is the rapid expansion of open online courses in 2012. Its relevance to large-scale open online learning should be assessed against the affected jurisdiction, learner population and form of provision. The wider development does not remove the need to establish the position through attributable evidence from the relevant jurisdiction or institution.
In practical terms, the matter under review should be reviewed against a stated method rather than general assurance. In reviewing the matter under review, the subject should be examined as a connected system of policy, people, resources, decisions and evidence. Transfer of decisions or records can expose weaknesses not visible in separate reviews of individual controls. A technically sound method remains inadequate if its limits are not clear to the body using the result.
Failure in relation to the relevant requirement may arise even where the stated policy is reasonable. Material concerns include inaccessible content or interaction, assessment methods that do not support valid judgements, reduced opportunities for timely support, and unclear identity and participation records. Materiality depends on the consequence and extent of an exception, not only on how often it appears in sampled records.
Assurance of the matter under review should draw on more than one form of evidence. Useful records include assessment validity and integrity reviews, teacher capability and workload information, supplier performance and exit arrangements, learner access and participation information, and delivery-mode design and approval records. Assurance should compare the documented arrangement with its operation and learner effect. System-wide assurance cannot be inferred from a favourable case chosen after the event.
Application in practice
Implementation of large-scale open online learning can be tested without imposing unnecessary reporting. Review of the stated expectation should map the complete process, identify the intended result and responsible authority at each stage, and test normal cases together with exceptions. The review should determine whether correction of an individual case is sufficient or broader action is required. Existing records may be used if reliable and relevant, but data collected for another purpose may not answer the assurance question.
Decisions concerning the stated expectation should remain traceable to the information available for the stated reference period. The reason for revision should be explicit, including whether it arises from new evidence, a methodological change or a different interpretation. Without this distinction, a reporting change may be mistaken for improvement or deterioration in educational practice.
- What outcome is intended?
- Which evidence establishes operation?
- Where do exceptions occur?
- Who controls each stage?
- What action is required by the finding?
What should be examined
Accountability for large-scale open online learning should follow decision-making authority. The decision must be referred to the authority capable of changing policy, allocating resources or formally accepting the remaining risk. The operating function may change, but responsibility for oversight and learner protection should remain clear.
The final record on the relevant requirement should identify the applicable expectation, the relevant scope, the evidence examined, the sampling basis, material exceptions and the reason for the conclusion. Departure from an illustrative method may be justified where equivalent outcome and evidence are established. Unresolved limitations should be stated with the conclusion and carried forward for action.
- Support staff and learners before using it to determine a learner or provider outcome.
- Define the educational purpose of the technology, including material exceptions and unequal effects.
- Test access before requiring use and retain evidence sufficient for independent review.
- Assure assessment validity before it is relied on for a decision with material effect.
- Provide alternative routes for material barriers, including material exceptions and unequal effects.
Jurisdictional and evidential limits
The quality significance of large-scale open online learning follows from a basic distinction between availability and effective provision. A decision concerning the control should recognise that a change in delivery mode should not weaken the defined learning outcomes, learner protection, accessibility or reliability of assessment. Review should cover the stages at which learners receive information, provision, assessment, support and remedy.
Proportionality in relation to the matter under review does not mean reduced protection for learners exposed to greater risk. For the matter under review, digital participation data should not be treated as a direct measure of learning. Log-ins, connection time and activity counts require interpretation alongside assessment and learner experience. For the relevant requirement, an isolated example cannot establish consistent operation, and an isolated failure should be evaluated for materiality, recurrence and systemic effect. Each exception should record its basis, authorisation, duration and review date.
The measure of progress on the assurance matter is not the amount of policy or documentation produced. The relevant measure is demonstrated public benefit, including detection and correction of material variation.