Explains evidential boundaries in relation to large-scale open online learning, with attention to decision authority, material exceptions and continuing assurance.
The present attention to large-scale open online learning follows the rapid expansion of open online courses in 2012 and requires a careful distinction between public commitment, institutional practice and demonstrated result. For the applicable expectation, interpretation should begin with the intended outcome, then identify the controls and evidence needed to show that the outcome is achieved across the declared scope.
For large-scale open online learning, the applicable expectation should be capable of consistent application. A provider should be able to trace the expectation from approved policy through implementation, monitoring, identified exceptions and corrective action. Criteria affecting learners should not permit materially different interpretation without an evidenced reason.
Meaning in practice
For decisions concerning large-scale open online learning, this analysis is informed by rapid expansion of open online courses in 2012. Its relevance to large-scale open online learning should be assessed against the affected jurisdiction, learner population and form of provision.
Review of the matter should be based on a stated method rather than general assurance. In work concerning large-scale open online learning, the subject should be examined as a connected system of policy, people, resources, decisions and evidence. Transfer of decisions or records can expose weaknesses not visible in separate reviews of individual controls.
Failure in relation to the applicable requirement may arise even where the stated policy is reasonable. Material concerns include inaccessible content or interaction, assessment methods that do not support valid judgements, reduced opportunities for timely support, and unclear identity and participation records. As regards large-scale open online learning, materiality depends on the consequence and extent of an exception, not only on how often it appears in sampled records.
When examining large-scale open online learning, assurance of the matter should draw on more than one form of evidence. Useful records include assessment validity and integrity reviews, teacher capability and workload information, supplier performance and exit arrangements, learner access and participation information, and delivery-mode design and approval records. Within the scope under review, system-wide assurance cannot be inferred from a favourable case chosen after the event.
Responsibilities and material risks
Implementation of large-scale open online learning can be tested without imposing unnecessary reporting. Review of the applicable expectation should map the complete process, identify the intended result and responsible authority at each stage, and test normal cases together with exceptions. Existing records may be used if reliable and relevant, but data collected for another purpose may not answer the assurance conclusion.
Decisions concerning the applicable expectation should remain traceable to the information available for the stated reference period. For large-scale open online learning, the reason for revision should be explicit, including whether it arises from new evidence, a methodological change or a different interpretation. Without this distinction, a reporting change may be mistaken for improvement or deterioration in educational practice.
- What outcome is intended?
- Which evidence establishes operation?
- Where do exceptions occur?
- Who controls each stage?
- What action is required by the finding?
Basis for a reliable conclusion
Accountability for large-scale open online learning should follow decision-making authority.
When examining large-scale open online learning, the final record on the applicable requirement should identify the applicable expectation, the relevant scope, the evidence examined, the sampling basis, material exceptions and the reason for the conclusion. Departure from an illustrative method may be justified where equivalent outcome and evidence are established. Unresolved limitations should be stated with the conclusion and carried forward for action.
- Support staff and learners before using it to determine a learner or provider outcome.
- Define the educational purpose of the technology.
- Test access before requiring use.
- Assure assessment validity before it is relied on for a decision with material effect.
- Provide alternative routes for material barriers.
Maintaining effective oversight
For large-scale open online learning, a change in delivery mode should not weaken the defined learning outcomes, learner protection, accessibility or reliability of assessment. Review should cover the stages at which learners receive information, provision, assessment, support and remedy.
Proportionality in relation to the matter does not mean reduced protection for learners exposed to greater risk. For the matter, digital participation data should not be treated as a direct measure of learning. In work concerning large-scale open online learning, log-ins, connection time and activity counts require interpretation alongside assessment and learner experience. For the applicable requirement, an isolated example cannot establish consistent operation, and an isolated failure should be evaluated for materiality, recurrence and systemic effect. Each exception should record its basis, authorisation, duration and review date.
Within the scope under review, progress should not be assessed by the amount of policy or documentation produced. For large-scale open online learning, the measure is demonstrated public benefit, including detection and correction of material variation.