Cross-system comparability in teaching and learning quality

The policy and evidence context for teaching and learning quality has been materially shaped by the global teaching and learning policy debate in 2013. In this case, the principal analytical task is to separate an observed difference from a conclusion about its cause.

For teaching and learning quality, the intended substantive result should remain the starting point for review. Education quality depends on sufficient numbers of competent staff who are prepared, supported and assigned work they can perform effectively. Inputs and formal commitments should be distinguished from demonstrated operation and outcome. Assurance should address actual effect and provide a means of correcting disadvantage that the arrangement did not intend.

Evidence and method

Relevant evidence for teaching and learning quality will normally include workforce plans and vacancy information, workload and allocation data, observation and learner feedback, support and supervision records, and qualification and competence records. The record for teaching and learning quality should retain disagreement between sources until its cause and effect are understood.

The stated reference is the global teaching and learning policy debate in 2013. Application to the analysis depends on evidence from the relevant jurisdiction or institution. As regards teaching and learning quality, authorities and providers should distinguish established fact, policy expectation and matters left to institutional judgement. Within the scope under review, that distinction should remain visible in the decision record, public reporting and later review.

For teaching and learning quality, the 2013/4 Education for All Global Monitoring Report, Teaching and Learning: Achieving Quality for All, places teaching quality and equitable learning at the centre of the post-2015 discussion. It links teacher preparation, deployment, support and working conditions with learner outcomes. Workforce indicators should consequently be interpreted alongside evidence of classroom practice, access to qualified teachers and differences between locations and learner groups.

For the analysis, comparison requires more than the use of a common label. For teaching and learning quality, definitions, reference periods, population coverage, institutional boundaries and collection practices must be sufficiently aligned for the observed difference to have a stable meaning.

Failure in relation to the issue may arise even where the stated policy is reasonable. Material concerns include use of staff outside their competence, vacancies or turnover affecting continuity, weak evaluation of teaching support, and professional learning disconnected from practice. For teaching and learning quality, materiality depends on the consequence and extent of an exception, not only on how often it appears in sampled records.

Patterns requiring examination

In the context of teaching and learning quality, the applicable expectation should be capable of consistent application. A sound interpretation should identify the unit of analysis, reference period, denominator, exclusions, missing values and any change in definition or collection practice. Definitions should provide a stable basis for decisions while allowing relevant differences to be identified and justified.

In work concerning teaching and learning quality, in this case, governing bodies should receive a concise account of the intended result, affected scope, principal risks, evidence limitations and unresolved exceptions. Closure requires evidence that the condition has changed; completion of planned activity is not sufficient.

Decisions concerning the comparison should remain traceable to the information available for the stated reference period.

• Forecast workforce need.
• Monitor workload and turnover.
• Prioritise support where learner need is greatest.
• Protect time for preparation and feedback, identifying the accountable function and affected scope.
• Evaluate professional learning in practice.

Implications for decision-makers

The review method for teaching and learning quality should be reproducible. The method for the measure is to prepare a comparability table before analysing results. Record common elements, material differences, breaks in series and the direction in which each limitation may affect the conclusion; do not rank systems where those limitations remain material. Within the scope under review, working papers should allow another competent reviewer to understand the evidence, judgement and treatment of material exceptions.

For teaching and learning quality, decision-makers using evidence on the measure should be told what the data cannot establish as clearly as what it can. The nature of the result and its applicable unit—system, institution, programme or learner group—should be explicit. Application in another setting depends on a separate examination of context and comparability.

The analysis should remain within the limits of the evidence. For teaching and learning quality, missing or delayed information may be patterned rather than random. Qualifications and participation in training are inputs. Material uncertainty should result in further enquiry or an expressly limited finding.

For decisions concerning teaching and learning quality, a clear objective, proportionate evidential basis and account of affected learners are required. An evidential gap in relation to teaching and learning quality should lead to a qualified conclusion and continued action, not administrative closure.