{"schema_version":"ICEQC_NEWS_ARTICLE_V1","id":"iceqc-news-b376adc3003d9dc0","slug":"2020-09-22-institutional-preparedness-indicators-definitions-and-comparability","language":"en","publication_status":"READY_FOR_IMPORT","publication_date":"2020-09-22","last_modified_date":"2020-09-22","title":"Institutional preparedness indicators: definitions and comparability","summary":"Assesses evidence on institutional preparedness indicators: definitions and comparability, with particular attention to definitions, population coverage, comparability.","category":{"code":"DATA_RESEARCH","label":"Data Research"},"article_type":"Data and research analysis","publisher":"International Council for Education Quality Certification (ICEQC)","jurisdictional_scope":"International","historical_reference_basis":"Education provider continuity and risk responsibilities","reference_authority":"Relevant public authorities and official international sources","sections":[{"heading":null,"paragraphs":["The policy and evidence context for institutional preparedness indicators has been materially shaped by the education provider continuity and risk responsibilities. In reviewing the available evidence, evidence should inform action without implying a level of precision, coverage or causal certainty that the underlying data cannot support.","Failure in relation to the measure may arise even where the stated policy is reasonable. Material concerns include data revisions not carried through to published conclusions, averages concealing distribution, proxy measures treated as direct outcomes, and changes in definition presented as changes in performance. In relation to institutional preparedness indicators, the assessment of an exception should address severity, persistence and the likelihood that the condition is more widely present."]},{"heading":"Evidence and method","paragraphs":["Implementation of institutional preparedness indicators should be organised around a decision that can be tested. Oversight of the available evidence should reflect the principle that a sound interpretation should identify the unit of analysis, reference period, denominator, exclusions, missing values and any change in definition or collection practice. In practice, the stated objective should connect to responsibility, committed resources, operating evidence and the outcome reported for oversight.","The historical reference basis is the education provider continuity and risk responsibilities. Its relevance to the comparison should be assessed against the affected jurisdiction, learner population and form of provision. The international development warrants attention, but a consequential conclusion still requires current, attributable and representative evidence for the affected scope.","The analysis of the available evidence should make its decision rule explicit. In reviewing the analysis, comparison requires more than the use of a common label. Definitions, reference periods, population coverage, institutional boundaries and collection practices must be sufficiently aligned for the observed difference to have a stable meaning. A stated decision rule enables comparable examination and limits retrospective explanations of adverse evidence."],"bullets":["Avoid causal claims unsupported by the design before using it to determine a learner or provider outcome.","Analyse missing information.","Report uncertainty and revisions.","Disaggregate material results.","Define the decision the indicator will inform."]},{"heading":"Patterns requiring examination","paragraphs":["Interpretation of institutional preparedness indicators should avoid two errors: treating a formal commitment as proof of effect, and treating one adverse case as proof that every part of the system has failed. In reviewing the comparison, measurement can reveal where outcomes differ; it does not by itself establish why they differ or which intervention will work. Oversight of the available evidence should reflect the principle that missing or delayed information may be patterned rather than random.","Evidence should be selected against a clearly defined question. For the available evidence, the most relevant material is likely to include population and sampling information, indicator definitions and metadata, uncertainty estimates where relevant, and coverage and missingness analysis. In relation to institutional preparedness indicators, each source has limitations; confidence depends on corroboration between independent records and transparent treatment of uncertainty.","The assurance record for the available evidence should retain the date of the evidence, the source responsible for it, the scope examined and the version of any instrument or definition applied. Traceable source and version information allow genuine improvement to be distinguished from administrative revision. Earlier conclusions should remain traceable if they affected a learner, provider or public decision."]},{"heading":"Implications for decision-makers","paragraphs":["A proportionate method is available for institutional preparedness indicators. For the available evidence, the reviewer should prepare a comparability table before analysing results. Record common elements, material differences, breaks in series and the direction in which each limitation may affect the conclusion; do not rank systems where those limitations remain material. The review record should preserve exceptions capable of showing a weakness in design, implementation or coverage.","Publication of findings on the available evidence should distinguish observed values, estimates and interpretation."],"bullets":["Are the populations defined on the same basis?","Do the reference periods align?","Are exclusions and missing records comparable?","Has a classification changed?","Is the remaining difference educationally material?"]},{"heading":"Limits of inference","paragraphs":["The intended substantive result should remain the starting point for review. A decision concerning institutional preparedness indicators should recognise that education indicators should support decisions by describing outcomes and variation with definitions and limitations that permit responsible interpretation. Formal adoption, expenditure and activity do not in themselves establish the intended result. Assurance should address actual effect and provide a means of correcting disadvantage that the arrangement did not intend.","Public reporting on the measure should distinguish established fact, analytical judgement and planned action. Material revisions should be traceable to their reason and effective date. Changes to definitions or evidence should be recorded separately from changes in educational performance.","Assurance concerning the measure requires corroborating evidence across the material scope. A reasoned conclusion should reconcile the governing requirement, evidence of operation, learner outcomes and residual risk, and remain open to better evidence."]}],"word_count":775,"content_hash":"sha256-ce1bfc53c79b5c767b78010e09d5bf1d6de6ace01d68d72cc48c7c7622ab610f","seo_keywords":["institutional preparedness indicators","international education data research","education quality","ICEQC"],"schema_type":"AnalysisNewsArticle"}