{"schema_version":"ICEQC_NEWS_ARTICLE_V1","id":"iceqc-news-9dffffc4900d7ff6","slug":"2017-02-28-responsible-education-data-use-methodological-considerations","language":"en","publication_status":"READY_FOR_IMPORT","publication_date":"2017-02-28","last_modified_date":"2017-02-28","title":"Responsible education data use: methodological considerations","summary":"Reviews responsible education data use: methodological considerations and identifies the conditions required for sound interpretation, responsible comparison and defensible.","category":{"code":"DATA_RESEARCH","label":"Data Research"},"article_type":"Data and research analysis","publisher":"International Council for Education Quality Certification (ICEQC)","jurisdictional_scope":"International","historical_reference_basis":"Expanding public reporting and comparative indicators","reference_authority":"Relevant public authorities and official international sources","sections":[{"heading":null,"paragraphs":["The expanding public reporting and comparative indicators provide the immediate reference point for consideration of responsible education data use in 2017. In this case, the available evidence should be interpreted with close attention to definitions, population coverage, collection methods and the limits of comparison."]},{"heading":"Evidence base for responsible education data use","paragraphs":["The required public outcome should be stated in operational terms. In reviewing responsible education data use, education indicators should support decisions by describing outcomes and variation with definitions and limitations that permit responsible interpretation. Formal adoption, expenditure and activity do not in themselves establish the intended result. Authorities and providers require evidence of operation and effect, with a route to identify and correct unequal or unintended consequences."],"bullets":["Define the decision the indicator will inform.","Avoid causal claims unsupported by the design.","Analyse missing information.","Test comparability.","Document numerator and denominator."]},{"heading":"Coverage and comparability","paragraphs":["The reference basis—the expanding public reporting and comparative indicators—is evidential rather than self-executing. The material may reveal patterns or evidential gaps, but it neither directs a legal outcome nor establishes causation. In applying it to responsible education data use, users should review the source definitions, population coverage, reference period and stated limitations before transferring a system-level finding to an individual provider or learner group.","A focused examination of the analysis requires a clear analytical discipline. In reviewing the comparison, the subject should be examined as a connected system of policy, people, resources, decisions and evidence. Transfer points should be tested because responsibility and information may be lost between otherwise sound functions. In relation to responsible education data use, the decision question, affected scope and measure should align; otherwise the conclusion may be unsupported despite substantial documentation."]},{"heading":"Responsible interpretation","paragraphs":["Implementation of responsible education data use should be organised around a decision that can be tested. A decision concerning the issue should recognise that where an indicator is used as a proxy, the relationship between the proxy and the underlying educational outcome should be stated and tested. In practice, the stated objective should connect to responsibility, committed resources, operating evidence and the outcome reported for oversight.","The principal risks in relation to the available evidence are changes in definition presented as changes in performance, proxy measures treated as direct outcomes, incomplete coverage, and small differences overstated. The control environment should be assessed as a connected system rather than as unrelated individual risks. In relation to responsible education data use, review should follow the sequence of decisions and records rather than assess documents in isolation."],"bullets":["What action is required by the finding?","Where do exceptions occur?","What outcome is intended?","Which evidence establishes operation?","Who controls each stage?"]},{"heading":"Limitations and reporting","paragraphs":["The evidential record for responsible education data use should permit a reviewer to trace the matter from decision to outcome. This may require uncertainty estimates where relevant, indicator definitions and metadata, revision and comparability records, and disaggregated results, supported by coverage and missingness analysis and population and sampling information. Conflicting records, absent populations and uncertain follow-through require additional testing.","Implementation of the measure can be tested without imposing unnecessary reporting. For the analysis, the reviewer should map the complete process, identify the intended result and responsible authority at each stage, and test normal cases together with exceptions. Findings should establish whether the matter is isolated or indicates a condition requiring systemic response. Information should not be treated as sufficient merely because it is already available; its relevance to the present question must be established."]},{"heading":"Limitations and reporting","paragraphs":["Publication of findings on responsible education data use should distinguish observed values, estimates and interpretation.","The basis and limits of any conclusion concerning the available evidence should be explicit. For the analysis, measurement can reveal where outcomes differ; it does not by itself establish why they differ or which intervention will work. A decision concerning the available evidence should recognise that international comparison can identify variation, but institutional and policy context remains necessary before a practice is transferred from one setting to another. Material limitations should be stated with the finding presented to decision-makers and affected learners.","The evidential trail should allow an affected decision to be identified, examined and corrected. For the analysis, the responsible body should be able to identify the evidence considered, the judgement made, the person or body authorised to make it and the action that followed. Historical decisions should be assessed against the information then available, with later amendments separately dated and explained.","Where the issue involves partners, suppliers or several public bodies, responsibility should be mapped across the complete service. Contractual or inter-agency arrangements should identify who holds records, informs learners and acts on incidents. Division of delivery responsibilities must not create gaps in learner protection.","Assessment of the measure should reconcile more than one source of evidence and control."]}],"word_count":779,"content_hash":"sha256-844988166971b31c5df1a65ee9dc7cd84a7865c0a2c2326d393e68ae771dfabd","seo_keywords":["responsible education data use","international education data research","education quality","ICEQC"],"schema_type":"AnalysisNewsArticle"}