{"schema_version":"ICEQC_NEWS_ARTICLE_V1","id":"iceqc-news-a8b94a959592fe79","slug":"2007-12-07-improving-science-learning-outcomes-a-structured-review-method","language":"en","publication_status":"READY_FOR_IMPORT","publication_date":"2007-12-07","last_modified_date":"2007-12-07","title":"Improving science learning outcomes: a structured review method","summary":"Sets out a controlled approach to improving science learning outcomes: a structured review method, linking diagnosis, accountable action, outcome measures and verification.","category":{"code":"QUALITY_IMPROVEMENT_METHODS","label":"Quality Improvement Methods"},"article_type":"Quality improvement method","publisher":"International Council for Education Quality Certification (ICEQC)","jurisdictional_scope":"International","historical_reference_basis":"PISA 2006 results released in December 2007","reference_authority":"Organisation for Economic Co-operation and Development","sections":[{"heading":null,"paragraphs":["Against the background of the PISA 2006 results released in December 2007, education authorities and providers should review how science learning outcomes is defined, implemented and evidenced. The analysis of the improvement objective proceeds on the basis that effective improvement requires ownership, a time-bound intervention and independent confirmation that the intended result has been achieved. The assessment addresses decisions capable of affecting learners, institutions or the proper use of entrusted educational resources.","The PISA 2006 cycle gives particular emphasis to science while also assessing reading and mathematics among 15-year-old students. Analysis of improving the improvement objective should state the unit of analysis, reference period, coverage, exclusions and treatment of missing information. A comparison is reliable only if material differences remain visible. Data used for the improvement objective should be interpreted against stable definitions and an identifiable population. A revision or break in series should not be reported as a change in performance."]},{"heading":"Scope of the improvement","paragraphs":["The reference basis—the PISA 2006 results released in December 2007—is evidential rather than self-executing. The material may reveal patterns or evidential gaps, but it neither directs a legal outcome nor establishes causation. Analysis of science learning outcomes should state the unit of analysis, reference period, coverage, exclusions and treatment of missing information. Comparative findings should not conceal differences capable of changing their meaning.","For the corrective programme, the public interest is not confined to institutional compliance. Corrective action concerning the issue under review should address the identified cause, assign responsibility and set a review period. Residual risk should remain open until sustained improvement is demonstrated.","Corrective action concerning the corrective programme should address the identified cause, assign responsibility and set a review period. Data used for improving the corrective programme should be interpreted against stable definitions and an identifiable population. Broad intentions should be converted into decisions capable of review. In relation to improving science learning outcomes, the decision record should distinguish the scope supported by evidence from any scope that remains unresolved.","The governing expectation for the corrective programme should be capable of consistent application. Oversight of the practice should reflect the principle that effectiveness should be judged against an agreed outcome and reference period, not against completion of activities alone. Public information on the intervention should state the applicable scope and limitations in terms that affected users can understand, including the basis for any later correction.","Failure in relation to the intervention may arise even where the stated policy is reasonable. Material concerns include tasks that do not assess the stated outcome, results used beyond the evidence they support, inconsistent judgement between markers or locations, and reasonable adjustment altering the assessed outcome. Review should consider whether an exception is prolonged, recurring or capable of affecting learners outside the cases examined.","Improvement work on the intervention should begin with a verified problem, defined baseline and measurable outcome. Completion should depend on evidence of effect rather than completion of planned activity. For the issue under review, the most relevant material is likely to include approval and change-control records, authorship and identity controls proportionate to risk, appeal and correction records, and moderation and exception records. In relation to improving science learning outcomes, each source has limitations; confidence depends on corroboration between independent records and transparent treatment of uncertainty."]},{"heading":"Implementation responsibilities","paragraphs":["For operational review of science learning outcomes, authorities and providers should proceed in a defined sequence. In reviewing the practice, responsible bodies should prioritise actions by learner impact and control weakness, establish dependencies, test implementation at suitable intervals and retain unresolved items until effectiveness is verified. Amend the plan where evidence does not support the original causal assumption. Improvement work on improving the practice should begin with a verified problem, defined baseline and measurable outcome.","Improvement of the intervention should proceed through controlled tests where risk permits. Corrective action concerning the practice should address the identified cause, assign responsibility and set a review period.","The analysis of the issue under review should remain within the limits of the evidence. Measures should remain in place long enough to detect recurrence and unintended effects. Oversight of the improvement objective should reflect the principle that reliability without validity produces consistent but potentially irrelevant results. Validity without adequate consistency may expose learners to unequal judgement. Material uncertainty should result in further enquiry or an expressly limited finding.","The assurance record for the issue under review should retain the date of the evidence, the source responsible for it, the scope examined and the version of any instrument or definition applied. Intervention in the issue under review should be proportionate to the identified condition and tested where risk permits. Wider implementation should follow evidence of benefit and acceptable unintended effects. Records concerning the practice should remain traceable from source evidence to decision and follow-up. Superseded conclusions should be retained where they informed a material outcome.","Review of the intervention should give particular attention to adverse cases, unequal effects and errors that learners may be unable to identify or remedy after the event. The action record should identify who is responsible and when implementation is due. Closure requires evidence that the condition has changed; completion of planned activity is not sufficient.","No individual measure is sufficient to establish effective operation of the practice across the affected scope. Examination of the matter should give particular attention to adverse cases, unequal effects and errors that learners may be unable to identify or remedy after the event."]}],"word_count":898,"content_hash":"sha256-ab791ab98fe25db42d5d001bfdb03e0bcefdee88516b74ff4f4f330a2b5e752f","seo_keywords":["improving science learning outcomes","education quality improvement","education quality","ICEQC"],"schema_type":"TechArticle"}