{"schema_version":"ICEQC_NEWS_ARTICLE_V1","id":"iceqc-news-bc22171f2c8bdd01","slug":"2013-11-20-records-required-to-substantiate-assessment-reliability","language":"en","publication_status":"READY_FOR_IMPORT","publication_date":"2013-11-20","last_modified_date":"2013-11-20","title":"Records required to substantiate assessment reliability","summary":"Sets out the scope, decision criteria and evidential basis relevant to records required to substantiate assessment reliability, including the treatment of material limitations.","category":{"code":"STANDARDS_INTERPRETATION","label":"Standards Interpretation"},"article_type":"Standards interpretation","publisher":"International Council for Education Quality Certification (ICEQC)","jurisdictional_scope":"International","historical_reference_basis":"International comparative assessment practice","reference_authority":"Relevant public authorities and official international sources","sections":[{"heading":null,"paragraphs":["In 2013, consideration of assessment reliability must take account of the international comparative assessment practice and the responsibilities it places before education systems. The analysis of the stated expectation proceeds on the basis that the requirement should be read as an assurance obligation: the provider must be able to explain the control, show its operation and account for material exceptions."]},{"heading":"Applicable scope","paragraphs":["The historical reference basis is the international comparative assessment practice. Its relevance to assessment reliability should be assessed against the affected jurisdiction, learner population and form of provision.","Implementation of the relevant requirement should be organised around a decision that can be tested. In reviewing the control, evidence is sufficient when it is current, attributable, representative of the relevant scope and capable of being reconciled with other available records. In relation to records required to substantiate assessment reliability, resources and activity should be reconciled with the operating evidence and result for which the responsible function is accountable.","The principal risks in relation to the control are uncontrolled changes to assessment, inconsistent judgement between markers or locations, reasonable adjustment altering the assessed outcome, and weak assurance of authorship or performance. The risks are interdependent; failure of one control may conceal or disable another. In relation to records required to substantiate assessment reliability, review should follow the sequence of decisions and records rather than assess documents in isolation.","A focused examination of the stated expectation requires a clear analytical discipline. A decision concerning the relevant requirement should recognise that a reliable record should identify what occurred, when it occurred, who was responsible, the authority for the action and any later correction. Records should remain protected against unauthorised alteration while legitimate amendments remain visible. An imprecise scope or measure may produce a credible-looking record that does not answer the relevant decision question.","Relevant evidence for records required to substantiate assessment reliability will normally include marking criteria and calibrated judgement, appeal and correction records, moderation and exception records, approval and change-control records, and assessment maps to learning outcomes. Evidence outside the relevant period or scope should be identified and given no more weight than its limitations permit. Contradictory evidence should be investigated and resolved, not omitted from the record."]},{"heading":"Implementation and evidence","paragraphs":["Care is required in drawing conclusions about assessment reliability. For the assurance question, reliability without validity produces consistent but potentially irrelevant results. Validity without adequate consistency may expose learners to unequal judgement. Oversight of the assurance question should reflect the principle that interpretive guidance should not create an obligation that is absent from the governing instrument or applicable law. Limitations should be prominent wherever the finding may influence a consequential decision.","Accountability and effective correction both depend on a record that can be followed from evidence to decision. For the assurance question, the responsible body should be able to identify the evidence considered, the judgement made, the person or body authorised to make it and the action that followed. Material changes require a traceable effective date and explanation so that prior reliance can be reviewed fairly."],"bullets":["Review differential and anomalous results.","Define the decision each assessment must support, with responsibility, scope and timing recorded.","Retain evidence sufficient for review.","Calibrate assessors, identifying the accountable function and affected scope.","Moderate material variation before it is relied on for a decision with material effect."]},{"heading":"Assessment of conformity","paragraphs":["Implementation of assessment reliability can be tested without imposing unnecessary reporting. For the stated expectation, the reviewer should specify mandatory fields, source ownership, access rights, retention and correction procedures. Test a sample from creation through use, amendment, reporting and disposal, including records created during disruption or by a delivery partner. Reuse of existing information is appropriate only where its purpose, scope and reliability correspond to the decision under review.","The final record on the issue under review should identify the applicable expectation, the relevant scope, the evidence examined, the sampling basis, material exceptions and the reason for the conclusion. Equivalent methods should be assessed by demonstrated result, with the basis for acceptance retained. The affected scope should remain open where a material limitation prevents assurance."],"bullets":["Can records be retrieved throughout the required period?","Is the record attributable?","Are partner records subject to equivalent controls?","Can an amendment be distinguished from the original?","Are access rights proportionate?"]},{"heading":"Review and corrective action","paragraphs":["Accountability for assessment reliability should follow decision-making authority. Evidence of material risk should be placed before the body with authority to act, together with a traceable decision.","The quality significance of records required to substantiate assessment reliability under review follows from a basic distinction between availability and effective provision. The analysis of the stated expectation proceeds on the basis that assessment should provide valid and sufficiently consistent evidence that the stated learning outcomes have been achieved by the learner receiving the result. Review should cover the stages at which learners receive information, provision, assessment, support and remedy.","No individual measure is sufficient to establish effective operation of the stated expectation across the affected scope. A conclusion should be revised when stronger evidence materially changes the assessment of implementation, outcome or risk."]}],"word_count":834,"content_hash":"sha256-bf9016fe92ab80eda45ce184ae5656350ac01f85c4b3c345f5c55c1b56b77647","seo_keywords":["records required to substantiate assessment reliability","education quality standards","education quality","ICEQC"],"schema_type":"TechArticle"}