ETS — ETS Standards for Quality and Fairness (2014)
9/12Calibrated Authority
“If the test includes automated scoring of complex responses, use human raters as a check on the automated scoring. The extent to which human raters are required will vary with the quality of the automated scoring and the consequences of the decisions made on the basis of the scores.”— ETS Standards for Quality and Fairness (2014), Chapter 10 Scoring, Standard 10.3 'Using Automated Scoring' · source ↗
D1
Traceability & inspectability
2
D2
Human authorship & accountability
1
D3
Disclosure & labeling
1
D4
Synthetic-identity / fabrication prohibition
1
D5
Human validation in loop
2
D6
Evidential-trust emphasis
2
Posture BalancedBoundary fit · fitsTrust-logic Evidential
Reitz, C.H. (2026). The Calibrated Authority Index — ETS — ETS Standards for Quality and Fairness (2014). https://calibrated-authority.chrishuberreitz.com/institutions/ets-standards-quality-fairness Quote: “If the test includes automated scoring of complex responses, use human raters as a check on the automated scoring. The extent to which human raters are required will vary with the quality of the automated scoring and the consequences of the decisions made on the basis of the scores.” Source: https://www.ets.org/pdfs/about/standards-quality-fairness.pdf