MI participated in the National Assessment of Educational Progress (NAEP) Automated Scoring Challenge sponsored by the National Center for Education Statistics (NCES). NAEP is a gold-standard, nationally representative assessment known as “The Nation’s Report Card.” To date, all NAEP constructed response items have been handscored. The purpose of the Challenge was to determine whether automated scoring models could perform well with a representative subset of grade 4 and 8 NAEP Reading constructed response items. In addition to the score predictions, the Challenge required that submissions demonstrate interpretability of models, analyze models for potential bias based on student demographic characteristics, and provide cost information for putting an automated scoring system into operational use.
Over two dozen teams participated in the Challenge. MI was one of four grand prize winners with our score predictions exhibiting the greatest accuracy of all the submissions. MI’s submission was a team effort that involved MI’s automated scoring, psychometric, and handscoring staff.
MI looks forward to discussing our scoring engine and how we can assist your assessment scoring. Contact MI at MIMarketing@measinc.com
You can read the NCES press release about the prize announcement here: https://nces.ed.gov/whatsnew/press_releases/1_21_2022.asp.
More details on the scoring results are available here: https://github.com/NAEP-AS-Challenge/info/blob/952ee82cc52163b17bccc1f12174ce246b8dff9b/results.md.