Services

Psychometric Services

Our psychometric team, comprised of psychometricians, data analysts, and software developers, offers extensive educational assessment experience and knowledge of best practices. Guided by the Standards for Educational and Psychological Testing (American Educational Research Association, American Psychological Association, and the National Council on Measurement in Education, 2014), our work is psychometrically sound and adheres to federal peer review requirements.

We offer a full range of psychometric services, including:

• field test planning
• operational test design
• equating
• scaling
• classical item analysis
• IRT analysis
• differential item functioning analysis
• alignment studies
• reliability and validity analyses
• technical report production
• standard setting

Test Development Services

We offer comprehensive test development services for standards-based assessments. In addition to developing items and test forms that illuminate instruction, our capabilities include developing item and test specifications, performance and oral assessments, observation checklists, administration manuals, scoring criteria, and interpretation guides. Our test development staff spans content specialists, project directors, editors, graphic artists, and support staff, who are advised by our psychometric team.

Handscoring

MI handscores tens of millions of student responses annually. Our unified handscoring system allows us to conduct all hiring, training, qualifying, scoring, monitoring, communicating, and reporting tasks remotely. At the heart of this system is our state-of-the-art Virtual Scoring Center (VSC), comprised of VSC Capture (a system for acquiring images and decoding response data from paper tests), VSC Train (a secure training and practice application for raters and scoring leadership), and VSC Score (a secure user management, scoring, and reporting application).

At the company’s inception, MI developed an outstanding training method for the scoring of student constructed responses which has become the industry model. On this foundation, we use our scoring technologies to monitor rater performance effectively and efficiently. In addition to traditional measures of rater accuracy and agreement, we employ a host of automated quality-assurance score verifications to ensure the most appropriate score has been assigned to each response.

MI’s handscoring service offerings include conducting rangefinding proceedings, developing scoring tools and training materials, evaluating prompts and constructed-response items, recruiting and hiring scoring personnel, performing training activities, and supervising scoring efforts.

Automated Scoring

MI has led the field in automated scoring solutions since first adopted by schools, districts, and states in formative and summative contexts. MI’s Project Essay Grade (PEG) automated scoring engine currently provides nearly 10 million summative scores for students across the US.

PEG and the MI team have dominated public competitions testing the state of the art of automated scoring. These contests have spanned essay scoring (the Hewlett Foundation-sponsored Automated Student Assessment Prize [ASAP] phase one), short constructed response English language arts and science scoring (ASAP phase two) and reading constructed response scoring (the National Center for Education Statistics [NCES]-sponsored National Assessment of Educational Progress [NAEP] Automated Scoring Challenge).

In most operational assessment program contexts we recommend a hybrid scoring approach, in which an automated scoring engine is used alongside human raters. This approach is designed to leverage the respective strengths of automated and hand-scoring while mitigating their respective limitations.