preview

Automated Essay Scoring with E-Rater V.2.0

Better Essays

Research Report

Automated Essay Scoring With E-rater® v.2.0

Yigal Attali Jill Burstein

Research & Development

November 2005 RR-04-45

Automated Essay Scoring With E-rater® v.2.0

Yigal Attali and Jill Burstein ETS, Princeton, NJ

November 2005

As part of its educational and social mission and in fulfilling the organization 's nonprofit charter and bylaws, ETS has and continues to learn from and also to lead research that furthers educational and measurement research to advance quality and equity in education and assessment for all users of the organization 's products and services. ETS Research Reports provide preliminary and limited dissemination of ETS research prior to publication. To obtain a PDF or a print copy …show more content…

An analysis of the e-rater v.1.3 features revealed that some of them were implicitly measuring essay length (i.e., number of words in the essay) or had a nonmonotonic relationship with the human score. Features in e-rater v.2.0 were created by standardizing some features with regard to essay length, by altering the definition of others to take into account the nonmonotonic relationship 1

with the human score, and also by creating new features. Below is a description, by category, of the 12 features included in the new feature set. Errors in Grammar, Usage, Mechanics, and Style (Four Features) Criterion produces feedback about a total of 33 errors in four categories: grammar, usage, mechanics, and comments about style. Most of these features are identified using natural language processing techniques (Burstein et al., 2003). Counts of the errors in the four categories form the basis for four features in e-rater v.2.0, referred to herein as grammar, usage, mechanics, and style. However, the features actually computed are the rates of errors in the four categories and are calculated for each category by counting the total number of errors in that category and dividing this by the total number of words in an essay. Organization and Development (Two Features) In addition to the various errors, the Criterion feedback application automatically identifies

Get Access