I feel immense satisfaction when I get an essay back from one of my professors that reads “job well done.” I just about want to call my mom if I see “wow, insightful!” written on it. This is partly because a good grade tend to accompany praise, and partly because the praise was written by my professor. Someone I respect invested time in my product and had something good to say about it. I wonder how I’d feel if a computer told me “well done.” I think I’d be far less enthused. But new automated essay scoring (AES) software by edX uses artificial intelligence to grade essays and paragraph responses, in place of professors.

EdX is a nonprofit, online learning company founded by Harvard and MIT. It provides free resources for any institution that wishes to use it. Its AES software requires a professor to submit 100 graded student essays, which it uses to develop a grading rubric for future essays submitted by the professor. It was originally developed to accommodate the thousands of students enrolled in MOOC’s (massive open online courses) and other online courses, whose student-teacher ratios make human grading impossible.

Free online learning works in that it disseminates knowledge. Anant Agarwal, head of edX, says the resource allows a “planet-scale democratization of education.” This kind of access is necessary in an age when the number of people that require higher education exceed capacity at colleges that they can afford. Professor Vicki Forman of the University of Southern California (USC) notes that human contact in education is a luxury: “Education in this country has always been a two-tier system: those who can afford it and have access will have human input, personal contact and face to face communication about the world of ideas.” The reality is, though, that most university students today take at least one class online during their college education to save on costs, and usually enroll at community colleges. Community college students themselves are forced out of the classroom and onto the web due to overcrowding. In an effort to combat overcrowding, the benefits of personal interaction have been sacrificed.

AES could also be used in colleges and universities, as well as in online courses, as a way to free professors from the responsibility of grading every paper and give them more time to do research. The risk in using AES for classes at colleges and universities comes with equating the dissemination of knowledge via a computer with knowledge conveyed through teaching. Professor Chris Freeman of USC makes the distinction: “The MOOC’s are great in terms of access to knowledge on a massive, affordable scale. That’s important and laudable, but trying to turn that into a liberal arts education with the kind of individualized feedback that students need for writing is not feasible in this model.”

As students, we make a personal commitment to our writing—that it may be original, logical and inspired. Our success depends not only on our own dedication, but also on a partnership with dedicated professors who provide personalized, meaningful feedback. The student who develops a personal relationship with his or her professor and sees him or her as a mentor will improve far more in his or her understanding of the material, and in his or her ability to think critically, than the student who skips office hours. A computer may be able to tell me if my sentences are original, but not my ideas. It won’t be able to assess my ability to reason, organize information in a meaningful way, or provide adequate evidence for my arguments. And it isn’t available for office hours.

Dr. Agarwal, however, insists that the software has been tested for its comparability to human graders and says it “comes close” (I think this is the most depressing thought of all—that the software “comes close” to most human graders). Mark Shermis, a professor at the University of Akron in Ohio, says of critics of the technology, “Often, [critics] come from very prestigious institutions where, in fact, they do a much better job of providing feedback than a machine ever could. There seems to be a lack of appreciation of what is actually going on in the real world.” Dr. Les Perleman, director of writing and comparative media studies at MIT, strongly disagrees, having uncovered the flaws in edX’s testing methodology. He is one of 2,000 educators who have signed a petition called “Professionals Against Machine Scoring of Student Essays in High-Stakes Assessment.” Apparently they haven’t found a catchy acronym yet. 

Despite objections from high-level professors, it is inevitable that even top universities will implement the auto-grader to save money, accommodate more students and reorganize their professors’ priorities. But some teachers at USC are already uncomfortable with outsourcing grading to TAs (perhaps as uncomfortable as I am, at $5,000 per class). Professor Freeman remarks, “I have issues that I’m not going to be the one grading every paper that comes across my desk. That’s not my teaching philosophy. I like to pay attention to the work I see on the page and the body I see in the classroom.” But the university requires certain teachers to outsource grading. They are a part of a system that values prestige, requires research and provides jobs to graduate students. Could USC eventually require professors to outsource their work to AES software, in order to increase some class sizes or further emphasize time spent on research?

Dan Schnur, director of the Unruh Institute of Politics at USC, says that he would not feel comfortable putting his student’s grades in the hands of developing technology that fundamentally limits essential parts of the educational experience: “The loss of that two way communication worries me greatly, both for reasons relating to substantive guidance but also the importance of providing encouragement to students navigating new and unfamiliar subject matter.”

The fundamental problem is that technology can’t produce a good judge of a person’s abilities. We know this from the inaccuracy of California K-12 school rankings. For example, according to Darrell Steinberg, there is a discrepancy between a certain Long Beach high school's identity as a community of enthusiastic students and satisfied parents, and its ranking as the worst school in the state. If the state decides to implement AES to evaluate its students and teachers, it would only contribute to an already imperfect system. This is one of those “high-stakes” situations distinguished by the “Professionals Against Machine Scoring of Student Essays in High-Stakes Assessment” petition, because it could reallocate funding, get a teacher fired, or ignore a problem altogether.

While there are several potential abuses, I think AES has useful applications as a supplement to teaching. If I had access to AES today, I would use it as my last line of defense before turning in this article to my editor. This would be a low-stakes assessment of my abilities, because I could use the resulting information as I saw fit. It could correct for grammar mistakes that my professors overlook in favor of focusing on larger structural issues and ideas. It would be great when it comes to meeting deadlines, because its feedback is instant.

When it comes to my college essays, however, I expect my professor to read it. Not only for the warm fuzzy feeling of a teacher’s praise and encouragement, but because, like most students at USC, I want to make the most of my expensive opportunity. That means taking advantage of the uniqueness and creativity professors bring to learning.

 

Reach Contributor Amanda Kantor here; follow her here.