Advertisement
Blogs
Advertisement

Should a computer grade your essay?

Mon, 04/08/2013 - 10:09am
Jason Lomberg, Technical Editor

One should never swap quality for instant gratification. Yet that’s exactly what EdX, a nonprofit educational organization founded by Harvard and MIT, is doing with their automated grading software that promises “instant feedback” on students’ essays. Creativity need not apply.

The EdX software takes the results of 100 graded essays — assessed by a specific educator — and uses A.I. learning techniques to “train” itself to accurately predict what that teacher would “think” of a written work. That’s the theory, anyway.

We already know that roboticists can train virtual neural networks to imitate human behavior — and even pass the Turing test, depending on how you interpret the results of “Botprize”, a video game competition which sought to pass this critical A.I. threshold.

But most prior iterations of artificial constructs exhibiting “intelligence” — of the fallible sort we associate with humanity — were in a controlled setting, easily verifiable by objective scoring systems.

In the case of Botprize, the winning “bots” followed patterns — based on human observation — and mimicked human behavior in order to convince their flesh-and-blood opponents that they were, in fact, real. It was incumbent upon the human competitors to distinguish bot from human, and in more than 50% of cases, they couldn’t.

But the act of playing a video game doesn’t involve creativity — one of the distinguishing features of the human experience. We've often heard the maxim that — short of self-aware A.I. — computers will never compose a symphony, write a poem, or paint a masterpiece.

Every game — even “open-world” titles that stress exploration — contains preset paths and outcomes. Playing Xbox doesn’t allow you to “create”. It doesn’t facilitate creativity (beyond creative strategies). But writing does.

Educators rely on the practicality and convenience of automated scoring systems for multiple-choice exams. But essays can’t be so easily objectified. Creative writing involves variables — insight, creativity, logical progression — that don’t have a definitive ‘yes’ or ‘no’ answer.

Computers can “think” within their own set parameters. But they don’t have a will of their own. They can’t recognize the signposts of good writing because – in a very literal sense — they can’t think outside the box. So how can they possibly grade essays properly? No sample size is large enough to train the A.I. to recognize the infinite possibilities of well-written prose.

Moreover, is it moral to subject students to such a system?

An organization with the unfortunate title “Professionals Against Machine Scoring Of Student Essays In High-Stakes Assessment” doesn’t think so.

“Computers cannot ‘read.’ They cannot measure the essentials of effective written communication: accuracy, reasoning, adequacy of evidence, good sense, ethical stance, convincing argument, meaningful organization, clarity, and veracity, among others,” the group states on its website.

“To measure important writing skills, machines use algorithms that are so reductive as to be absurd: sophistication of vocabulary is reduced to the average length or relative infrequency of words, or development of ideas is reduced to average sentences per paragraph.”

And what’s the counter-argument? Breakneck speed.

“There is a huge value in learning with instant feedback,” said Dr. Agarwal, president of EdX. “We found that the quality of the grading is similar to the variation you find from instructor to instructor.”

So the automated scoring system may gloss over the fundamentals of good writing, but it’ll be quicker!

Such programs have been around for decades, but the sophistication of modern computing power has made them just “intelligent” enough to appeal to some misguided educators.

Already, prestigious centers of learning like Stanford have announced a partnership with EdX to develop a joint educational scheme built around their automated scoring system. And the not-for-profit enterprise is hoping more institutions adopt their standard.

For what it’s worth, I’m glad I matriculated at a time when human beings — not machines — judged my creativity.

Advertisement

Share this Story

X
You may login with either your assigned username or your e-mail address.
The password field is case sensitive.
Loading