Astonishing claim . . .

Can Programming Assignments Be Automatically Graded?

by Conrad Weisert
September 4, 2013
© 2013 Information Disciplines, Inc.

"In most programming classes . . . assignments are graded using automated test suites to verify conformance to specs."

—Philip Guo: "Teaching Programming the Way It Works Outside the Classroom",
Communications of the ACM, August, 2013, p. 10

I suppose if I taught a course to 200 students, I'd have to rely on some sort of automated grading. However, in my long experience I've neither taken nor taught a course, either academic or commercial, in which assignments and examinations weren't graded by the instructor (or sometimes by an unusually well-qualified graduate assistant).

If the program is complicated, of course, we might also run it through an automated verification. We hope that was all that Dr. Guo had in mind, but we wish he hadn't said most programming courses.

Letter Grades

What does it mean to say that a computer program "conforms to specs"? In an academic course what grade should such a program earn?

Any of those grades, even an F, can then be raised if the program shows clear evidence of a thoughtful, creative, or unusually thorough solution. On the other hand any grade can be lowered if the program violates essential criteria of good practice that were covered in the course, even if it yields the right answer.

To avoid surprises and later arguments I always explain these grading criteria at the beginning of a course, usually in the syllabus.

We remind students that a grade of A signifies an outstanding achievement, well beyond the merely adequate. An A does not signify, as some students expect, that there was "nothing wrong". Sometimes it helps to cite parallels with other creative disciplines: If Ernest Hemmingway turned in a short story in a Creative Writing course and got an A, what grade should his classmate earn for a story that contained no mistakes in grammar or spelling?

Return to Articles on education
IDI Home page

Last modified September 9, 2013