... prime^1.1

And as of this writing, the only known.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... system^2.1

This completely ignores the very important issue of what can be learned by the student in a given practice item, focusing only on what makes the grading of a given item more or less effective. This is by no means sufficient to ensure that a given assignment is pedagogically valuable to the students, but is intended a list of things necessary to maximize the pedagogical value of a given item.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... submissions,^3.1

In this case, a mechanism for the computer operator to invoke the submission on the next test case

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... necessary.''^3.2

It is nice that we are no longer talking about grading student's programs on punch cards and requiring a physical reset upon error.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... graders^3.3

If designer and instructor feels redundant, replace these terms with author and assigner.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... documentation^4.1

The possibility of using a static analyzer like lint[6] to enforce certain style requirements is also known.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... Agar^4.2

Agar is not an acronym, nor does it have any intrinsic meaning. It was chosen because it vaguely conjures ``automated grading'', but doesn't violate the author's distaste for acronyms.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... code^4.3

In general this will not be intentionally malicious code, but students are notoriously capable of discovering new and violent ways of making their programs crash.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... need^4.4

Or in most cases, allow.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... important^4.5

In the CS&E department here, we have assignments in C, C++, Python, VHDL, an assembly language, and Perl on a regular basis. Ideally these should all be equally supported.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... possible^4.6

Details of the tool interface can be found in Section 5.2.1

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... deployment^4.7

An earlier understanding of this idea would have greatly simplified the deployment and development process.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... world^4.8

Ignoring setting up mail service settings, which is a relatively simple one-time cost.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... adoption^4.9

In short, the learning curve can be insurmountable if the workings of the program are not completely transparent. This has been a problem in the development of Agar, but is one of the prime things that we hope to fix in Agar2

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... operation^5.1

This decision was strikingly upheld two months later when it was discovered that there was need for a ``Bulk Testing'' tool that would test a submission against a number of outputs at once and return what fraction of those were a match. To do this using a purely binary system would require, in this case, a 30-fold complexity increase, which could cause the automated testing to take longer than a human doing the same manually.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... shebang^5.2

For example, #!/usr/bin/python

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... test^5.3

In pseudo-code terms: for each test: for each submission: for each file: if the file matches the file-mask, invoke the test on that file.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... mask)^5.4

A fourth option, pictured, has been deprecated since the Marksense system has been moved from OCR to Optical Mark Recognition (OMR), and has been split out of Agar into an entirely command-line based suite of tools.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

... release^6.1

Note that the CVS repository never branches

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.