As mentioned in Chapter 4, in order to ensure maximal flexibility in testing tools, the interface for tools in Agar was intentionally kept as simple as possible. For the majority of tools this works quite well: each tool can be written to test one submission at a time and return a simple Pass or Fail value. Agar itself can then determine appropriate point values as a result. However, some tests are non-binary. One of the first places this became a difficulty was in the grading of multiple choice questions for an upper division course wherein the instructor desired a ``guessing penalty'' to be applied to any wrong answers. Thus each question had 3 possible outcomes: a positive score for a correct answer, a zero for no answer (an admitted ``I don't know''), and a negative score of an incorrect answer. This could have been dealt with by writing two separate tools for the testing: one that checks for a correct answer, and a secondary that is invoked when the first fails that checks for an incorrect answer. However, even at the time that this was first becoming a known issue the author was becoming increasingly aware that some grading is simply not a binary operation5.1. A more ``advanced'' tool interface was thus developed where testing tools can choose themselves how many points are to be awarded for a given invocation of the tool.
The choice was made to extend the original tool interface rather than completely redevelop with a more complicated (and rigorous) system for extending the toolset. It is very much not in line with the spirit of the project to force users to spend hours developing custom tools when a less robust interface would allow for faster development in the common case. As noted in Chapter 4, this decision turned out to be rather unimportant, but very much affected the complexity of the tool interface.