Existing Tools

• Bulk Tests - A tool that compares the output of an executable with a provided sample output file. Rather than directly compare the two, it splits on lines of ``$-$''s and treats each as a separate test. Uses the advanced interface to output a number of points equal to the percentage passed times the ``maxpoints'' argument.
• Copy File - A tool that always returns success. It is used to ensure that a given file exists in each students submission directory.
• C Driver Tests - Replaces the ``main'' function in a submission with the contents of a given file. Compiles and runs the result. Otherwise behaves the same as Diff Test.
• Diff Test - Run an executable with given input and check to see if the output matches a given output file. Options include a timeout and optional flags to the diff utility to allow more imprecise matches.
• Detect Windows Code - Check the given file for DOS/Windows style newlines.
• Edit Distance Test - Similar to the Diff Test, but utilizes Python's difflib library to check the edit distance between the output of the executable and the sample output.
• Exit Code - Returns 0 if the given executable returns 0, 1 otherwise.
• Failure - Always return failure. Useful for setting up a single top-level rubric item that contains many sub-tests. Most commonly this is used for a ``Penalties'' top-level, which contains a Late Test, Detect Windows Code, Detect Line Wraps, etc.
• Correct Filename - Check that the filename provided matches the file invoked on. Earlier attempts at CAA had less-robust submission identification systems and so students were requested to submit their code with a given filename. This is less important now This test is also one of the few that should generally use the ``Penalize after all fail'' option.
• Post Hand Grading - Assumes the file it is invoked on is a list of point scores, one line per problem. Uses the advanced interface to output the number of points specified on a given line.
• Invoke Make - Invoke ``make'' in the submission directory and return the results of that operation.
• Late Test - Check the full path of the given file for a given token, or LATE if none is specified.
• Early/Late Points - Grant a specific number of points for submissions turned in 1, 2, or 3 days before a given deadline, and 1, 2, or 3 days after. Uses the advanced interface.
• Detect Line Wraps - Detects lines of code that are greater than 80 characters long, when tabs are expanded. The size of the tab stops and the max line length may be specified
• Success - Returns success. Used primarily for things that are graded by hand to give students all points by default and then reduce the score with comments.
• Post OCR - Before the Marksense system was split out from Agar, this was used to grade a file of student response against a given answer key, one answer per invocation. Not generally used now.
• Regexp Code - Check the given file for matches of the given regular expression.