Due Date: Feb 26, 2009
In this project, you are to build unit tests with jUnit for your expression evaluator. You probably tested your evaluator by hand, sending in a bunch of random expressions. Hopefully, you sent in both good and bad strings during testing. Some of you more experienced developers may have built a script to rerun all of these tests and then scanned the output looking for errors.
In this project, you should formalize this ad hoc approach by using jUnit to run your unit tests. Further, you will use the excellent code coverage tool Clover by Cenqua to determine how thoroughly you have tested your code. Clover helps answer the question: "how much of my code have I actually tested?", which relates to "am I done testing?"
While building these tests, please contemplate the following: If I had given you a complete set of unit tests as part of the previous project's evaluator specification, imagine how much more precise it would have been! Students have so many questions about what is valid and what isn't. They are flying blind without a series of sample input/output pairs. This is why extreme programming advocates build the unit tests first. Unit tests are a great way to specify functionality and can therefore be used for designing and also debugging. You should express bugs to your fellow coders or yourself with a unit test. Once the unit test passes, that bug is squashed forever.
Create a test harness class called TestEval with a series of unit test methods. Here is the framework:
import junit.framework.TestCase;
public class TestEval extends TestCase {
...
}
You can make as many unit tests as you want, but I will be grading the thoroughness and appropriateness of your testing.
Note that you will almost certainly have to alter your software to make it testable. Also, please fix bugs as you discover them in your software.
Consider that the input is taken from System.in at the moment. You need to refactor your code so that your evaluator actually takes its input from an array, a string, or some other kind of string buffer. Then the main program should read all input from System.in and pass it to the refactored evaluator. In this manner, you may build unit tests to pass in strings like this:
public void testSimpleAddTwoInts() {
Eval e = new Eval("3+4");
int result = e.expression();
assertEquals("testing 3+4", 7, result);
}
where expression() is the method that invokes your evaluator. The grading program will invoke your TestEval class tests via jUnit per below.
Clearly if your code using System.exit to terminate upon error or whatever, it will stop any further unit testing. You must fix this. Errors in the parser should result in an exception that your unit tester can catch. You can create your own like EvalException (subclass Exception). You catch exceptions and set a flag, then test the flat:
public void testWhatever() {
boolean error = false;
try {
Eval t = new Eval("bad input");
}
catch (EvalException ee) {
error = true;
}
assertTrue(error); // expecting an error
}
As you build unit tests, you should check to see how much of your code you have actually tested. To do this, use Clover from the command line (or integrated into your development environment). Clover takes your source code and instruments it with tracking code via a source-to-source translation (using ANTLR). When you run the instrumented code manually or via jUnit, the Clover code logs how often each line of code is executed. After you have run all of your tests, you can generate a report to determine how much of the code your tests covered.
To get started, make sure the Clover jar is in your path (add it to your .bash_profile's CLASSPATH setting):
/home/public/cs601/clover.jar
Note that /home/public/cs601/clover.license is there too. The license must be in the same dir as the jar file for Clover to work.
Next you must instrument your code (I assume you have at least one test method in your TestEval class at this point). From the directory where your code lives, perhaps, ~userid/eval do this:
$ cd ~userid/eval $ java com.cenqua.clover.CloverInstr -i ~userid/eval/clover.db -d build Eval.java
Note: I suggest that you do not instrumenting your TestEval class, just do Eval and any other actual evaluator related code. The above command which will translate your Eval.java file to an instrumented version and stored in the build subdirectory. The path used with the -i option must be the fully qualified not relative path.
To run your unit tests, you'll need your TestEval.java file in the build subdir. Then just compile normally:
$ cd build $ cp ../TestEval.java . $ javac *.java
Run your unit tests:
$ java junit.textui.TestRunner TestEval
Of course, you can use the GUI jUnit interface also.
At this point, you can check to see how much code was covered by your test(s):
$ cd ~userid/eval $ java com.cenqua.clover.reporters.html.HtmlReporter -i clover.db -o coverage_html
This creates a subdirectory, coverage_html, that contains a full report of your code and how many times each line was executed. Open coverage_html/index.html to see the report. Note that when looking at your code in the report, the "hover text" tells you how many times each line was executed.
You should add more unit tests so that more and more of your Eval class is covered. Add a test, rerun via jUnit, and then rerun the coverage report generation. Check the report to see if you've covered more code. You should try to cover both forks of a conditional (i.e., true and false case).
You will create a jar file called expr-tests.jar containing source and *.class files and place in your lib directory (do not overwrite your expr.jar file):
https://www/svn/userid/cs601/expr2/trunk/lib
Do not include clover.db, the build directory containing your instrumented code, nor the coverage_html directory containing your HTML report. The easiest thing to do is probably just to erase all that before jar'ing up your project.
Do not use a package for your code. You do not need a class with main() in it for our grading purposes. Just make sure your testing class is called TestEval.
Please bring a stapled print out of your test harness, all other java files, and your printed HTML coverage reports generated from:
java com.cenqua.clover.reporters.html.HtmlReporter -i clover.db -o coverage_html
after you have finished writing all of your unit tests.
I will run your project via
$ java -cp /home/public/cs601/junit.jar:expr-tests.jar junit.textui.TestRunner TestEval
You will be graded on the following:
Your grade is a floating point number from 0..10. There are 8 points associated with unit tests and 2 points associated with the code coverage.