(This is part 4 of a series of posts on HTML5 from the point of view of a .NET developer)
To continue yesterday's description of the frameworks and libraries I'm using for most of my projects, I'd like to write a bit about the unit-testing and test running environment I'm currently preferring. (As always: nothing here is explicit sage advice - it's really just a collection of tools which works for me at the current point in time. My preferences might change, if I come across other tools which improve my workflow.)
For JavaScript unit testing, my current main tool is Qunit for specifying and implementing tests. I've added a few small additions which allow me to run individual tests (based on URL parameters) instead of full test modules; a feature I mainly use during development when I interactively work on certain features. (Unlike some of my friends who are very strict TDD-followers, I'm a very interactive developer, and spend quite a bit of time in Chrome's debugger and console while writing code. During this phase, I sometimes use my unit-tests simply to start off an interactive exploration in the JavaScript REPL console …)
Together with Qunit, I regularly use Sinon.js which provides a very nice implementation for spies, mocks, and stubs. Most importantly though, it can fake timers, XHR and server-requests.
When faking timers, Sinon takes over window.setTimeout() and friends and allows you to explicitly control the passing of time. You can for example do things like the following:
var sinonTimers = sinon.useFakeTimers();
var called = false;
window.setTimeout(function() {
called = true;
}, 100);
ok(!called,"Callback has not been invoked");
sinonTimers.tick(99);
ok(!called,"Callback has still not been invoked");
sinonTimers.tick(1);
ok(called,"Callback has now been invoked");
sinonTimers.restore();
This allows you to run time-based tests (timeouts, animations, …) without actually having to wait for wall-clock time to pass.
Running Tests
For automated execution of tests, I mostly use TeamCity which is also the CI environment for the compilation of our server-side components. To run the client-side JavaScript unit tests alongside the .NET unit tests, we rely on Phantom.JS which is a head-less WebKit implementation with a JavaScript API. This means that you can use JavaScript to drive an invisible browser, which provides you with a full WebKit-style implementation of the DOM and the complete stack of client-side behavior.
To combine Phantom.js with Qunit, we're using an adaptation of two scripts written by José F. Romaniello, the first one describes how to enable command-line execution of tests with Phantom and the second one allows the TeamCity test runner to interpret the results of your tests. You can find some more detail about the required output format for TeamCity here.
In TeamCity, we then configured a build step as a command-line invocation which calls "c:\windows\system32\cmd.exe" with the parameter "/c c:\some\directory\runtests.bat". runtests.bat in turn simply invokes Phantom.js with the above-mentioned JS file to start the test run by pointing to a URL which includes all JavaScript tests in our project.
Future plans
I have to admit that - as always - the situation is quite nice, but currently not 100% perfect. The detailed error information for failed unit tests, for example, is currently only available in the build log but not in the test overview. But this is one of these things which simply seem to stay in place as soon as a state of "good enough" has been reached.
One other things we plan for the future is to use additional test runners which will execute the tests on different real browser instances. But for now, the above is good enough and gives us very reliable and consistent results.
