QUnit and the Command Line: One Step Closer,

Previously I talked about getting QUnit JavaScript tests running on the command line using a simple Rhino setup.  Hacking a few lines together meant that we could run our tests outside the constraints of a browser and potentionally as part of an automated build (e.g. Hudson).  We had just one big problem: QUnit can run outside of a browser, but it still makes LOTS of assumptions based on being in a browser.

Foremost was the fact that the QUnit.log callback passed back HTML in the message parameter.  I wasn’t the only one to catch onto this issue (GitHub issue 32).  While I was still formulating a plan of attack to refactor out all of the browser oriented code, Jörn Zaefferer was putting a much simpler fix into place.  His commit added an additional details object to all QUnit.log calls which will contains the original assertion message w/o HTML and possibly the raw expected and actual values (not yet documented on the main page).  Problem solved!

Or so it seemed.

As I tried to hack my example CLI test runner to use the quick fix I ran into several issues.

Core QUnit logic still includes browser assumptions

Even with changes to wrap browser calls and guard against assuming certain browser objects existing, qunit.js is full of browser code.  It would be great if the core unit testing code could be separated by the code necessary to execute properly within a browser page and separate from how test results should be displayed on an HTML page.  If these three responsibilities were found in 3 different objects, it would be simple to replace one or more with ones that fit the scenario much more closely without needing to resort to hacks or breaking backward compatibility.

 

Single Responsibility Principle

Lifecycle callbacks break down outside of a browser

QUnit has some really nice lifecycle callbacks which anyone needing to integrate a testing tool can use.  They include callbacks for starting and finishing each test, individual assertions, and the whole test suite.  The first thing I wanted to add was reporting of the total number of passing and failing tests along with execution time when the tests were all done.  This looked like a simple job for QUnit.begin and QUnit.done.

It turns out that QUnit.begin won’t get called unless there is a window “load” event …. which doesn’t happen in Rhino …. so that is out.  To make matters worse, QUnit.done is getting called twice! For each test!  This means that my “final” stats are spammed with each test run.  With help of Rhino’s debugger app, I saw that the culprit were the successive “done” calls near the end of the “test” function.  Not sure how to fix that yet.

“Truthiness” in JavaScript is a double edged sword

Most of the time it is great not having to worry about if a value is really “False” or if it is undefined or some other “falsey” value.  Only one problem, zero (0) is falsey too.  Going back to my C days this isn’t too big of a deal (and can be used in clever ways).  However, if you are checking for the existence of properties of an object by doing a boolean expression … don’t.  Sure, undefined is falsey and if an object doesn’t have a property it will return undefined … but what if the value of that proeperty really IS undefined or in my case zero.  No good.

var foo = {cat: "dog", count: 0 }
!!foo.cat   // true
!!foo.bar   // false (doesn't exist, returns undefined)
!!foo.count // false (0 is falsey)

** The double bang (!!boolean) convention is a convienent way for converting between a falsey or truthy value to the explicit values TRUE and FALSE

Unit test frameworks should standardize the order of arguments!

This is a personal pet peeve.  I wish all unit testing framework would standardize whether the EXPECTED value or the ACTUAL value should go first on equals assertions.  JUnit puts expected first.  TestNG puts it second.  QUnit puts it second.  This is damn confusing if you have to switch between these tools frequently.

I moved my current code to a new GitHub repo -> QUnit-CLI.  As I learn more and find a better solution, I will keep this repo updated.  Currently the code outputs one line for each passing test and a more detailed set of lines for each failing test (including individual assertion results).  Because of the QUnit.done problem above, the “final” test suite results are shown twice for each test (making them not very “final”) which shows the total test results and execution time. [Edited to have correct link]

~~~~~~~~~~

Side Note: As an end goal, I would like to build a CLI for QUnit that will output Ant+JUnit style XML output which would make integrating these test results with other tools a piece of cake.  I CAN’T FIND THE XML DOCUMENTED ANYWHERE!  Lots of people have anecdotal evidence of how the XML will be produced but no one seems to have a DTD or XSD that is “official”.  If anyone knows of good documentation of the XML reports Ant generates for JUnit runs please let me know.  Thanks.

~~~~~~~~~~

λ This post has nothing to do with Lisp or Clojure λ