Two Guys Arguing

QUnit-CLI: Running QUnit with Rhino

Posted in git, javascript, software testing by benjaminplee on 11.26.10

Rhino - wikimediaPreviously I talked about wanting to run QUnit outside the browser and about some issues I ran into.  Finally, I have QUnit running from the command line: QUnit-CLI

After a good deal of hacking and a push from jzaefferer, I gotten the example code in QUnit-CLI to run using Rhino and no browser in sight.  This isn’t a complete substitute for in-browser testing, but makes integration with build servers and faster feedback possible.

/projects/qunit-cli/js suite.js
PASS - Adding numbers
FAIL - Subtracting numbers
PASS|OK|subtraction function exists|
FAIL|EQ|Intended bug!!!|Expected: 0, Actual: -2
PASS - module without setup/teardown (default)
PASS - expect in testPASS - expect in test
PASS - module with setup
PASS - module with setup/teardown
PASS - module without setup/teardown
PASS - scope check
PASS - scope check
PASS - modify testEnvironment
PASS - testEnvironment reset for next test
PASS - scope check
PASS - modify testEnvironment
PASS - testEnvironment reset for next test
PASS - makeurl working
PASS - makeurl working with settings from testEnvironment
PASS - each test can extend the module testEnvironment
PASS - jsDump output
PASS - raises
PASS - mod2
PASS - reset runs assertions
PASS - reset runs assertions2
PASS: 22  FAIL: 1  TOTAL: 23
Finished in 0.161 seconds.

The first hurdle was adding guards around all QUnit.js’s references to setTimeout, setInterval, and other browser/document specific objects.  In addition I extended the test.js browser checks to include all of the asynchronous tests and fixture tests.  Finally I cleaned up a bit of the jsDump code to work better with varying object call chains.  My alterations can be found on my fork here.

The second hurdle was getting QUnit-CLI using my modified version of QUnit.js and adjusting how Rhino errors are handled.  Adding a QUnit submodule to the QUnit-CLI git repository easily fixed the first (I previously posted my notes on git submodules and fixed branches).  QUnit.js’s borrowed jsDump code is used to “pretty-print” objects in test messages.  jzaefferer ran into an issue when running QUnit’s own tests through QUnit-CLI resulting in the cryptic error:

js: "../qunit/qunit.js", line 1021: Java class "[B" has no public instance field or method named "setInterval".
at ../qunit/qunit.js:1021
at ../qunit/qunit.js:1002
at ../qunit/qunit.js:1085
at ../qunit/qunit.js:1085
at ../qunit/qunit.js:1085
at ../qunit/qunit.js:110
at ../qunit/qunit.js:712 (process)
at ../qunit/qunit.js:304
at suite.js:84

It turns out that errors objects (e.g. ReferenceError) throw in Rhino include an additional property of rhinoException which points to the underlying Java exception that was actually thrown.  The error we saw is generated when the jsDump code walks the error object tree down to a byte array off of the exception.  Property requests executed against this byte array throw the Java error above, even if they are done part of a typeof check, e.g.

var property_exists = (typeof !== 'undefined');

Once I figured this out, I wrapped the object parser inside QUnit.jsDump to properly pretty-print error objects and delegate to the original code for any other type of object.

var current_object_parser = QUnit.jsDump.parsers.object;
QUnit.jsDump.setParser('object', function(obj) {
  if(typeof obj.rhinoException !== 'undefined') {
    return + " { message: '" + obj.message + "', fileName: '" + obj.fileName + "', lineNumber: " + obj.lineNumber + " }";
  else {
    return current_object_parser(obj);

With these changes we have a decent command line executable test suite runner for QUnit.  With a bit more work QUnit-CLI will hopefully be able to print Ant/JUnit style XML output and/or include stack traces when errors bubble out of test code.

Tagged with: , , , ,

5 Responses

Subscribe to comments with RSS.

  1. Ariya said, on 11.27.10 at 1:58 pm

    Ever thought of using V8? See for an example.

    • benjaminplee said, on 11.27.10 at 4:10 pm

      Very interesting. Yes, I would like to have examples to go with all the major javascript engines.

  2. […]  QUnit w/ Rhino posts.  At every step of the way I thought “Oh this should be simple and clever” and […]

  3. Karl said, on 06.07.11 at 1:53 pm

    thanks for publishing your findings on all of this. I’m honestly astounded that there aren’t more people wanting to run pure unit tests for their javascript code from the command line and your approach seems to be the best I’ve found.

  4. […] Cet article indique une version modifiée de QUnit qui fait le lien entre QUnit et QUnit-CLI. Téléchargez les 2 projets : – QUnit modifié – QUnit-CLI […]

Leave a Reply to faire du TDD javascript en ligne de commande avec QUnit | Barre Verte ! Cancel reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s