Two Guys Arguing

Cucumber: Making UATs the healthy choice

Posted in software testing, Uncategorized by benjaminplee on 11.01.09

My current project team has been eating its vegetables.  Has yours?

Selenium is a great tool for testing webapps from a true browser driven user’s perspective.  By driving most mainstream browsers via a core Javascript library, a Selenium script can easily test how your app behaves (or misbehaves) just like a real user would see.

Webrat lets you quickly write robust and thorough acceptance tests for a web application.”  Written in Ruby, Webrat allows acceptance tests to exercise an arbitrary web application through its Selenium integration, or Rails apps without starting a server.

Cucumber allows code to be written in a simple domain specific language (DSL) that anyone can read.  Layer Cucumber on top of Webrat and Selenium, and you have User Acceptance Tests (UATs) that the client can read (and ideally write), that the entire development team understands, and which can be run as BDD or regression tests.

Cucumber example taken from

Cucumber example taken from

Without going into the details of how each of these libraries work, below are some of my notes, tips, and impressions of using Cucumber to write UATs for a greenfield Java EE application over the past ~10 weeks.

  • Ideally, each test would run in its own transaction so that they wouldn’t interfere with each other and could be run in parallel.  Using Cucumber (Ruby) to test a totally separate web application running on Tomcat (Java) presents a problem.  The team debated several possible solutions using JRuby and having the Cucumber build spawn an embedded Tomcat instance but decided that the cost outweighed the benefit to the current project.
  • Since we were forced to use pure JDBC by our client, we decided to use ActiveRecord migrations to build our tables, and generate SQL which we could re-leverage from our Cucumber tests.  Custom creation Webrat steps were written to push setup test data for each model.
  • DB2 support with the Ruby ActiveRecord drivers is lacking proper install instructions on Linux (damn thing requries a full DB2 install to run). By leveraging JRuby, we were able to use the DB2 JDBC drivers instead which worked much better (without installing DB2).  Getting Cucumber, Webrat, and Selenium running in JRuby took a bit of finesse and shebang wrangling but eventually worked.
  • Outputting Cucumber results to a JUnit style XML output made our integration on our Hudson CI server simple and easy to read. (The Chuck Norris plugin helps too)
  • We tag each test with a story number (e.g. @10045) so that we can quickly run all tests for a given feature easily: cucumber –tags @10045.
  • Our QA lead works to define and write-up our Cucumber tests during pre-iteration planning and before each story enters our work queue (the team uses a modified Kanban board approach to pull work through each iteration, Cucumber is our first queue).  When she has a test written, she will tag it with @in-process to let us know that it is ready to be worked on, but not yet implemented.
  • We have custom Rake tasks to run all of our “finished” tests as well as just our “in-process” ones.  The in-process task will fail if any tests pass (they should be marked as finished).  Unfortunately this rake task doesn’t work with the Hudson/JUnit build.
  • Ongoing issues we still struggle with are:
    • making sure that small variations in DSL verbiage don’t muddy up our tests (it is easy for the team to accidentally end up with two different commands which do the same or similar actions without a clear distinction)
    • cross browser and separate environment testing happens on different dedicated servers on dependent builds so that they don’t end up stepping on each other
    • as it is with any testing library, care needs to be taken to keep only common tests grouped together and setup/background work common to all tests that NEED it.  Pushing too much work into common actions results in slow tests
    • Tests are supremely faster and more valuable than manual tests, but are not as fast as JUnit tests and will never replace manual exploratory testing.

Overall the team has been very pleased with our ability to drive our development from the tests and verify that new changes don’t break existing functionality from the users’ perspectives.

* Another Asynchrony team is working on leveraging Cuke4Duke to do similar testing of a Java thick client.  Should be interesting.

Edit: Thanks to Amos King for some proof reading help.  You can find his blog over at Dirty Information.


mario math – part 3

Posted in Uncategorized by youngnh on 08.17.09

There is a counter that only allows Mario to accelerate vertically for 8 consecutive frames. Release KEY_JUMP at any time before frame 8 and gravity takes over.

When you are on the ground and press KEY_JUMP, the code calculates an initial velocity for Mario, changes his position accordingly and then calculates a decreased velocity owing to friction and gravity before that frame’s tick() ends. If you are pressing KEY_JUMP the next frame, this slower velocity is thrown away and another new initial velocity, smaller than the one before it, but still upwards, is added to Mario’s position and again a final, slower velocity is calculated before the frame’s end. This repeats until frame 8 when your jump runs out and gravity is the only y acceleration acting on Mario.

Here’s a quick table that displays velocities and heights, as it’s not really worth the rigmarole involved in deriving complex equations for so discrete a frame set.

Frame Initial Velocity Height Final Velocity
1 7 * 1.9 13.3 8.305
2 7 * 1.9 26.6 8.305
3 6* 1.9 38 6.69
4 5 * 1.9 47.5 5.075
5 4 * 1.9 55.1 3.46
6 3 * 1.9 60.8 1.845
7 2 * 1.9 64.6 0.23
8 1 * 1.9 66.5 -1.385

Here I’ve written the velocities with a positive sign, since upwards is considered positive on most x/y axes. The code that handles this actually uses negative numbers as it’s axis is upside down, but this shouldn’t affect relative distances and positive numbers look better on blogs so I’m going to use them.

Also, the velocity for the first two frames is not a typo, it really is set to the same value both frames, and as we’ll see after studying gravity, hitting jump for a single frame gives Mario just enough juice to clear hills that are a single block high.

Maven! Leeggo My Eggo (tests)

Posted in Uncategorized by benjaminplee on 07.23.09

Warning: venting of frustration has commenced.

I ran into a huge problem today.  While trying to find a temporary workarond, I found a much more confusing problem.

While setting up another team’s project in Maven/Continuum, I found that they had a package of Java code which wouldn’t compile (some external API had changed).  This is not good at all b/c A) their code doesn’t work right now and B) it is committed that way to SVN.  For the time being, the idea crossed my mind to modify their pom.xml so that Maven just wouldn’t compile/test the offending package which would at least keep things moving from an infrastructure setup point of view.

A couple quick tries, a few google searches, a few more tries, and a lot of confusion later and I still can’t get Maven’s compiler plugin to ignore any or all of my tests.  Despite what looks to be an exact example of what I want on the plugin’s site here, and a mailing list post here, I doesn’t work.

Modifying the plugin’s main configuration only works to exclude classes from the main source folder.  This is also true if you move the configuration to the “default-compile” execution.  BUT, if you move it to the “default-testCompile” execution, nothing happens.  Ignored.  If I change the configuration to force the tests to be output in Java 1.4 code, it errors b/c they contain annotatoins (what you would expect) but the <excludes> is ignored like it was never there.


^^^ Doesn’t work
Very strange and annoying.

If anyone has any thoughts or ideas, please let me know.  If I ever figure out definitely what is going on, I will post what I find here.

At least I am not alone.

Palm Pre SDK BSOD … Fixed!

Posted in Uncategorized by benjaminplee on 07.16.09

Today I downloaded the Palm Pre SDK (not the leaked version but the real one so far as I could tell) and was all set to try my hand at webOS …. when I restarted my machine to find that I could no longer login to Windows XP. As the logon process started, the screen would flicker and BSOD w/ the following error:

STOP: c000021a {Fatal System Failure}
The Windows Logon Process system process terminated unexpectedly with a status of 0xc0000139

Luckily a coworker burned me a KNOPPIX linux live CD and while sitting in my hotel room and bugging Mr Youngng for linux help I was able to find that several other people had ran into the same issue with the Palm Pre SDK and other software issues. The common culprit seemed to be a mixture of logon based .dll’s. Luckily I was able to copy msvcrt.dll from another installation to my broken one and was able to login once again.

** The one I found in the broken installation was last modified in 2000 (opposed to 2004 for the one that “worked”) and 1/3 smaller **

Hopefully this information will help some other poor fool as foolish as I.

Tagged with:

TECHnc Cleveland 2009 and Ignite

Posted in Uncategorized by benjaminplee on 05.25.09

A great friend of mine is helping coordinate a rather cool event coming up in June in the Cleveland area.  TECHnic as he is calling it is going to be a big summer picnic bringing together anyone who has an interest in technology (namely the members of several Cleveland area technical user groups) for a day of music, food, and nerd talk.  If you are interested, you can find more info here.

A few other friends have mentioned going Ignite events in their home cities.  The basic premise is that a group of people present 5 minute presentations where they have only 20 slides which automatically switch every 15 seconds.  From what I can tell, they end up with some pretty interesting talks.

I would love to attend/help put together an event like either of these in the St. Louis area but with my current travel schedule I don’t know if I could.  Maybe in a couple of months.

maven – give me my files!

Posted in Uncategorized by benjaminplee on 04.16.09

Every time I work with Maven I like a ton of things about it and get really annoyed by a few.  Today I was faced with an issue for the second time: I have a Maven project that produces a .jar as its artifact and I want to have a build goal produce a directory with that .jar AND copies of the .jar’s that it depends on.  Without this, my only means of distributing my code is if the recipient uses Maven also.  I partially solved this a few months ago and today found a more complete solution.  Its not complete (the .jars end up in a horribly ugly directory under the target folder) but it works … and that’s what matters.

The solutions uses the Maven Assembly plugin.  First, I needed a custom assembly configuration which I put in a file called assembly.xml in the root of my project:

<?xml version="1.0" encoding="UTF-8"?>

This file describes how the plugin should assembly my project; moving all of my dependencies, unpacked, into a single folder.  There is a huge assortment of options available ranging from include/exclude parameters to how/where to move files.  I am still slowly working my way through the reference doc.

Second, I modified my pom.xml to configure the Assembly plugin:


Finally, a call to mvn assembly:directory will compile your project, run the tests, package your project and produce the desired folder. Done.

a thankful Level-X programmer

Posted in Uncategorized by benjaminplee on 04.08.09

After tonight’s Google Reader catch-up session, I happened across Jeff Atwood’s post: The Eight Levels of Programmers.  The article reminded me of a joke a friend of mine and I came up with a while back where we debated if all software developers could be given a numbered rank … where the rank corresponded to the number of future software versions required to replace them.  While we were really joking about some especially horrible code we had just found … the sad truth is lots of developers would have some low numbers (assuming companies were willing to pay for the upgraded software).

The article, my current project, the bad economy, and a few recent talks with my wife have got me thinking about how lucky I am.  I get paid to do something I would gladly do for free while working with some amazing people.  My wife, bless her heart, has a hard time understanding how I can work in front of a computer all day and still want to hack away at some personal project that night.  I am just lucky I guess.

Despite the luck, it’s a lot of hard work.  It’s amazing how fast things change and how no matter how many blogs/books/feeds you read, personal projects you work on, or how much experience you have … a year from now it might all be irrelevant. Add on top of that the difficulty of trying to make a name for yourself and honing your personal/communication/team/etc skills and your left with a daunting task.

I am not sure what the future holds, or what level of programmer (on either scale) I am today … but I DO know I want it higher tomorrow and am wiling to put in the time to make it happen.


… at least until the software becomes self aware and take over the world.  Then it’s time to see if hard coding all of those DO_NOT_KILL_LIST.add(“Benjamin P Lee”); really work.


** This is my first post from Windows Live Writer which I am trying upon @brianbuttonxp ‘s suggestion.  So far, so good.

Temp Files = No More Dumb Commits

Posted in Uncategorized by benjaminplee on 03.26.09

While writing a simple utility API used to modify Excel documents I ran into an annoying situation.  My end-to-end integration test was loading a test file, making a modification, verifying that the file was changed, and reversing the modification.  This worked great except that each time I ran the tests (all the time), the file’s timestamp would be changed.  This meant that every time I wanted to commit back to SVN, the file showed up as modified.

After noticing that this simple test file had a few dozen commits associated with it without any real change in values … I needed to find a better solution to setting-up and tearing-down my test data.

Enter File.createTempFile().  While surfing the always helpful Java Almanac I found a nice little tip to on how to create a temporary file in the system’s default temp directory.  According to the javadoc, the method will create a new empty file in the sytem’s default temp directory with a given prefix and file extension.  After the chosen prefix, the utility will generate a new random code.  This combined with File.deleteOnExit() allowed me to have my tests execute against a new file each time that is automatically cleaned up when the JVM closes which saves my commit logs.

It might not be perfect as a long term solution, but for now, it perfectly solved my problem.

First Post!

Posted in Uncategorized by youngnh on 02.28.09