Two Guys Arguing

TestNG Group Gotcha

Posted in software testing by benjaminplee on 03.19.09

I have been forcing myself to use TestNG lately because its what my current client uses and to try something new.  Overall I have been happy with it, but I don’t think I will be switching from JUnit any time soon on projects where I have a choice.  Yes, TestNG has several features which JUnit doesn’t, but JUnit 4 made up a lot of ground from when TestNG was first released in response to the shortcomings of JUnit 3.  Plus, a lot of the features I just don’t think I would use very often.

One feature that did make a lot of sense to me was the idea of test groups.  Basically, the @Test annotation allows you to specify any number of strings as group names that the given test will belong to.  Then, when you want to execute a test suite, you can specify which groups are included or excluded from your suite.  Sounds simple enough and would allow you to divide up your test executions if you wanted to.  So I gave it a shot.

I wrote up a new test, gave it a label of “integration” (distinguishing it from my normal unit tests) and tried to run it.  ERRRR.  NullPointerException.  After several minutes verifying I had things setup correctly I realized the problem was that my @BeforeSuite and @BeforeMethod methods were not being called.  But the the test was working a minute ago ….  a quick check of the docs revealed that all of the @BeforeXXX and @AfterXXX methods ALSO can have a groups setting.  Meaning that when I told TestNG to run just the tests with a group of “integration” I was really telling it to only execute methods with that group.

This wasn’t the behavior I was expecting, but at least I understood what was going on after I read the docs.  If I added the group to my support methods, everything would be fine.  Only one problem.  The @BeforeXXX and @AfterXXX methods in question are in a base class and shouldn’t need to know about any groups a test class wants to use.

Enter the alwaysRun annotation property.  According to the TestNG docs:

For before methods (beforeSuite, beforeTest, beforeTestClass and beforeTestMethod, but not beforeGroups): If set to true, this configuration method will be run regardless of what groups it belongs to.
For after methods (afterSuite, afterClass, …): If set to true, this configuration method will be run even if one or more methods invoked previously failed or was skipped.

By adding this to the base class, the correct methods are run each time.  I am not 100% sure this won’t have other side effects that I haven’t found yet, but as it stands, it allows me to use groups in my tests AND use the base class to setup and teardown some of our common objects.  Its not a perfect solution, but it works.

About these ads

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.