Do you annotate your TestNG tests with groups? We currently have four groups: FAST_TEST, SLOW_TEST, INTEGRATION_TEST, WEBDRIVER_TEST.

@Test(groups = TestGroup.FAST_TEST)
public class IncomeCalculatorTest {

  @Test public void test_negative_income_is_not_allowed() {
    ..
  }
}

Recent, we have had trouble with the execution order of our tests. We are using Spring and one or more tests seem to dirty the context without annotating it with @DirtiesContext. We have several hundred tests so identifying the one isn’t that easy. Same problems? Vote for SPR-7811. A workaround is to control the execution order of the tests. Now we are back on our topic.

TestNG allows you to use beanshell scripts inside a suite file. The access is limited to the current method and its groups. Nevertheless you can do a lot with this information.

You want to run tests only, that require a transactional spring context? Or you want tests from a specific package only? Here we go:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="beanshell-demo-suite">
  <test name="allTransactionalSpringContextTests">
    <method-selectors>
      <method-selector>
        <script language="beanshell">
          <![CDATA[org..AbstractT..Tests.class.isAssignableFrom(method.getDeclaringClass())]]>
        </script>
      </method-selector>
    </method-selectors>
    <packages>
      <package name="mycompany.*"/>
    </packages>
  </test>
  ..
  <test name="allTestsInPackageX-Y-Z">
    <method-selectors>
      <method-selector>
        <script language="beanshell">
          <![CDATA[method.getDeclaringClass().getPackage().getName().contains("X-Y-Z")]]>
        </script>
      </method-selector>
    </method-selectors>
    <packages>
      <package name="mycompany.*"/>
    </packages>
  </test>
</suite>

(org..AbstractT..Tests should be org.springframework.test.context.testng.AbstractTransactionalTestNGSpringContextTests)

Got it? If you can group your tests based on there names, packages, base classes, .. you don’t need to annotate all your tests with groups. No more missing or wrong tagged test, hallelujah!

Disclaimer: Sure there will be a lot of situations where you need test groups. E.g. when you use the dependsOn feature heavily. This blog shows what else is possible.