Dec 10

Do you annotate your TestNG tests with groups? We currently have four groups: FAST_TEST, SLOW_TEST, INTEGRATION_TEST, WEBDRIVER_TEST.

@Test(groups = TestGroup.FAST_TEST)
public class IncomeCalculatorTest {

  @Test public void test_negative_income_is_not_allowed() {
    ..
  }
}

Recent, we have had trouble with the execution order of our tests. We are using Spring and one or more tests seem to dirty the context without annotating it with @DirtiesContext. We have several hundred tests so identifying the one isn’t that easy. Same problems? Vote for SPR-7811. A workaround is to control the execution order of the tests. Now we are back on our topic.

TestNG allows you to use beanshell scripts inside a suite file. The access is limited to the current method and its groups. Nevertheless you can do a lot with this information.

You want to run tests only, that require a transactional spring context? Or you want tests from a specific package only? Here we go:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="beanshell-demo-suite">
  <test name="allTransactionalSpringContextTests">
    <method-selectors>
      <method-selector>
        <script language="beanshell">
          <![CDATA[org..AbstractT..Tests.class.isAssignableFrom(method.getDeclaringClass())]]>
        </script>
      </method-selector>
    </method-selectors>
    <packages>
      <package name="mycompany.*"/>
    </packages>
  </test>
  ..
  <test name="allTestsInPackageX-Y-Z">
    <method-selectors>
      <method-selector>
        <script language="beanshell">
          <![CDATA[method.getDeclaringClass().getPackage().getName().contains("X-Y-Z")]]>
        </script>
      </method-selector>
    </method-selectors>
    <packages>
      <package name="mycompany.*"/>
    </packages>
  </test>
</suite>

(org..AbstractT..Tests should be org.springframework.test.context.testng.AbstractTransactionalTestNGSpringContextTests)

Got it? If you can group your tests based on there names, packages, base classes, .. you don’t need to annotate all your tests with groups. No more missing or wrong tagged test, hallelujah!

Disclaimer: Sure there will be a lot of situations where you need test groups. E.g. when you use the dependsOn feature heavily. This blog shows what else is possible.

Tagged with:
Oct 09

The Problem
The project structuring of SoapUI (not the Pro version) is limited. When you use the maven-soapui-plugin to automate your tests, you will define multiple plugins – each pointing to a different project file. Your pom.xml will be messed up with sequences like these:

          <plugin>
            <groupId>eviware</groupId>
            <artifactId>maven-soapui-plugin</artifactId>
            <version>${soapui.version}</version>
            <executions>
              <execution>
                <id>soapui-test-xy</id>
                <phase>test</phase>
                <configuration>
                  <projectFile>${basedir}/src/test/soapui/bkr.xml</projectFile>
                  <outputFolder>${project.build.directory}/soapui</outputFolder>
                  <exportAll>true</exportAll>
                  <printReport>true</printReport>
                  <host>localhost</host>
                </configuration>
                <goals>
                  <goal>test</goal>
                </goals>
              </execution>
            </executions>
          </plugin>

The Solution
The solution is very simple. Instead of executing your functional tests with maven just run them from your unit testing framework:

  @Test
  public void test_bkr() throws Exception {
    SoapUITestCaseRunner runner = new SoapUITestCaseRunner();
    runner.setHost("my-host");
    runner.setProjectFile("./src/test/soapui/bkr.xml");
    SoapUiConfigLogger.logConfig(runner);
    runner.run();
  }

Now you have full flexibility of how to organize your tests:

  • package structure
  • test-class name
  • test-method name
  • test-groups (TestNG)
  • ..

I want to test external services that are already running. If you want to test your own services you have to start-up and tear-down jetty. Either you try to use @Before.. and @After.. annotations or you bind your tests to maven’s integration-test and use pre-integration and post-integration for that.

Tagged with:
preload preload preload