Parallel Test Execution

TODAY Framework 5.0 introduced basic support for executing tests in parallel within a single JVM when using the Infra TestContext Framework. In general, this means that most test classes or test methods can be run in parallel without any changes to test code or configuration.

For details on how to set up parallel test execution, see the documentation for your testing framework, build tool, or IDE.

Keep in mind that the introduction of concurrency into your test suite can result in unexpected side effects, strange runtime behavior, and tests that fail intermittently or seemingly randomly. The Infra Team therefore provides the following general guidelines for when not to run tests in parallel.

Do not run tests in parallel if the tests:

  • Use TODAY Framework’s @DirtiesContext support.

  • Use Infra App’s @MockBean or @SpyBean support.

  • Use JUnit 4’s @FixMethodOrder support or any testing framework feature that is designed to ensure that test methods run in a particular order. Note, however, that this does not apply if entire test classes are run in parallel.

  • Change the state of shared services or systems such as a database, message broker, filesystem, and others. This applies to both embedded and external systems.

If parallel test execution fails with an exception stating that the ApplicationContext for the current test is no longer active, this typically means that the ApplicationContext was removed from the ContextCache in a different thread.

This may be due to the use of @DirtiesContext or due to automatic eviction from the ContextCache. If @DirtiesContext is the culprit, you either need to find a way to avoid using @DirtiesContext or exclude such tests from parallel execution. If the maximum size of the ContextCache has been exceeded, you can increase the maximum size of the cache. See the discussion on context caching for details.
Parallel test execution in the Infra TestContext Framework is only possible if the underlying TestContext implementation provides a copy constructor, as explained in the javadoc for TestContext. The DefaultTestContext used in Infra provides such a constructor. However, if you use a third-party library that provides a custom TestContext implementation, you need to verify that it is suitable for parallel test execution.