Test Dependencies in PHPUnit 3.4

Back in July, I came across an academic paper (more academic papers on testing that I read recently) titled "JExample: Exploiting Dependencies Between Tests to Improve Defect Localization".

"A well-designed test suite should exhibit high coverage to improve our chances of identifying any defects. When tests fail, we want to quickly localize defects, so we want our attention to be focussed on the relevant failing tests to identify the root cause of the defect. However, when some part of the base-code gets changed, a small defect can cause a domino effect of multiple failing unit tests. This is a problem, because the person changing the code has no other option than to browse all failing unit tests to try and deduce a single root cause. This task can prove to be quite difficult when that person is unfamiliar with the test code that fails."

For the upcoming PHPUnit 3.4 I have implemented support for the idea expressed in the paper mentioned above.

Let us have a look at the following code:

<?php
class DependencyFailureTest extends PHPUnit_Framework_TestCase
{
    public function testOne()
    {
        $this->assertTrue(FALSE);
    }
 
    /**
     * @depends testOne
     */
    public function testTwo()
    {
    }
}
?>

When we run the test above, the execution of the second test will be skipped as the test it depends on (testOne() did not pass:

sb@ubuntu ~ % phpunit --verbose DependencyFailureTest
PHPUnit 3.4.0-dev by Sebastian Bergmann.

DependencyFailureTest
FS

Time: 0 seconds

There was 1 failure:

1) testOne(DependencyFailureTest)
Failed asserting that <boolean:false> is true.
/home/sb/DependencyFailureTest.php:6

There was 1 skipped test:

1) testTwo(DependencyFailureTest)
This test depends on "DependencyFailureTest::testOne" to pass.

FAILURES!
Tests: 2, Assertions: 0, Failures: 1, Skipped: 1.

Some details about this new feature:

  • A test may have more than one @depends annotation.
  • @depends method references the test implemented in method() of the same class.
  • @depends class::method references the test implemented in class::method().
  • PHPUnit does not change the order in which tests are executed, you have to ensure that the dependencies of a test can actually be met before the test is run.

Please play with this new feature and provide feedback about its usefulness and usability.