Test Dependencies in PHPUnit 3.4
Back in July, I came across an academic paper (more academic papers on testing that I read recently) titled "JExample: Exploiting Dependencies Between Tests to Improve Defect Localization".
"A well-designed test suite should exhibit high coverage to improve our chances of identifying any defects. When tests fail, we want to quickly localize defects, so we want our attention to be focussed on the relevant failing tests to identify the root cause of the defect. However, when some part of the base-code gets changed, a small defect can cause a domino effect of multiple failing unit tests. This is a problem, because the person changing the code has no other option than to browse all failing unit tests to try and deduce a single root cause. This task can prove to be quite difficult when that person is unfamiliar with the test code that fails."
For the upcoming PHPUnit 3.4 I have implemented support for the idea expressed in the paper mentioned above.
Let us have a look at the following code:
<?php
class DependencyFailureTest extends PHPUnit_Framework_TestCase
{
public function testOne()
{
$this->assertTrue(FALSE);
}
/**
* @depends testOne
*/
public function testTwo()
{
}
}
?>
When we run the test above, the execution of the second test will be skipped as the test it depends on (testOne() did not pass:
sb@ubuntu ~ % phpunit --verbose DependencyFailureTest PHPUnit 3.4.0-dev by Sebastian Bergmann. DependencyFailureTest FS Time: 0 seconds There was 1 failure: 1) testOne(DependencyFailureTest) Failed asserting that <boolean:false> is true. /home/sb/DependencyFailureTest.php:6 There was 1 skipped test: 1) testTwo(DependencyFailureTest) This test depends on "DependencyFailureTest::testOne" to pass. FAILURES! Tests: 2, Assertions: 0, Failures: 1, Skipped: 1.
Some details about this new feature:
- A test may have more than one
@dependsannotation. @depends methodreferences the test implemented inmethod()of the same class.@depends class::methodreferences the test implemented inclass::method().- PHPUnit does not change the order in which tests are executed, you have to ensure that the dependencies of a test can actually be met before the test is run.
Please play with this new feature and provide feedback about its usefulness and usability.
14/11/2008 at 09:43 Permalink
Reply
14/11/2008 at 14:03 Permalink
I am wondering if it would not be more convenient if @depends and @dataProvider would accept short forms:
@depends one
@dataProvider two
for
function testOne()
function providerTwo()
Reply
14/11/2008 at 14:18 Permalink
Reply
14/11/2008 at 15:34 Permalink
Will you be adding the functionality to have the returned data from a dependency passes as a parameter to the dependent method. I think this will be really great for reducing code in setUp?
Reply
02/12/2008 at 21:06 Permalink
Reply
16/01/2009 at 19:16 Permalink
In the documentation for markTestSkipped(), (seeing if the mysqli extension was loaded), every test would show as skipped - which is largely redundant after the first instance.
Reply
15/05/2009 at 18:26 Permalink
I find this extension very useful, though i'm still not able to use it: I'm trying to get the version 3.4 of phpunit from pear following the instructions in:
http://www.phpunit.de/manual/3.4/en/installation.html
but i always get the version 3.3.16. How can I get it?
Thanks in advance
Reply
27/08/2009 at 16:33 Permalink
Reply
18/09/2009 at 06:24 Permalink
I just put together a quick test that utilizes @depends and @dataProvider and it's working perfectly. Perhaps there's something else going on in your code that's causing a failure in your specific situation?
David
Reply
24/04/2010 at 12:14 Permalink
in TextUI/ResultPrinter.php:
moved the ``$this->lastTestFailed = FALSE;'' instruction
from endTest (last line of function [635 for my version])
to startTest (inserted before first line [605]).
What about that change? Is it a good fix (or could it possibly induce some bad side effects)?
Thank you
Reply
26/04/2010 at 15:43 Permalink
My "previous message" was not validated, maybe too tangled... Simplified, it was:
Run this test class:
class DependencySkipTest extends PHPUnit_Framework_TestCase
{
public function testA() {$this->assertTrue(FALSE);}
/** @depends testA */
public function testB() {}
public function testC() {$this->assertTrue(TRUE);}
}
Expected output:
FS.
Actual output:
FS
The "." for testC is not printed in PHPUnit_TextUI_ResultPrinter::endTest, because $lastTestFailed has been set to TRUE (in addSkippedTest) when testB was skipped, then has NOT been reset to FALSE before testC (endTest was not called for testB (neither was startTest), which is logical).
Now you can understand what I wanted to "fix" ;)
Hope this helps
Reply