The Cost of Test Isolation - Follow-Up

This is a follow-up to "The Cost of Test Isolation (and other PHPUnit Features)".

Since the previous posting, I have added a backup/restore mechanism for static attributes of classes to PHPUnit. This is yet another feature of PHPUnit that makes the testing of code that uses global state (which includes, but is not limited to, global and superglobal variables as well as static attributes of classes) easier.

The following profiling graph illustrates the performance penalty incurred by this feature on the test execution.

Profiling PHPUnit with Xdebug

The following profiling graph was generated for an execution of PHPUnit with the --no-syntax-check --no-globals-backup --no-static-backup switches that disable these features (and also the syntac check of test code).

Profiling PHPUnit with Xdebug

My recommendation is: disable the backup/restore mechanism of globals and static attributes of classes. Not only because this greatly reduces the time required to run the tests. It also exposes tests, and thus the tested code, that interacts with global state. If, and only if, this interaction with global state cannot be avoided you can select the tests for which the backup/restore mechanism(s) should be enabled in a fine-grained way.

If you are interested in the problems associated with code that interacts with global state, especially with regard to testing it, join me for my "Untestable Code session at the php|tek 2009 conference in Chicago, IL, US in May.