A Tool's Tale

When Noah Sussman asked me to give a Code as Craft Technology Talk last week when I was consulting for Etsy I immediately said yes. However, I was a bit surprised when the talk was announced under the title "An Evening with Sebastian Bergmann". When I read that title the first time, it was just minutes after Arne, Stefan and I had talked about one of our favourite scenes from "A Knight's Tale":

Chaucer: I'm a writer.

Wat: A what?

Chaucer: A wha- a what? A writer. You know, I write, with ink, and parchment. Geoffrey Chaucer's the name, writing's the game. You've probably read my book? The Book of the Duchess? No? Well, it was allegorical.

Roland: Well, we won't hold that against you, that's for every man to decide for himself.

Just like with a talk that I gave last year, I suddenly had a chain of associations in my head that I just had to follow. And down the rabbit hole I went once more ...

Hi! I have no idea why this talk is titled "An Evening with Sebastian Bergmann". I hope that this evening will turn into an interesting discussion about PHPUnit and all things testing. To break the ice, and to not appear completely unprepared, I came up with the following slides ...

So I recently talked with my friends Arne and Stefan about the movie "A Knight's Tale". Somehow the idea stuck in my head that a variation of that title might be a good idea for a talk of mine. After dismissing "A Fool's Tale" I arrived at "A Tool's Tale". So that's what we're stuck with and what I am going to try right now ...




What I just did was a variation on how Geoffrey Chaucer introduces himself in the movie.

Some time in 2001 I started to work on PHPUnit because I wanted to have something like JUnit for PHP. The initial "port" was completed within one weekend. The code was ugly because I had to emulate exceptions which PHP 4 did not have. On November 27th 2001 I checked the code into cvs.php.net.

A lot has changed since 2001. Hopefully not very many developers are still stuck with PHP 4. As of PHPUnit 2.0, which was released on July 14 2004, the day after PHP 5.0.0 was released, PHP 5 is required to run PHPUnit.

PHPUnit is neither the only testing framework nor the only quality assurance tool for PHP. Over the last years a nice ecosystem of static analysis tools started to grow.

I get this question a lot: why do only Germans work on tools that tell me that my code is bad? I do not have an answer for this, sorry. But the statement is also not true: PHP_CodeSniffer is not developed by a German.

PHP has changed and so the have the tools we use to write and maintain code. In 2006 the PHPUnit code was migrated from CVS to Subversion ...

... and in December 2009 from Subversion to Git and GitHub.

PHPUnit is no longer one big monolithic package. It has been refactored to components that are being reused by other testing frameworks, for instance. Unfortunately, this refactoring was not without problems.

PHPUnit gets more and more convenience functionality. Here are a couple of examples:




Tests can be written, both unintentionally and intentionally, in bad ways. Bad tests can lie. They can give you a false sense of security by reporting that something is tested when it really is not.

This is why I started iplementing coutermeasures against bad tests such as the (ultimately useless but still interesting) assertion counting ...

... as well as the strict execution mode.

There is code in PHPUnit that I am not proud of. I am trying to make the world a better place by eliminating one singleton at a time, for instance. Unfortunately, cleaning up code sometimes breaks things. Contrary to what a popular opinion on Twitter is, I do not like breaking backwards compatibility in PHPUnit and try really hard to avoid it.

This is a "good" example of such a backwards compatibility breakage. It happened in PHPUnit 3.6 because I eliminated a Singleton. Had I known that many developers used this API (instead of the XML configuration file) to set up a code coverage blacklist or whitelist I would probably not have done the change. If more people would have tested the release candidates leading up to PHPUnit 3.6 they could have told me ... oh well.

At this point the "Evening with Sebastian Bergmann" turned into the interesting discussion I had hoped for.