Sebastian Bergmann

Benchmarks such as the PHP / GCC / ICC Benchmark I posted quite a while ago on this blog are synthetic. They test "raw bytecode execution" speed that cannot be translated into real-world situations per se.

Flickr recently migrated from PHP 4 to PHP 5. And here are their real world numbers:

Wow.

I am looking forward to John Allspaw's slides from his presentation on "operational efficiency hacks" that he will be giving at next week's Web2.0 Expo.

As people keep asking me for an update to my previous benchmarks (PHP / GCC / ICC Benchmark, PHP 5.1 / GCC Benchmark (Update), and PHP 5.1 Performance, I quickly ran a new benchmark today.

The image above shows a stacked Bar Chart rendered using the Cairo backend of the Graph eZ Component of benchmark data from the PHP branches 3.0 through 5.3-CVS.

• We can clearly see that the introduction of bytecode generation/interpretation in PHP 4 gave a huge performance boost (PHP 4 is 6.5 times faster than PHP 3 for raw execution speed).

• The focus of PHP 5.0 was on features, not performance.
• The focus of PHP 5.1 was on performance and the new executer delivered on its promise (PHP 5.1 is 2.5 times faster than PHP 4.4 and PHP 5.0 for raw execution speed).
• The focus of PHP 5.2 was on stability.

• PHP 5.3 brings new features (namespaces, late statics binding, and the garbage collector, for instance) but the refactoring of the argument stack, for example, and other optimization patches have lead to a measurable performance improvement (PHP 5.3 is 1.2 times faster than PHP 5.2 for raw execution speed).

A modified version of the Zend/bench.php that is compatible with earlier versions of PHP has been used during the benchmark.

The PHP binaries were compiled using gcc (GCC) 4.2.3 (Ubuntu 4.2.3-1ubuntu1) and CFLAGS="-march=native -mtune=native -O2 -pipe" on an IBM ThinkPad X60s with an Intel Core Duo L2400 CPU running Linux 2.6.24-5-generic (Ubuntu Hardy Heron).

The extended body of this blog entry contains the raw benchmark data.

Last month I got a bit bored and built PHP 4.4.4, PHP 5.0.5, PHP 5.1.6, and current PHP_5_2 (the last two each with CALL, GOTO, and SWITCH VMs) with GCC 3.4.6 and GCC 4.1.1 with -O{0|1|2|3|s}. Yes, this means I built 80 PHP binaries. No, I did not do this manually.

Below are the results of running bench.php with each of the binaries that I built.



As you can see, PHP 5.1 and PHP 5.2 are both around three times faster than PHP 4.4 and PHP 5.0 (GCC 3.4.6, -O2).



Each binary ran the benchmark script five times, the numbers shown are the respective mean from the five runs.

The numbers for PHP 5.1.6 and PHP 5.2 are based upon the CALL-based virtual machine as GOTO and SWITCH did not work for most of the optimization levels.

Update: Yesterday I built PHP 4.4.4, PHP 5.0.5, PHP 5.1.6, and PHP 5.2.0 with ICC 9.1.042 using -O{0|1|2|3|s} and -fast.

Below is a chart that directly compares GCC 3.4.6, GCC 4.1.1, and ICC 9.1.042.



The GCC versions used were Gentoo sys-devel/gcc-4.1.1-r1 and Gentoo sys-devel/gcc-3.4.6-r2 together with Gentoo sys-devel/binutils-2.17.

The CFLAGS used for GCC were "-march=pentium-m -msse3 -O{0|1|2|3|s} -pipe".

The configure options used were "--disable-all --disable-cgi".

The extended body of this posting contains the raw test results as well as detailed information on the hardware used to run the benchmarks.

I updated my previous benchmark of PHP 5.1. This time around I built PHP with GCC 3.4.5, GCC 4.0.2, and GCC 4.1.0.

The bottom line result seems to be that GCC 4 generates slightly slower code:

GCC 3.4.5: 15.223
GCC 4.0.2: 17.896
GCC 4.1.0: 17.847

When I have more time, I will investigate this further using alternative optimizations, including profiling-guided ones.