In the last post, we walked through how to install HipHop PHP on a Ubuntu 13.04 EC2. Well thats great but it now leads us to the question of how fast HipHop PHP actually is. The problem with “toy benchmarks” is they tend to not really capture the real performance characteristics of whatever you’re benchmarking. This is why comparing the performance of a “Hello World” app across various languages and frameworks is generally a waste of time, since its not capturing a real world scenario. Luckily, I actually have some “real world”‘ish benchmarks from my PHP: Does “big-o” complexity really matter? post a couple of months ago.
Ok so great, lets checkout the repository, run the benchmark with HipHop and Zend PHP, and then marvel at how HipHop blows Zend PHP out of the water.
Wtf?
Well so that is weird, in 3 out of the 4 tests HipHop is an order of magnitude slower than Zend PHP. Clearly, something is definitely not right. I double checked the commands and everything is being run correctly. I started debugging the readMemoryScan function on HipHop specifically and it turns out that the problem function is actually str_getcsv. I decided to remove that function as well as the array_maps() since I wasn’t sure if HipHop would be able to optimize given the anonymous function being passed in. The new algorithms file is algorithms_hiphop.php which has str_getcsv replaced with an explode and array_map replaced with a loop.
Running the same benchmarks again except with the new algorithms file gives you:
Wow. So the HipHop implementation is clearly faster but what’s even more surprising is that the Zend PHP implementation gains a significant speedup just by removing str_getcsv and array_map.
Anyway, as expected, HipHop is a faster implementation most likely due to its JIT compilation and additional optimizations that it’s able to add along the way.
Despite the speedup though, Facebook has made it clear that HipHop will only support a subset of the PHP language, notably that the dynamic features will never be implemented. At the end of the day, its not clear if HipHop will gain any mainstream penetration but hopefully it’ll push Zend to keep improving their interpreter and potentially incorporate some of HipHop’s JIT features.
Per Dan’s comment below, HHVM currently supports almost all of PHP 5.5s features.
Why is str_getcsv so slow?
Well benchmarks are all fine and well but I was curious why str_getcsv was so slow on both Zend and HipHop. Digging around, the HipHop implementation looks like:
So basically just a wrapper around fgetcsv that works by writing the string to a temporary file. I’d expect file operations to be slow but I’m still surprised they’re that slow.
Anyway, looking at the Zend implementation it’s a native C function that calls into php_fgetcsv but doesn’t use temporary files.
Looking at the actual implementation of php_fgetcsv though its not surprising its significantly slower compared to explode().