An oft-forgotten feature of PHP is its shared-nothing architecture as pointed out in this pro-PHP rant (countering all the PHP bashing going on at the moment):

...the important thing to take from that (compared to mod_perl / mod_python / mod_* or even “X” application server.) is the interpreter returning to a fresh state after every request (no globals hanging around or otherwise)

A fresh state every time - which means you can load balance across any number of PHP servers without the complexity of session state synchronisation or server affinity. Infinite horizontal scaling.

Addendum: Another benefit of fresh execution state on each request is that it's very hard to crash the system, which goes a long way to explain why PHP is ubiquitous in cheap hosting environments:

With PHP it’s very hard for a script to take down the runtime environment—the web server—I’d argue that you’d have to be deliberately trying to do so, perhaps filling up disk space or otherwise. Innocent mistakes, specific instances of runtime problems (e.g. script execution too long) and bugs remain local to specific requests and the PHP script handling them. On the next request, we begin again from scratch.