Tag Archives: benchmarking
Benchmarking Asyncronous PHP vs NodeJS Properly
I recently read Phil Sturgeon's Benchmarking Codswallop: NodeJS v PHP and was interested to see that PHP performed pretty well compared to NodeJS once it was run asynchronously.
So we decided to run the test ourselves and he would fix the NodeJS settings. I would follow Phil's instructions for the PHP side as best as possible.
We used the same VPS. It was a 512MB droplet from DigitalOcean in NY2.
If you are wondering, to get ReactPHP running with composer, simply do the following once you're in the folder with all the files:
curl -sS https://getcomposer.org/installer | php php composer.phar install
It's not in Phil's instructions, but that's the missing part he alludes to.
Blocking PHP Results:
real 3m21.114s user 0m8.217s sys 0m0.496s
Non-Blocking PHP (ReactPHP) Results:
real 0m11.971s user 0m10.897s sys 0m0.704s
The PHP results were fairly similar to Phil's result. That's good, because we haven't changed anything and tried to replicate his experiment.
I used two versions of Node for the test - the latest stable tag at 0.10.22, and the latest unstable, at 0.11.8.
Around version 0.4, Node added connection pooling and a concept called an Agent. An agent can be added to an individual http request, or to all requests on the server. For example, if I had a free or limited account on some API, I might not want to hit it with more than 10 simultaneous connections at a time. In that case, I would assign an Agent to the request and set its `maxSockets` property to 10.
Unfortunately, the `globalAgent`, which runs for all http requests without an explicit Agent (which is what you do about 99% of the time), has a limit of 5. That's right - just 5. One of Node's biggest selling points is speed, yet outgoing requests are completely hamstrung by this absurd opt-out connection pooling scheme that most people miss. It's not like these are expensive database connections - these are outgoing http requests, and all of them share the same 5-connection limit.
As a general rule, setting `http.globalAgent.maxSockets=Infinity` is what you want. It's one of the first things I do in any program that makes outgoing requests.
Of course, I'm not the first person to rant about this; not by a long shot. From famed NodeJS developer substack:
This module disables a lot of infuriating things about core http that WILL cause bugs in your application if you think of http as just another kind of stream: http requests have a default idle timeout of 2 minutes. This is terrible if you just want to pipe together a bunch of persistent backend processes over http. There is a default connection pool of 5 requests. If you have 5 or more extant http requests, any additional requests will HANG for NO GOOD REASON. hyperquest turns these annoyances off so you can just pretend that core http is a fancier version of tcp and not the horrible monstrosity that it actually is. I have it on good authority that these annoyances will be fixed in node 0.12.
Luckily, this behavior is now fixed as of about four months ago, and the last few Node unstable releases contain the fix. The next stable release, Node 0.12 is around the corner and thankfully will make a big difference in lots of users' applications.
v0.10.22 (default maxSockets=5)
real 0m30.691s user 0m10.089s sys 0m0.628s
v0.10.22 (maxSockets=64, mentioned in the followup article)
real 0m11.299s user 0m9.973s sys 0m0.768s
v0.10.22 (maxSockets=Infinity, this is what most servers should be running)
real 0m10.339s user 0m9.269s sys 0m0.624s
v0.11.8 (default is maxSockets=Infinity)
real 0m7.480s user 0m6.372s sys 0m0.680s
Node v0.11.x brings along with it a host of http optimizations that make it even faster than v0.10.22 without connection pooling.
At this point, as Phil Sturgeon mentioned, this is more of a network test than anything else; although latest Node is about 60% faster in this test than ReactPHP. Of course, when choosing a framework, one must consider much more than raw speed in singular benchmarks, but they can be entertaining.
We thought it best, given the popularity of the original post, to set the record straight about how it would perform if the NodeJS code were corrected. In general, if you're running Node v0.10 or below, set `http.globalAgent.maxSockets=Infinity` (and the same for https, if you use it), unless you absolutely have a need for connection pooling.