Category Archives: Web Development

WPPerformanceTester – A WordPress Plugin to Benchmark Server Performance

Everyone who read our most popular blog post, WordPress Hosting Performance Benchmarks may have noticed a new test this year (2015) called WPPerformanceTester. It was something I built during the tests to add a new benchmark to see what the underlying performance of the server the test websites were hosted on. It wasn't hugely meaningful because I had no real basis to compare from except the benchmarks I had just generated. So it really played no role in the actual rankings and outcomes of the testing.

But the vision for it and value has slowly become more apparent. In my testing, Pagely had an unusually slow WordPress benchmark (testing WordPress database functions). It was acknowledged by their team and they have since announced a migration to a newer Amazon technology called Aurora which gave Pagely a 3-4x performance increase.

So without further ado, I'd like to announce WPPerformanceTester is now live on GitHub and licensed under the GPLv3. All problems, errors and issues should be submitted on GitHub.

What Tests Does WPPerformanceTester Run?

  • Math - 100,000 math function tests
  • String Manipulation - 100,000 string manipulation tests
  • Loops - 1,000,000 loop iterations
  • Conditionals - 1,000,000 conditional logic checks
  • MySql (connect, select, version, encode) - basic mysql functions and 1,000,000 ENCODE() iterations
  • $wpdb - 250 insert, select, update and delete operations through $wpdb

Industry Benchmarks

WPPerformanceTester also allows you to see how your server's performance stacks up against our industry benchmark. Our industry benchmark is the average of all submitted test results. After you run WPPerformanceTester, you will have the option to submit the benchmark with or without writing a review of your web host. Please consider submitting without a review so that our benchmark improves. If you feel inclined to write a review, please feel free. They will be published in an upcoming project that ties together many of the projects I've been working on here at Review Signal.

Please Note

WPPerformanceTester is a single node testing tool (if you're running a distributed/clustered system it will not give a complete picture, but only of the servers that execution touches.

Furthermore, WPPerformanceTester is not the be-all and end-all of performance testing or web host quality. Our WordPress Hosting Performance Benchmarks performs a variety of tests and that only gives insight into performance. It doesn't look at customer service quality, pricing, and other important dimensions of a good web hosting service.

WPPerformanceTester should be used as one tool in a performance toolbox. I hope it's valuable and helpful, but please keep in mind the larger picture as well. If you care about service quality, we also maintain the largest web hosting review database. My goal is to cover every aspect, and WPPerformanceTester marks a small step in that direction of being able to give consumers a complete picture of web hosting quality in the WordPress space.

Benchmarking Asyncronous PHP vs NodeJS Properly

This article was jointly written by Samuel Reed, a freelance JavaScript developer, and Kevin Ohashi, Founder of Review Signal. NodeJS testing was handled by Samuel Reed and PHP by Kevin Ohashi on the same VPS.

I recently read Phil Sturgeon's Benchmarking Codswallop: NodeJS v PHP and was interested to see that PHP performed pretty well compared to NodeJS once it was run asynchronously.

My first reaction was to share the article with my good friend Samuel Reed who is a JavaScript and NodeJS lover to see what his thoughts were. He instantly wrote back to me "Not that it matters, but that bench was completely broken node-side, we have an absurd connection pooling default."

So we decided to run the test ourselves and he would fix the NodeJS settings. I would follow Phil's instructions for the PHP side as best as possible.

The Server

We used the same VPS. It was a 512MB droplet from DigitalOcean in NY2.

PHP

If you are wondering, to get ReactPHP running with composer, simply do the following once you're in the folder with all the files:

curl -sS https://getcomposer.org/installer | php

php composer.phar install

It's not in Phil's instructions, but that's the missing part he alludes to.

Blocking PHP Results:

real 3m21.114s
user 0m8.217s
sys 0m0.496s

Non-Blocking PHP (ReactPHP) Results:

real 0m11.971s
user 0m10.897s
sys 0m0.704s

The PHP results were fairly similar to Phil's result. That's good, because we haven't changed anything and tried to replicate his experiment.

NodeJS

I used two versions of Node for the test - the latest stable tag at 0.10.22, and the latest unstable, at 0.11.8.

Around version 0.4, Node added connection pooling and a concept called an Agent. An agent can be added to an individual http request, or to all requests on the server. For example, if I had a free or limited account on some API, I might not want to hit it with more than 10 simultaneous connections at a time. In that case, I would assign an Agent to the request and set its `maxSockets` property to 10.

Unfortunately, the `globalAgent`, which runs for all http requests without an explicit Agent (which is what you do about 99% of the time), has a limit of 5. That's right - just 5. One of Node's biggest selling points is speed, yet outgoing requests are completely hamstrung by this absurd opt-out connection pooling scheme that most people miss. It's not like these are expensive database connections - these are outgoing http requests, and all of them share the same 5-connection limit.

As a general rule, setting `http.globalAgent.maxSockets=Infinity` is what you want. It's one of the first things I do in any program that makes outgoing requests.

Of course, I'm not the first person to rant about this; not by a long shot. From famed NodeJS developer substack:

This module disables a lot of infuriating things about core http that WILL cause 
bugs in your application if you think of http as just another kind of stream:

http requests have a default idle timeout of 2 minutes. This is terrible if you 
just want to pipe together a bunch of persistent backend processes over http.

There is a default connection pool of 5 requests. If you have 5 or more extant 
http requests, any additional requests will HANG for NO GOOD REASON.

hyperquest turns these annoyances off so you can just pretend that core http is 
a fancier version of tcp and not the horrible monstrosity that it actually is.

I have it on good authority that these annoyances will be fixed in node 0.12.

Luckily, this behavior is now fixed as of about four months ago, and the last few Node unstable releases contain the fix. The next stable release, Node 0.12 is around the corner and thankfully will make a big difference in lots of users' applications.

NodeJS Results

v0.10.22 (default maxSockets=5)

real 0m30.691s
user 0m10.089s
sys 0m0.628s

v0.10.22 (maxSockets=64, mentioned in the followup article)

real 0m11.299s
user 0m9.973s
sys 0m0.768s

v0.10.22 (maxSockets=Infinity, this is what most servers should be running)

real 0m10.339s
user 0m9.269s
sys 0m0.624s

v0.11.8 (default is maxSockets=Infinity)

real 0m7.480s
user 0m6.372s
sys 0m0.680s

Node v0.11.x brings along with it a host of http optimizations that make it even faster than v0.10.22 without connection pooling.

Conclusion

At this point, as Phil Sturgeon mentioned, this is more of a network test than anything else; although latest Node is about 60% faster in this test than ReactPHP. Of course, when choosing a framework, one must consider much more than raw speed in singular benchmarks, but they can be entertaining.

We thought it best, given the popularity of the original post, to set the record straight about how it would perform if the NodeJS code were corrected. In general, if you're running Node v0.10 or below, set `http.globalAgent.maxSockets=Infinity` (and the same for https, if you use it), unless you absolutely have a need for connection pooling.

Loading...

Interested in seeing which web hosting companies people love (and hate!)? Click here and find out how your web host stacks up.