Tag Archives: benchmarks

WordPress.com VIP Review (2018)

WordPress.com VIP participated for the second time in our WordPress Hosting Performance Benchmarks (2018). This review is based off the results of that test. This year WordPress.com VIP entered in the same Enterprise price tier again.

In the previous test, WP.com VIP aced the Enterprise test and earned the Top Tier WordPress Hosting Performance award from Review Signal. Considering who is behind it, the expectations behind WP.com VIP are enormous.

The Product

Plan Name Plan Monthly Price Plan Visitors Allowed Plan Memory/RAM Plan Disk Space Plan Bandwidth
Plan Sites Allowed
WordPress.com VIP $5,000 Unlimited Unlimited Unlimited Unlimited 5

Performance Review

Load Storm

Plan Tier Total Requests Total Errors Peak RPS Average RPS Peak Response Time(ms) Average Response Time (ms) Total Data Transferred (GB) Peak Throughput (MB/s) Average Throughput (MB/s)
Enterprise 1,749,966 3 1,400.53 972.2 7,484 108 147.94 118.68 82.19

The Load Storm test is designed to simulate real users coming to the site, logging in and browsing the site bursting some of the caching mechanisms typically found on managed WordPress hosts.

Three errors out of 1.75 million requests. VIP also had the fastest average response time at 108ms. Simply astonishing.

Load Impact

Plan Tier Requests Errors Data Transferred (GB) Peak Average Load Time (Seconds) Peak Average Bandwidth (Mbps)
Peak Average Requests/Sec
Enterprise 2085775 6 119.88 0.102 1860 3960

The Load Impact test makes sure static caching is effective so that if a page gets a lot of traffic the site will keep responding without issue.

Six errors in over 2 million requests is another top notch performance. VIP also had the lowest average response time of any company.

Uptime

Plan Tier UptimeRobot StatusCake
Enterprise 99.95 100

The uptime results were great. 100 and 99.95% is very respectable.

WebPageTest / WPPerformanceTester

Plan Tier Enterprise
Dulles 0.334
Denver 1.112
LA 0.649
London 0.444
Frankfurt 0.353
Rose Hill, Mauritius 0.874
Singapore 0.395
Mumbai 0.544
Japan 0.358
Sydney 0.506
Brazil 0.363

I rarely comment about WebPageTest results but WordPress.com VIP deserves a special mention here. There were 11 test locations and VIP was the fastest in 9/11 and was 22 ms and 9 ms behind the fastest in the other two test locations. I've never seen any company be so dominant across the globe in terms of performance.

Plan Tier PHP Bench WP Bench
Enterprise 9.18 798.084597

The WPPerformanceTester results were normal.

Conclusion

WordPress.com VIP's sophomore performance didn't experience a slump. In fact, they seemed to have stepped up their game. Not only did they put on a better Load Storm performance, their response times across the globe were fantastic. WordPress.com VIP earned themselves a second Top Tier WordPress Hosting Performance award this year and was quite a stand out in many regards in a very crowded and well performing field.

WP Engine Review (2018)

WPEngine [Reviews] participated for the fifth time in our WordPress Hosting Performance Benchmarks (2018). This review is based off the results of that test. This year WPEngine entered in the Enterprise tier again. In the previous test, WPEngine struggled with the load tests. They returned this year and entered with the most expensive offering in the Enterprise tier and earned themselves a Top Tier WordPress Hosting Performance award.

The Product

Plan Name Plan Monthly Price Plan Visitors Allowed Plan Memory/RAM Plan Disk Space Plan Bandwidth
Plan Sites Allowed
Premium Dedicated $5,400 Capable of 10M+ (no hard limit, depends on site) N/A 1TB Unlimited 150

Performance Review

Load Storm

Plan Tier Total Requests Total Errors Peak RPS Average RPS Peak Response Time(ms) Average Response Time (ms) Total Data Transferred (GB) Peak Throughput (MB/s) Average Throughput (MB/s)
Enterprise 1,914,457 786 1,564.62 1,063.59 11,541 509 115.54 95.23 64.19

The Load Storm test is designed to simulate real users coming to the site, logging in and browsing the site bursting some of the caching mechanisms typically found on managed WordPress hosts.

WPEngine's Load Storm test was good. Minimal errors and stable response time.

Load Impact

Plan Tier Requests Errors Data Transferred (GB) Peak Average Load Time (Seconds) Peak Average Bandwidth (Mbps)
Peak Average Requests/Sec
Enterprise 1461819 0 78 0.665 1210 2780

The Load Impact test makes sure static caching is effective so that if a page gets a lot of traffic the site will keep responding without issue.

Zero errors, that's the magic number we love to see. WP Engine aced the Load Impact test.

Uptime

Plan Tier UptimeRobot StatusCake
Enterprise 99.95 99.96

WPEngine [Reviews]  maintained above 99.9% uptime on both monitors.

WebPageTest / WPPerformanceTester

Plan Tier Enterprise
Dulles 0.436
Denver 1.155
LA 0.948
London 1.155
Frankfurt 1
Rose Hill, Mauritius 2.109
Singapore 1.536
Mumbai 1.891
Japan 1.062
Sydney 1.428
Brazil 1.238

The response times on WPT were normal.

Plan Tier PHP Bench WP Bench
Enterprise 10.766 418.4100418

The WPPerformanceTester results normal.

Conclusion

After a disappointing 2016 showing, WPEngine [Reviews]  stepped up their game and earned a Top Tier WordPress Hosting Performance award from Review Signal. It's always good to see companies working to improve their service offering. WPEngine is one of the oldest companies in the WordPress hosting space and they are showing they can support even the largest clients as many new competitors spring up. Great performance from WP Engine.

Presslabs Review (2018)

Presslabs participated for the fourth time in our WordPress Hosting Performance Benchmarks (2018). This review is based off the results of that test. Presslabs entered a plan into the Enterprise category, one level up from last year's $201-500 range. Last year Presslabs earned an honorable mention in the $201-500 price tier.

The Product

Plan Name Plan Monthly Price Plan Visitors Allowed Plan Memory/RAM Plan Disk Space Plan Bandwidth
Plan Sites Allowed
Enterprise $2599 USD + $0.1/1000 page views 0 included, all traffic is charged with $0.1/1000 page views Unmetered Unmetered Unmetered
up to 25 (depending on traffic and requirements)

Performance Review

Load Storm

Plan Tier Total Requests Total Errors Peak RPS Average RPS Peak Response Time(ms) Average Response Time (ms) Total Data Transferred (GB) Peak Throughput (MB/s) Average Throughput (MB/s)
Enterprise 2,012,822 56 1,576.23 1,118.23 10,099 208 116.54 90.85 64.75

The Load Storm test is designed to simulate real users coming to the site, logging in and browsing the site bursting some of the caching mechanisms typically found on managed WordPress hosts.

An excellent performance on Load Storm from Presslabs. No signs of load and minimal errors.

Load Impact

Plan Tier Requests Errors Data Transferred (GB) Peak Average Load Time (Seconds) Peak Average Bandwidth (Mbps)
Peak Average Requests/Sec
Enterprise 1991403 0 104.96 0.184 1980 4580

The Load Impact test makes sure static caching is effective so that if a page gets a lot of traffic the site will keep responding without issue.

Zero errors and sub 200ms average response time. Excellent.

Uptime

Plan Tier UptimeRobot StatusCake
Enterprise 100 100

Perfect.

WebPageTest / WPPerformanceTester

Plan Tier Enterprise
Dulles 0.378
Denver 1.352
LA 0.85
London 0.49
Frankfurt 0.381
Rose Hill, Mauritius 1.335
Singapore 0.485
Mumbai 0.721
Japan 0.66
Sydney 0.551
Brazil 1.336

Presslabs' WPT results were good.

Plan Tier PHP Bench WP Bench
Enterprise 7.215 823.0452675

The WPPerformanceTester results had the fastest PHP Bench and second fastest WP Bench.

Conclusion

Presslabs stepped up a price tier and improved from honorable mention last year to earning Top Tier WordPress Hosting Performance award in 2018 at the Enterprise tier. It's nice to see another worthy competitor in the Enterprise space that can handle large sites.

WP Oven Review (2018)

WP Oven participated for the third time in our WordPress Hosting Performance Benchmarks (2018). This review is based off the results of that test. This year WP Oven moved up a tier to the $51-100 price tier.

In the previous test, WP oven achieved perfection and earned themselves Top Tier status from Review Signal. They had zero errors on every test and perfect uptime.

The Product

Plan Name Plan Monthly Price Plan Visitors Allowed Plan Memory/RAM Plan Disk Space Plan Bandwidth
Plan Sites Allowed
Professional $79.95 Unlimited 8GB 80GB 8 TB Unlimited Sites

Performance Review

Load Storm

Plan Tier Total Requests Total Errors Peak RPS Average RPS Peak Response Time(ms) Average Response Time (ms) Total Data Transferred (GB) Peak Throughput (MB/s) Average Throughput (MB/s)
$51-100 438,758 1 342.95 243.75 10,051 370 30.96 25.71 17.2

The Load Storm test is designed to simulate real users coming to the site, logging in and browsing the site bursting some of the caching mechanisms typically found on managed WordPress hosts.

WP Oven had a whopping single error on Load Storm and kept a sub 400ms response time. This was the fewest of any company in the price tier. An excellent showing.

Load Impact

Plan Tier Requests Errors Data Transferred (GB) Peak Average Load Time (Seconds) Peak Average Bandwidth (Mbps)
Peak Average Requests/Sec
$51-100 675881 0 36.07 0.398 619 1380

The Load Impact test makes sure static caching is effective so that if a page gets a lot of traffic the site will keep responding without issue.

WP Oven had zero errors and kept a sub 400ms response time. Perfect.

Uptime

Plan Tier UptimeRobot StatusCake
$51-100 99.98 99.97

Both monitors were above 99.9%, which is good.

WebPageTest / WPPerformanceTester

Plan Tier $51-100
Dulles 0.406
Denver 1.118
LA 0.739
London 1.124
Frankfurt 1.206
Rose Hill, Mauritius 2.04
Singapore 2.103
Mumbai 2.221
Japan 1.421
Sydney 2.099
Brazil 1.316

The WPT results look normal.

Plan Tier PHP Bench WP Bench
$51-100 9.634 211.1040743

The WPPerformanceTester results are normal.

Conclusion

WP Oven proved they could step up a price tier in 2018 and still earn itself Review Signal's Top Tier WordPress Hosting Performance award. It's nice to see a relatively unknown company do consistently well in our benchmarks. Another year, another great performance, keep up the great work WP Oven!

Kinsta Review (2018)

Kinsta participated for the fourth time in our WordPress Hosting Performance Benchmarks (2018). This review is based off the results of that test. This year Kinsta participated in the 25-50, 51-100, 101-200, 201-500, and Enterprise price tiers.

In the previous test, Kinsta earned Top Tier status in all four price brackets it entered.  In 2018, Kinsta started offering cheaper plans and has competed in an additional, lower price tier.

The Products

Plan Name Plan Monthly Price Plan Visitors Allowed Plan Memory/RAM Plan Disk Space Plan Bandwidth
Plan Sites Allowed
Starter $30 20000 - 5GB Unlimited 1
Pro $60 40,000 - 5GB Unlimited 2
Business 1 $100 100,000 - 10GB Unlimited 3
Business 2 $200 250000 - 20GB SSD Unlimited 10
Enterprise 1 $600 1 Million - 80GB SSD Unlimited 60

Performance Review

Load Storm

Plan Tier Total Requests Total Errors Peak RPS Average RPS Peak Response Time(ms) Average Response Time (ms) Total Data Transferred (GB) Peak Throughput (MB/s) Average Throughput (MB/s)
$25-50 294,176 3 225.5 163.43 10,036 355 21.38 17.58 11.88
$51-100 425,597 1 332.65 236.44 10,017 377 30.98 25.62 17.21
$101-200 556,007 48 436.15 308.89 10,101 375 40.45 32.44 22.47
$200-500 688,856 0 543.12 382.7 4,407 358 50.08 40.47 27.82
Enterprise 1,720,465 0 1,391.42 955.81 3,906 417 112.7 91.9 62.61

The Load Storm test is designed to simulate real users coming to the site, logging in and browsing the site bursting some of the caching mechanisms typically found on managed WordPress hosts.

Kinsta aced the Load Storm tests across every tier. Zero errors in multiple price tiers and minimal errors in the others. The response times were kept fast and stable. It scaled up from the smallest test to Enterprise test without issue.

Load Impact

Plan Tier Requests Errors Data Transferred (GB) Peak Average Load Time (Seconds) Peak Average Bandwidth (Mbps)
Peak Average Requests/Sec
$25-50 337031 0 18.05 0.41 304 694
$51-100 672240 0 36 0.416 583 1300
$101-200 1016163 0 54.42 0.395 863 1920
$200-500 1014718 0 54.34 0.411 866 1930
Enterprise 1367747 0 73.4 0.802 1120 2540

The Load Impact test makes sure static caching is effective so that if a page gets a lot of traffic the site will keep responding without issue.

Perfection. Zero errors across 5 separate tests.

Uptime

Plan Tier UptimeRobot StatusCake
$25-50 100 100
$51-100 100 100
$101-200 100 100
$200-500 100 100
Enterprise 100 100

Perfect. 100% across 5 plans and two separate monitors.

WebPageTest / WPPerformanceTester

Plan Tier $25-50 $51-100 $101-200 $200-500 Enterprise
Dulles 0.387 0.373 0.377 0.396 0.382
Denver 1.287 1.163 1.411 1.134 1.245
LA 0.833 0.79 0.81 0.832 1.154
London 1.076 1.104 1.073 1.177 1.331
Frankfurt 1.014 1.009 1.01 1.031 1.103
Rose Hill, Mauritius 2.436 2.446 2.433 2.439 2.444
Singapore 1.902 1.729 1.728 1.712 1.774
Mumbai 2.114 2.279 2.221 2.119 2.108
Japan 1.186 1.183 1.234 1.268 1.281
Sydney 1.609 1.766 1.763 1.772 1.611
Brazil 1.513 1.364 1.384 1.508 1.404

WPT test results looked normal.

Plan Tier PHP Bench WP Bench
$25-50 8.853 1036.26943
$51-100 8.783 1173.70892
$101-200 8.974 1100.110011
$200-500 9.048 1047.120419
Enterprise 8.787 1207.729469

The WPPerformanceTester results looked normal. The WP Bench results stood out though. The database connections are consistently very fast compared to their competitors at higher price tiers when plans tend to run separate and/or multiple database servers.

Conclusion

Kinsta earned itself five Top Tier WordPress Hosting Performance Awards in 2018. It had perfect uptime across every plan. It had zero errors on Load Impact. It had 2/5 Load Storm tests without any errors and only 52 errors total from 3.685 million requests. No signs of stress showed on any test. Kinsta has consistently brought their A game to WordPress Hosting Performance Benchmarks, this year was no exception. It's nice to see them offering newer, cheaper plans that allow people on smaller budgets access their offerings. A huge congratulations to them for earning the most Top Tier Hosting Performance awards of any company in 2018.

Pantheon Review (2018)

Pantheon participated for the fourth time in our WordPress Hosting Performance Benchmarks (2018). Last year Pantheon earned two Top Tier awards. This review is based off the results of this years test.

Pantheon stepped up their game this year and earned three Top Tier awards and one honorable mention. Pantheon also has the honor of being the highest rated company on Review Signal for reviews based on customer opinions (84%).

The Products

Plan Name Plan Monthly Price Plan Visitors Allowed Plan Memory/RAM Plan Disk Space Plan Bandwidth
Plan Sites Allowed
Personal Plan $25 10,000 (with no overage charges) 1024MB 5GB Unlimited 1
Professional $100 100,000 2048 20GB Unlimited 1
Business $400 500,000 8192 30GB Unlimited 1
Elite $1,667 Unlimited Unlimited 100GB+ Unlimited 1

Performance Review

Load Storm

Plan Tier Total Requests Total Errors Peak RPS Average RPS Peak Response Time(ms) Average Response Time (ms) Total Data Transferred (GB) Peak Throughput (MB/s) Average Throughput (MB/s)
$25-50 292,393 5 220.72 162.44 4,287 162 22.79 17.5 12.66
$51-100 445,581 37 336.4 247.54 15,086 172 33.78 26.65 18.77
$200-$500 702,823 7,065 544.27 390.46 11,462 455 52.65 43.64 29.25
Enterprise 1,370,325 88 1,088.48 761.29 7,147 154 106.71 85.72 59.28

The Load Storm test is designed to simulate real users coming to the site, logging in and browsing the site bursting some of the caching mechanisms typically found on managed WordPress hosts.

Pantheon overall did very well. The only issue was in the $200-500 price range it started to show signs of load. It wasn't a huge issue and the plan still earned an honorable mention in the price tier.

Load Impact

Plan Tier Requests Errors Data Transferred (GB) Peak Average Load Time (Seconds) Peak Average Bandwidth (Mbps)
Peak Average Requests/Sec
$25-50 426896 0 23.73 0.159 401 880
$51-100 847397 0 47.26 0.196 773 1650
$200-$500 1257940 0 69.92 0.173 1020 2240
Enterprise 2046874 0 113.47 0.181 1660 3650

The Load Impact test makes sure static caching is effective so that if a page gets a lot of traffic the site will keep responding without issue.

Pantheon aced the LoadImpact tests across every price tier. Zero errors on all four tests and maintained a sub 200 millisecond response time average. Couldn't ask for a better performance.

Uptime

Plan Tier UptimeRobot StatusCake
$25-50 100 100
$51-100 100 100
$200-$500 99.97 100
Enterprise 100 100

Not a whole lot to say here other than excellent. 100% uptime across the board except one monitor out of two showing 99.97%.

WebPageTest / WPPerformanceTester

Plan Tier $25-50 $51-100 $200-$500 Enterprise
Dulles 0.355 0.354 0.353 0.374
Denver 1.57 1.493 1.236 1.322
LA 0.797 0.804 0.804 0.712
London 0.448 0.446 0.45 0.474
Frankfurt 0.369 0.356 0.375 0.428
Rose Hill, Mauritius 1.093 1.074 1.088 0.912
Singapore 0.433 0.449 0.045 0.426
Mumbai 0.692 0.64 0.662 0.522
Japan 0.419 0.39 0.393 0.366
Sydney 0.528 0.503 0.516 0.497
Brazil 0.405 0.408 0.4 0.384

The WPT tests were pretty phenomenal, I rarely comment on these because it's rare a company stands out so much. Once you left the US, Pantheon was often the fastest amongst their peers. Their global coverage appears to be top notch out of the box.

Plan Tier PHP Bench WP Bench
$25-50 9.914 163.2120124
$51-100 9.229 276.2430939
$200-$500 8.723 294.55081
Enterprise 8.708 235.9046945

The WPPerformanceTester results were pretty consistent. Since it's supposed to be the same platform that scales up, the results looking similar seems to make sense. The slight improvement in PHP bench scores as you scale is nice, but without a lot of repeated testing, unclear if that's by design or just a happy coincidence.

Conclusion

Another year, another fabulous performance from Pantheon. Not only are they the highest rated company on Review Signal's reviews - they consistently earn top tier performance awards in our testing. They stepped up their game from last year and earned an extra Top Tier performance award and an honorable mention. Their global response times were maybe the most impressive feature. They earned fastest average response time in 3/4 tiers on Load Impact and 2/4 on Load Storm, along with often having the fastest response time in their price tiers on WebPageTest. Overall, it was a pretty impressive performance from Pantheon this year.

SiteGround Review (2018)

SiteGround participated for the fifth time in our WordPress Hosting Performance Benchmarks (2018). This review is based off the results of that test. Last year SiteGround competed in four price tiers and earned 3 Top Tier awards.  This year SiteGround competed in the <$25, $25-50, and Enterprise tier.

SiteGround is one of the oldest competitors in our benchmarks and have for years now topped our reviews based on what people say on social media leading the shared category by a whopping 13% over the next highest competitor at the time of writing (Sep 2018).

The Products

Plan Name Plan Monthly Price Plan Visitors Allowed Plan Disk Space Plan Bandwidth
Plan Sites Allowed
GrowBig $14.95 Suitable for ~ 25,000 Visits Monthly 20GB Unmetered Unlimited
GoGeek $29.95 Suitable for ~ 100,000 Visits Monthly 30GB Unmetered Unlimited
Enterprise $4,800 Unlimited $1 per 1GB for distrbuted storage + shared file system 10TB Custom

Enterprise was a custom setup: 8 servers - 2 load balancers, 4 application nodes (PHP workers) and two MySQL database servers. Every server had 16GB of RAM. The total RAM for the whole solution was 128GB RAM.

Performance Review

Load Storm

Total Requests Total Errors Peak RPS Average RPS Peak Response Time(ms) Average Response Time (ms) Total Data Transferred (GB) Peak Throughput (MB/s)
Average Throughput (MB/s)
<25 316,826 14 242.02 176.01 5,361 445 21.18 16.98 11.76
25-50 317,462 16 245.1 176.37 4,569 404 21.23 17.25 11.79
Enterprise 1,443,091 15 1,151.40 801.72 15,086 209 105.15 84.68 58.42

Sources: <25, 25-50, Enterprise

The Load Storm test is designed to simulate real users coming to the site, logging in and browsing the site bursting some of the caching mechanisms typically found on managed WordPress hosts.

SiteGround aced the LoadStorm tests having minimal errors across every price tier. The enterprise configuration really shined here and makes it clear the value of all the redundancy and scalabliity where it had half the average response time and handled 1.43 million requests without issue.

Load Impact

Requests Errors Data Transferred (GB) Peak Average Load Time (Seconds) Peak Average Bandwidth (Mbps)
Peak Average Requests/Sec
<25 318163 4 16.97 0.556 264 605
25-50 316927 0 16.89 0.531 280 642
Enterprise 1627518 7 87.96 0.618 1370 3100

Sources: <2525-50, Enterprise

The Load Impact test makes sure static caching is effective so that if a page gets a lot of traffic the site will keep responding without issue.

SiteGround again aced the Load Impact tests. Nearly no errors on <25 and Enterprise and zero on the 25-50 test.

Uptime

UptimeRobot StatusCake
<25 99.99 99.99
25-50 99 98.94
Enterprise 99.96 100

The <25 and Enterprise didn't have any uptime issues. Unfortunately the 25-50 plan ran into some issues. SiteGround issued a statement about the issue:

"The recorded downtime for the SiteGround GoGeek hosting plan during the test was not a result of actual server availability issues. The uptime is below 99.9% because during performance testing an automatic limitation system temporarily kicked in, in a way that prevented the uptime test to be properly executed. SiteGround confirmed that this particular limitation system should not have been active on the tested accounts and is currently not active on any of their production servers."

WebPageTest / WPPerformanceTester

PHP Bench WP Bench
<25 10.49 1157.407407
25-50 10.44 890.4719501
Enterprise 10.457 196.6181675

The WPPerformanceTester results were average. The distributed nature of the Enterprise shows quite clearly how having redundant and scalable systems can reduce thoroughput on the WP Bench (MySQL) when you run multiple and separate database servers.

<25 25-50 Enterprise
Dulles 0.432 0.508 0.812
Denver 1.489 1.358 1.342
LA 0.909 0.894 1.06
London 0.937 1.082 0.998
Frankfurt 1.087 1.235 0.999
Rose Hill, Mauritius 1.945 2.108 2.636
Singapore 2.088 2.564 1.908
Mumbai 1.957 2.329 2.109
Japan 1.617 1.769 1.394
Sydney 1.781 2.126 1.588
Brazil 1.392 1.565 1.187

Nothing special to say here, it performed as expected.

Conclusion

SiteGround had excellent load testing performances. Load Storm and Load Impact results were great across the board. The <25 plan also had the second fewest errors cumulatively in its price tier. The 25-50 plan looked solid except there was an uptime issue which disqualified it from earning top tier status. SiteGround's Enterprise offering was a new entry in the tier. They made quite a splash with an 8 server configuration which handled our heaviest load tests without batting an eye. It's nice to see SiteGround offering great performance at an entry level price point and being able to scale that performance up to an Enterprise grade solution.

Web Hosting

CloudWays Review (2018)

CloudWays participated for the fourth time in our WordPress Hosting Performance Benchmarks (2018). This review is based off the results of that test. This year CloudWays participated in the 25-50, 51-100 and 201-500 price tiers.

In years past it's been interesting to see CloudWays compete with the same stack on different platforms. This year is the furthest departure from that we've seen so far. It's also the first time CloudWays has earned Top Tier status for two out of three plans that competed this year. It's also important to note, the Digital Ocean plan was originally $70 when testing started but Digital Ocean reduced their pricing causing the cost of the plan to drop dramatically to $42 hence the competing in a different tier above (51-100).

The Products

Plan Monthly Price Plan Visitors Allowed Plan Memory/RAM Plan Disk Space Plan Bandwidth
Plan Sites Allowed
Vultr 4GB New York $44 Unlimited 4GB 60GB SSD 3 TB Unlimited
DigitalOcean 4GB $42 Unlimited 4GB 80gb 4TB unlimited
AWS EC2 - 2XL - USA N.Virginia $495.50 Unlimited 32GB Starts from 4GB (variable) 2GB Unlimited

Performance Review

Load Storm

Total Requests Total Errors Peak RPS Average RPS Peak Response Time(ms) Average Response Time (ms) Total Data Transferred (GB) Peak Throughput (MB/s) Average Throughput (MB/s)
CloudWays AWS 770,304 822 607.12 427.95 15,083 324 52.27 42.27 29.04
CloudWays Vultr 328,015 0 249.5 182.23 7,372 360 22.27 18.55 12.37
CloudWays DO 442,424 243 331.95 245.79 15,097 1,131 30.11 21.79 16.73

Sources: AWS, Vultr, DO

The Load Storm test is designed to simulate real users coming to the site, logging in and browsing the site bursting some of the caching mechanisms typically found on managed WordPress hosts.

The Vultr plan had no issue, the AWS and DO had a few errors but around 0.1% which is negligible. The big issue was Digital Ocean's response time started to increase as the load increased which knocked it out of earning Top Tier status. AWS and Vultr did great overall. With the price change, I wonder if the smaller test would have been handled better though.

Load Impact

Requests Errors Data Transferred (GB) Peak Average Load Time (Seconds) Peak Average Bandwidth (Mbps)
Peak Average Requests/Sec
CloudWays AWS 1049669 0 57 0.36 908 1990
CloudWays Vultr 335275 0 18.21 0.544 278 624
CloudWays DO 457906 30 24.89 1.92 312 702

Sources: AWS, Vultr, DO.

The Load Impact test makes sure static caching is effective so that if a page gets a lot of traffic the site will keep responding without issue.

AWS and Vultr handled it perfectly with zero errors. Digital Ocean had a miniscule 30 errors but an increased response time as the test went on. Great showing overall on the Load Impact test for AWS and Vultr.

Uptime

UptimeRobot StatusCake
CloudWays AWS 100 100
CloudWays Vultr 100 100
CloudWays DO 100 100

Perfect. Across the board, perfect uptime.

WebPageTest / WPPerformanceTester

PHP Bench WP Bench
CloudWays AWS 8.831 266.5955745
CloudWays Vultr 9.616 346.1405331
CloudWays DO 13.421 135.7036233

The WPPerformanceTester results are normal.

CloudWays AWS CloudWays Vultr CloudWays DO
Dulles 0.312 0.33 0.328
Denver 1.137 1.305 1.101
LA 0.924 1.014 1.069
London 0.763 0.725 0.718
Frankfurt 0.816 0.785 0.819
Rose Hill, Mauritius 1.867 2.497 1.86
Singapore 2.281 2.163 2.25
Mumbai 1.65 2.287 1.646
Japan 1.578 1.613 1.573
Sydney 1.837 1.978 2.068
Brazil 1.1 1.195 1.172

The WPT tests look good. The Dulles test scores were some of the fastest, especially the CloudWays AWS server which was located in the same testing data center.

Conclusion

Hard work does pay off. CloudWays has been participating for years and continually has been improving. Two Top Tier awards for AWS and Vultr plans. The Digital Ocean plan unfortunately didn't share the same honor but it seems it was competing above its weight with a price drop that would have put it one cost tier below now.

$500+/Month Enterprise WordPress Hosting Performance Benchmarks (2016)

LoadStormLogo

Sponsored by LoadStorm. The easy and cost effective load testing tool for web and mobile applications.

The full company list, product list, methodology, and notes can be found here

This post focuses only on the results of the testing in the $500+/month price bracket for WordPress Hosting.

Enterprise WordPress Hosting Introduction

This is super exciting for me to test the ultra high end of the market. The past three years I've focused entirely on entry level plans, but the market has changed tremendously since I started and there is a very real demand for Enterprise WordPress hosting. I think this is the first time that a lot of these companies have been benchmarked, especially at this scale and level. So I hope this adds a new and incredibly valuable door for the minority of sites out there that really need to handle massive amounts of users.

The Enterprise testing this year had some fundamental differences from all the other testing that need to be discussed upfront. These are huge and expensive systems that are normally customized on a per-customer basis by these companies. They all offer a much more hands on experience than hosting plans at the other end of the spectrum and charge accordingly. For that reason, I felt it was only responsible to change how they were tested slightly.

The first change is there is no default setup, which is what I test in every other price tier. The companies were given explicit permission to customize their platform and knew what tests were coming their way. Some even ran their own load tests to make sure they were going to perform as advertised and made changes. This is what I would expect from plans charging hundreds, if not thousands of dollars per month for large sites. So I wanted to let them perform their normal services for this tier.

Uptime monitoring was reduced for many companies in this tier. Since these plans are very expensive and consume huge amounts of resources, I didn't want to keep my test sites eating up lots of money and resources. If they had other plans entered into the system, I created a composite based on what all their other plans averaged for uptime.

 

$500+/Month Enterprise WordPress Hosting Products

review_signal_table_enterprise

$500+/Month Enterprise WordPress Hosting Performance Benchmarks Results

1. Load Storm

Test 500-10,000 Concurrent Users over 30 Minutes, 10 Minutes at Peak

Company Total Requests Total Errors Peak RPS Average RPS Peak Response Time(ms) Average Response Time(ms) Total Data Transferred (GB) Peak Throughput (MB/s) Average Throughput (MB/s)
Kinsta 1314178 274 1041.28 730.1 15014 340 75.7 60.75 42.06
Pagely 1388436 18 1108.3 775.24 9086 259 76.97 61.75 42.76
Pantheon 1295178 9964 1014.58 719.54 15101 786 30.86 24.18 17.15
Pressable 1538237 7255 1162.63 854.58 15099 733 29.18 21.95 16.21
Pressidium 1349118 3792 1076.52 749.51 11798 324 73.63 60.18 40.91
WordPress.com VIP 4660190 8151 3726.38 2588.99 8186 101 197.82 158.29 109.9
WPEngine 1515128 247976 1211.18 841.74 19797 281 52.1 40.34 28.94

Discussion of Load Storm Test Results

First off, this is the biggest load tests I've run to date. I had limited resources and wanted to test a high enough number to really put some stress on these systems. 10,000 concurrent users seemed like a reasonable choice based on limited resources and high enough to be meaningful for sites that are truly getting a lot of traffic.

Kinsta and Pagely [Reviews] had basically flawless performances. Flat average response times, minimal errors and no spikes.

WordPress.com VIP had a nearly perfect looking run except for some minor issue with wp-login that might be security related but persisted the entire test at a tiny level (0.17%). The average response time was impressively flat and the fastest of any company by a good bit at 101ms. They also maintained the lowest peak response time. WP VIP also loaded a lot of extra scripts that nobody else did, which increased their transfer data to be multiple times higher than anyone else.

Pantheon [Reviews], Pressable and Pressidium each had minor spikes but put on nearly perfect performances otherwise.

WPEngine [Reviews] ran into what looks to be a similar issue to the other tests, wp-login/admin security issues. Which caused a lot of errors and makes the test look not great. However, their average response time was flat, but it's really hard to say with such a high error rate (16.37%).

 

2. Blitz.io

Test 1-5000 Concurrent Users over 60 seconds

Blitz Test Quick Results Table

Company Hits Errors Timeouts Average Hits/Second Average Response Time Fastest Response Slowest Response
Kinsta 135485 7 0 2258 85 83 87
Pagely 146339 0 0 2439 4 3 14
Pantheon 138607 4 27 2310 62 60 80
Pressable 129866 13 2 2164 132 131 139
Pressidium 143452 0 2 2391 26 24 35
WordPress.com VIP 146200 0 73 2437 6 3 21
WPEngine 108168 12939 1061 1803 158 6 346

Discussion of Blitz Test 1 Results

This test is just testing whether the company is caching the front page and how well whatever caching system they have setup is performing (generally this hits something like Varnish or Nginx).

Who performed without any major issues?

Kinsta, Pagely [Reviews], PantheonPressablePressidium and WordPress.com VIP all handled 5000 concurrent hits to the frontpage without any issue. The largest spread in response times among all of them was a minuscule 20ms. Pagely even managed an impressive perfect no errors or timeouts.

Who had some major issues?

WPEngine [Reviews] struggled with this test. Around 20 seconds into the test, there was a substantial increase in response time which continued to slowly increase for the rest of the test. The errors and timeouts started to kick in 5 seconds later at the 25 second mark and also gradually increased until the test ended.

3. Uptime Monitoring

Both uptime monitoring solutions were third party providers that offer free services. UptimeRobot was paid for and monitoring on a 1 minute interval. All the companies were monitored over approximately two months (May-June 2016).

For Enterprise testing, many of the plans were only setup for a short period of time because of the enormous cost involved with setting these up. Only WordPress.com VIP and WPEngine were monitored directly. The rest are composite scores based on the other plans companies entered in and the company's average uptime as denoted with an asterisk (*).

Uptime Robot & StatusCake

Company StatusCake UptimeRobot
Kinsta* 99.98  100
Pagely*  99.98  99.98
Pantheon*  99.99  99.99
Pressable*  99.92  99.90
Pressidium*  99.97  99.99
WordPress.com VIP 100 100
WPEngine 100 100

* Composite uptime based on all the plans entered in 2016 testing from a company.

Every company in the enterprise tier seems capable of keeping their servers online, thankfully.

4. WebPageTest.org

Every test was run with the settings: Chrome Browser, 9 Runs, native connection (no traffic shaping), first view only.

Company WPT Dulles WPT Denver WPT LA WPT London WPT Frankfurt WPT South Africa
Kinsta 0.718 0.588 0.958 1.256 1.741 5.844
Pagely 0.752 0.758 0.953 1.243 2.029 9.885
Pantheon 0.809 0.563 1.02 1.284 1.826 4.882
Pressable 1.056 0.894 1.207 1.691 2.126 7.244
Pressidium 0.848 0.661 1.165 1.279 1.634 5.819
WordPress.com VIP 1.02 0.786 0.918 1.471 1.755 3.045
WPEngine 0.813 0.592 1.07 1.223 1.743 3.814
Company WPT Singapore WPT Shanghai WPT Japan WPT Sydney WPT Brazil
Kinsta 2.084 22.391 2.055 1.643 1.891
Pagely 2.455 23.148 2.203 2.117 2.153
Pantheon 2.336 22.723 1.95 1.852 2.032
Pressable 2.707 22.521 2.227 2.807 2.205
Pressidium 2.202 22.477 2.265 1.662 1.797
WordPress.com VIP 1.809 24.098 1.83 1.386 1.916
WPEngine 2.255 22.971 2.115 1.722 1.846

It's not surprising that these companies deliver content pretty quick all around the world. What is interesting is WordPress.com VIP was the fastest to Sydney, Japan, Singapore, South Africa, and LA. Kinsta was the fastest in Dulles and Shanghai. Pantheon was fastest in Denver. WPEngine was the fastest to London. Pressidium was the fastest to Brazil. I'm not sure how meaningful it is, but it's interesting to see the most expensive product having the fastest load times in locations all across the world.

5. WPPerformanceTester

Company PHP Bench [Seconds] (lower=faster) WP Bench [Queries Per Second](higher=faster)
Kinsta 11.37 320.82
Pagely 9.136 249.81
Pantheon 11.322 216.31
Pressable 10.834 491.64
Pressidium 10.958 367.24
WordPress.com VIP 2.244 500.25
WPEngine 13.178 533.9

I'm not sure what WordPress.com VIP is running, but it put up the absolute fastest scores in the PHP bench that I've seen by a wide margin. Roughly triple the speed of the next fastest which had a 6.5 second score. Every other company looked to be in the normal range between 9-13 seconds.

Another interesting part of the results here is that nobody was really going much faster than 500 queries per second in the WP Bench. I don't think a single one is running a local database which put up some blazing fast speeds in the lower tiers. If you're looking to host enterprise WordPress sites, you lose that no network latency performance, but certainly gain in reliability and scalibility.

Conclusion

White glove service and hefty price tags makes for some spectacular performance. It's nice to see that if you really have a site getting millions of visitors per day, there are a lot of really solid choices out there who can handle the mega WordPress sites that need Enterprise level hosting.

Top Tier WordPress Hosting Performance

review_signal_2016_trophy_enterprise

Kinsta, Pagely [Reviews], PantheonPressablePressidium and WordPress.com VIP all offer Top Tier Enterprise WordPress Hosting. None of them had any real struggles with keeping their servers up, the 10,000 and 5,000 user load tests. If you can afford them, they all seem worthy and capable.

Individual Host Analysis

Kinsta

Kinsta had essentially perfect LoadStorm and Blitz tests. They also had no flaws in any other tests. I'm at a loss for words to praise their performance.

Pagely [Reviews]

Pagely aced it. The fewest errors on LoadStorm and a no errors on Blitz. I can't find any faults with Pagely's Enterprise offering.

Pantheon [Reviews]

Pantheon really stepped it up for the Enterprise testing. Effortlessly went through the Blitz test. They had a some minor spikes in the LoadStorm test and their average response time started to creep upwards but nothing worth being concerned over. Overall, a top tier performance.

Pressable

Pressable performed nearly identical to Pantheon. Excellent Blitz test, some minor spikes and increase in response times in the LoadStorm test. The uptime was the lowest of everyone with UptimeRobot having an average of 99.90% which has been my border for pass/fail. I gave them top tier, but they were about as close as you can get to the edge.

Pressidium

Pressidium had a nearly perfect Blitz test with 2 timeouts and did excellent on the LoadStorm test which had 2 very minor spikes but maintained a nearly flat average response time otherwise. Easily a top tier performance.

WordPress.com VIP 

WordPress.com VIP was by far the most expensive plan tested and it put on a fantastic performance. It had a near perfect Blitz test. Despite having what appeared to be a security issue on the LoadStorm test it had the fastest average response time at 101ms and moved more data than any other company by a wide margin because of the custom scripts. But that didn't seem to negatively impact their performance at all. I'm also not sure what sort of hardware they are running by they blew my WPPerformanceTester PHP bench out of the water. Despite the highest price tag, they put on an amazing show and easily earned Top Tier Enterprise WordPress Hosting status.

WPEngine [Reviews]

Unfortunately, WPEngine was the only company in this tier not to do well in this tier. They struggled in both load tests. LoadStorm looked like it may have been security related, but Blitz looked like it really had trouble with the load. I believe the plan I tested cost $600/month, but the sales team wasn't willing to give me specific pricing for their enterprise tier.