Author Archives: Kevin Ohashi

avatar

About Kevin Ohashi

Kevin Ohashi is the geek-in-charge at Review Signal. He is passionate about making data meaningful for consumers. Kevin is based in Washington, DC.

Ethics and One Tweet’s Impact on Digital Ocean

I am speaking at HostCamp (side event to WordCamp Europe) in a couple weeks on the topic of Ethics in WordPress Hosting. I'm not really sure WordPress hosting has any specific differences from web hosting in general when talking about ethics. But ethical behavior in the web hosting space is something I talk about a lot. I also aggressively call out people/companies behaving unethically on this blog in the web hosting space. [1] [2] [3] [4]

As I was writing a response to a short interview to introduce the topic, I tried to think about a relevant example of why ethics matter in web hosting. A very recent event jumped to mind, someone tweeting that Digital Ocean [Reviews] shut down their company.

This tweet was sent by @w3Nicolas.

The stats are staggering:

That's only in the communities I participate in, I was sent the link by multiple people in other groups as well. I'm sure tens of thousands of people, if not more, read about this incident.

 

This is a view into what that tweet did to Digital Ocean's data here on Review Signal (I track Twitter data and sentiment about web hosting companies for the unfamiliar). I pulled the past 30 days of Digital Ocean information.

The tweet was sent on May 31, the 4th data point. We see an enormous jump in tweet volume. The preceding days had an average of 248 tweets per day. May 31 had 2000 and June 1 had 2489 tweets, nearly 10X the normal volume for two days. By June 4, we're down to 274 tweets, a normal volume. The internet outrage machine was out in force and spreading the word.

Digital Ocean responded on Twitter with Moisey Uretsky, a cofounder, intervening to escalate and resolve the issue. Digital Ocean also released a post-mortem on June 4 about what happened as promised (Nice to see a company keep their word and admit mistakes).

What does this have to do with Ethics?

Why did I even write this story and what does it have to do with ethics? The question I was trying to answer when I started thinking about this incident and digging into the data is "Why should hosting companies and those who do business with them care about ethics?"

A lot of developers and entrepreneurs read a story about a guy who was shutdown without warning, and then locked out seemingly permanently without being treated fairly. It strikes a chord with people when someone is being treated wrongly/badly with no explanation, especially when it's their livelihood that is impacted. It violates a fundamental moral code of fairness and trust.

The impact for a perceived ethical violation in this case was tens of thousands of people reading a negative story. It generated heated discussions and some very negative comments.

My data showed a tremendous increase in negative messages with the ratio dropping to 34% (Digital Ocean has historically over 70% positive messages).

They were quick to jump into some of the communities and address the issue. The post-mortem on Twitter received 225 Likes and 62 Retweets. That's 2.4% the amount of retweets and 4.9% the Likes. The impact of addressing the issue and trying to improve made a tiny fraction of the impact.

I will be clear here, I don't think Digital Ocean acted maliciously or unethically (intentionally). It sounds like a combination of automated system and a couple human mistakes lead to a very bad outcome for a customer that attracted a lot of attention. The way it was portrayed evoked feelings of an ethical violation of fairness and trust.

Digital Ocean's post-mortem's conclusion:

We wanted to share the specific details around this incident as accurately and quickly as possible to give the community insight into what happened and how we handled it. We recognize the impact this had on a customer, and how this represented a breach of trust for the community, and for that we are deeply sorry. We have a number of takeaways to improve the technical, process, and people missteps that led to this failure. The entire team at DigitalOcean values and remains committed to the global community of developers.

So when companies think about how they should behave, I want to use this example as an argument that people do care about companies behaving ethically and awareness of their behavior can quickly be amplified when a person's story resonates.

WordPress.com VIP Review (2018)

WordPress.com VIP participated for the second time in our WordPress Hosting Performance Benchmarks (2018). This review is based off the results of that test. This year WordPress.com VIP entered in the same Enterprise price tier again.

In the previous test, WP.com VIP aced the Enterprise test and earned the Top Tier WordPress Hosting Performance award from Review Signal. Considering who is behind it, the expectations behind WP.com VIP are enormous.

The Product

Plan Name Plan Monthly Price Plan Visitors Allowed Plan Memory/RAM Plan Disk Space Plan Bandwidth
Plan Sites Allowed
WordPress.com VIP $5,000 Unlimited Unlimited Unlimited Unlimited 5

Performance Review

Load Storm

Plan Tier Total Requests Total Errors Peak RPS Average RPS Peak Response Time(ms) Average Response Time (ms) Total Data Transferred (GB) Peak Throughput (MB/s) Average Throughput (MB/s)
Enterprise 1,749,966 3 1,400.53 972.2 7,484 108 147.94 118.68 82.19

The Load Storm test is designed to simulate real users coming to the site, logging in and browsing the site bursting some of the caching mechanisms typically found on managed WordPress hosts.

Three errors out of 1.75 million requests. VIP also had the fastest average response time at 108ms. Simply astonishing.

Load Impact

Plan Tier Requests Errors Data Transferred (GB) Peak Average Load Time (Seconds) Peak Average Bandwidth (Mbps)
Peak Average Requests/Sec
Enterprise 2085775 6 119.88 0.102 1860 3960

The Load Impact test makes sure static caching is effective so that if a page gets a lot of traffic the site will keep responding without issue.

Six errors in over 2 million requests is another top notch performance. VIP also had the lowest average response time of any company.

Uptime

Plan Tier UptimeRobot StatusCake
Enterprise 99.95 100

The uptime results were great. 100 and 99.95% is very respectable.

WebPageTest / WPPerformanceTester

Plan Tier Enterprise
Dulles 0.334
Denver 1.112
LA 0.649
London 0.444
Frankfurt 0.353
Rose Hill, Mauritius 0.874
Singapore 0.395
Mumbai 0.544
Japan 0.358
Sydney 0.506
Brazil 0.363

I rarely comment about WebPageTest results but WordPress.com VIP deserves a special mention here. There were 11 test locations and VIP was the fastest in 9/11 and was 22 ms and 9 ms behind the fastest in the other two test locations. I've never seen any company be so dominant across the globe in terms of performance.

Plan Tier PHP Bench WP Bench
Enterprise 9.18 798.084597

The WPPerformanceTester results were normal.

Conclusion

WordPress.com VIP's sophomore performance didn't experience a slump. In fact, they seemed to have stepped up their game. Not only did they put on a better Load Storm performance, their response times across the globe were fantastic. WordPress.com VIP earned themselves a second Top Tier WordPress Hosting Performance award this year and was quite a stand out in many regards in a very crowded and well performing field.

WP Engine Review (2018)

WPEngine [Reviews] participated for the fifth time in our WordPress Hosting Performance Benchmarks (2018). This review is based off the results of that test. This year WPEngine entered in the Enterprise tier again. In the previous test, WPEngine struggled with the load tests. They returned this year and entered with the most expensive offering in the Enterprise tier and earned themselves a Top Tier WordPress Hosting Performance award.

The Product

Plan Name Plan Monthly Price Plan Visitors Allowed Plan Memory/RAM Plan Disk Space Plan Bandwidth
Plan Sites Allowed
Premium Dedicated $5,400 Capable of 10M+ (no hard limit, depends on site) N/A 1TB Unlimited 150

Performance Review

Load Storm

Plan Tier Total Requests Total Errors Peak RPS Average RPS Peak Response Time(ms) Average Response Time (ms) Total Data Transferred (GB) Peak Throughput (MB/s) Average Throughput (MB/s)
Enterprise 1,914,457 786 1,564.62 1,063.59 11,541 509 115.54 95.23 64.19

The Load Storm test is designed to simulate real users coming to the site, logging in and browsing the site bursting some of the caching mechanisms typically found on managed WordPress hosts.

WPEngine's Load Storm test was good. Minimal errors and stable response time.

Load Impact

Plan Tier Requests Errors Data Transferred (GB) Peak Average Load Time (Seconds) Peak Average Bandwidth (Mbps)
Peak Average Requests/Sec
Enterprise 1461819 0 78 0.665 1210 2780

The Load Impact test makes sure static caching is effective so that if a page gets a lot of traffic the site will keep responding without issue.

Zero errors, that's the magic number we love to see. WP Engine aced the Load Impact test.

Uptime

Plan Tier UptimeRobot StatusCake
Enterprise 99.95 99.96

WPEngine [Reviews]  maintained above 99.9% uptime on both monitors.

WebPageTest / WPPerformanceTester

Plan Tier Enterprise
Dulles 0.436
Denver 1.155
LA 0.948
London 1.155
Frankfurt 1
Rose Hill, Mauritius 2.109
Singapore 1.536
Mumbai 1.891
Japan 1.062
Sydney 1.428
Brazil 1.238

The response times on WPT were normal.

Plan Tier PHP Bench WP Bench
Enterprise 10.766 418.4100418

The WPPerformanceTester results normal.

Conclusion

After a disappointing 2016 showing, WPEngine [Reviews]  stepped up their game and earned a Top Tier WordPress Hosting Performance award from Review Signal. It's always good to see companies working to improve their service offering. WPEngine is one of the oldest companies in the WordPress hosting space and they are showing they can support even the largest clients as many new competitors spring up. Great performance from WP Engine.

Presslabs Review (2018)

Presslabs participated for the fourth time in our WordPress Hosting Performance Benchmarks (2018). This review is based off the results of that test. Presslabs entered a plan into the Enterprise category, one level up from last year's $201-500 range. Last year Presslabs earned an honorable mention in the $201-500 price tier.

The Product

Plan Name Plan Monthly Price Plan Visitors Allowed Plan Memory/RAM Plan Disk Space Plan Bandwidth
Plan Sites Allowed
Enterprise $2599 USD + $0.1/1000 page views 0 included, all traffic is charged with $0.1/1000 page views Unmetered Unmetered Unmetered
up to 25 (depending on traffic and requirements)

Performance Review

Load Storm

Plan Tier Total Requests Total Errors Peak RPS Average RPS Peak Response Time(ms) Average Response Time (ms) Total Data Transferred (GB) Peak Throughput (MB/s) Average Throughput (MB/s)
Enterprise 2,012,822 56 1,576.23 1,118.23 10,099 208 116.54 90.85 64.75

The Load Storm test is designed to simulate real users coming to the site, logging in and browsing the site bursting some of the caching mechanisms typically found on managed WordPress hosts.

An excellent performance on Load Storm from Presslabs. No signs of load and minimal errors.

Load Impact

Plan Tier Requests Errors Data Transferred (GB) Peak Average Load Time (Seconds) Peak Average Bandwidth (Mbps)
Peak Average Requests/Sec
Enterprise 1991403 0 104.96 0.184 1980 4580

The Load Impact test makes sure static caching is effective so that if a page gets a lot of traffic the site will keep responding without issue.

Zero errors and sub 200ms average response time. Excellent.

Uptime

Plan Tier UptimeRobot StatusCake
Enterprise 100 100

Perfect.

WebPageTest / WPPerformanceTester

Plan Tier Enterprise
Dulles 0.378
Denver 1.352
LA 0.85
London 0.49
Frankfurt 0.381
Rose Hill, Mauritius 1.335
Singapore 0.485
Mumbai 0.721
Japan 0.66
Sydney 0.551
Brazil 1.336

Presslabs' WPT results were good.

Plan Tier PHP Bench WP Bench
Enterprise 7.215 823.0452675

The WPPerformanceTester results had the fastest PHP Bench and second fastest WP Bench.

Conclusion

Presslabs stepped up a price tier and improved from honorable mention last year to earning Top Tier WordPress Hosting Performance award in 2018 at the Enterprise tier. It's nice to see another worthy competitor in the Enterprise space that can handle large sites.

WP Oven Review (2018)

WP Oven participated for the third time in our WordPress Hosting Performance Benchmarks (2018). This review is based off the results of that test. This year WP Oven moved up a tier to the $51-100 price tier.

In the previous test, WP oven achieved perfection and earned themselves Top Tier status from Review Signal. They had zero errors on every test and perfect uptime.

The Product

Plan Name Plan Monthly Price Plan Visitors Allowed Plan Memory/RAM Plan Disk Space Plan Bandwidth
Plan Sites Allowed
Professional $79.95 Unlimited 8GB 80GB 8 TB Unlimited Sites

Performance Review

Load Storm

Plan Tier Total Requests Total Errors Peak RPS Average RPS Peak Response Time(ms) Average Response Time (ms) Total Data Transferred (GB) Peak Throughput (MB/s) Average Throughput (MB/s)
$51-100 438,758 1 342.95 243.75 10,051 370 30.96 25.71 17.2

The Load Storm test is designed to simulate real users coming to the site, logging in and browsing the site bursting some of the caching mechanisms typically found on managed WordPress hosts.

WP Oven had a whopping single error on Load Storm and kept a sub 400ms response time. This was the fewest of any company in the price tier. An excellent showing.

Load Impact

Plan Tier Requests Errors Data Transferred (GB) Peak Average Load Time (Seconds) Peak Average Bandwidth (Mbps)
Peak Average Requests/Sec
$51-100 675881 0 36.07 0.398 619 1380

The Load Impact test makes sure static caching is effective so that if a page gets a lot of traffic the site will keep responding without issue.

WP Oven had zero errors and kept a sub 400ms response time. Perfect.

Uptime

Plan Tier UptimeRobot StatusCake
$51-100 99.98 99.97

Both monitors were above 99.9%, which is good.

WebPageTest / WPPerformanceTester

Plan Tier $51-100
Dulles 0.406
Denver 1.118
LA 0.739
London 1.124
Frankfurt 1.206
Rose Hill, Mauritius 2.04
Singapore 2.103
Mumbai 2.221
Japan 1.421
Sydney 2.099
Brazil 1.316

The WPT results look normal.

Plan Tier PHP Bench WP Bench
$51-100 9.634 211.1040743

The WPPerformanceTester results are normal.

Conclusion

WP Oven proved they could step up a price tier in 2018 and still earn itself Review Signal's Top Tier WordPress Hosting Performance award. It's nice to see a relatively unknown company do consistently well in our benchmarks. Another year, another great performance, keep up the great work WP Oven!

Kinsta Review (2018)

Kinsta participated for the fourth time in our WordPress Hosting Performance Benchmarks (2018). This review is based off the results of that test. This year Kinsta participated in the 25-50, 51-100, 101-200, 201-500, and Enterprise price tiers.

In the previous test, Kinsta earned Top Tier status in all four price brackets it entered.  In 2018, Kinsta started offering cheaper plans and has competed in an additional, lower price tier.

The Products

Plan Name Plan Monthly Price Plan Visitors Allowed Plan Memory/RAM Plan Disk Space Plan Bandwidth
Plan Sites Allowed
Starter $30 20000 - 5GB Unlimited 1
Pro $60 40,000 - 5GB Unlimited 2
Business 1 $100 100,000 - 10GB Unlimited 3
Business 2 $200 250000 - 20GB SSD Unlimited 10
Enterprise 1 $600 1 Million - 80GB SSD Unlimited 60

Performance Review

Load Storm

Plan Tier Total Requests Total Errors Peak RPS Average RPS Peak Response Time(ms) Average Response Time (ms) Total Data Transferred (GB) Peak Throughput (MB/s) Average Throughput (MB/s)
$25-50 294,176 3 225.5 163.43 10,036 355 21.38 17.58 11.88
$51-100 425,597 1 332.65 236.44 10,017 377 30.98 25.62 17.21
$101-200 556,007 48 436.15 308.89 10,101 375 40.45 32.44 22.47
$200-500 688,856 0 543.12 382.7 4,407 358 50.08 40.47 27.82
Enterprise 1,720,465 0 1,391.42 955.81 3,906 417 112.7 91.9 62.61

The Load Storm test is designed to simulate real users coming to the site, logging in and browsing the site bursting some of the caching mechanisms typically found on managed WordPress hosts.

Kinsta aced the Load Storm tests across every tier. Zero errors in multiple price tiers and minimal errors in the others. The response times were kept fast and stable. It scaled up from the smallest test to Enterprise test without issue.

Load Impact

Plan Tier Requests Errors Data Transferred (GB) Peak Average Load Time (Seconds) Peak Average Bandwidth (Mbps)
Peak Average Requests/Sec
$25-50 337031 0 18.05 0.41 304 694
$51-100 672240 0 36 0.416 583 1300
$101-200 1016163 0 54.42 0.395 863 1920
$200-500 1014718 0 54.34 0.411 866 1930
Enterprise 1367747 0 73.4 0.802 1120 2540

The Load Impact test makes sure static caching is effective so that if a page gets a lot of traffic the site will keep responding without issue.

Perfection. Zero errors across 5 separate tests.

Uptime

Plan Tier UptimeRobot StatusCake
$25-50 100 100
$51-100 100 100
$101-200 100 100
$200-500 100 100
Enterprise 100 100

Perfect. 100% across 5 plans and two separate monitors.

WebPageTest / WPPerformanceTester

Plan Tier $25-50 $51-100 $101-200 $200-500 Enterprise
Dulles 0.387 0.373 0.377 0.396 0.382
Denver 1.287 1.163 1.411 1.134 1.245
LA 0.833 0.79 0.81 0.832 1.154
London 1.076 1.104 1.073 1.177 1.331
Frankfurt 1.014 1.009 1.01 1.031 1.103
Rose Hill, Mauritius 2.436 2.446 2.433 2.439 2.444
Singapore 1.902 1.729 1.728 1.712 1.774
Mumbai 2.114 2.279 2.221 2.119 2.108
Japan 1.186 1.183 1.234 1.268 1.281
Sydney 1.609 1.766 1.763 1.772 1.611
Brazil 1.513 1.364 1.384 1.508 1.404

WPT test results looked normal.

Plan Tier PHP Bench WP Bench
$25-50 8.853 1036.26943
$51-100 8.783 1173.70892
$101-200 8.974 1100.110011
$200-500 9.048 1047.120419
Enterprise 8.787 1207.729469

The WPPerformanceTester results looked normal. The WP Bench results stood out though. The database connections are consistently very fast compared to their competitors at higher price tiers when plans tend to run separate and/or multiple database servers.

Conclusion

Kinsta earned itself five Top Tier WordPress Hosting Performance Awards in 2018. It had perfect uptime across every plan. It had zero errors on Load Impact. It had 2/5 Load Storm tests without any errors and only 52 errors total from 3.685 million requests. No signs of stress showed on any test. Kinsta has consistently brought their A game to WordPress Hosting Performance Benchmarks, this year was no exception. It's nice to see them offering newer, cheaper plans that allow people on smaller budgets access their offerings. A huge congratulations to them for earning the most Top Tier Hosting Performance awards of any company in 2018.

Pantheon Review (2018)

Pantheon participated for the fourth time in our WordPress Hosting Performance Benchmarks (2018). Last year Pantheon earned two Top Tier awards. This review is based off the results of this years test.

Pantheon stepped up their game this year and earned three Top Tier awards and one honorable mention. Pantheon also has the honor of being the highest rated company on Review Signal for reviews based on customer opinions (84%).

The Products

Plan Name Plan Monthly Price Plan Visitors Allowed Plan Memory/RAM Plan Disk Space Plan Bandwidth
Plan Sites Allowed
Personal Plan $25 10,000 (with no overage charges) 1024MB 5GB Unlimited 1
Professional $100 100,000 2048 20GB Unlimited 1
Business $400 500,000 8192 30GB Unlimited 1
Elite $1,667 Unlimited Unlimited 100GB+ Unlimited 1

Performance Review

Load Storm

Plan Tier Total Requests Total Errors Peak RPS Average RPS Peak Response Time(ms) Average Response Time (ms) Total Data Transferred (GB) Peak Throughput (MB/s) Average Throughput (MB/s)
$25-50 292,393 5 220.72 162.44 4,287 162 22.79 17.5 12.66
$51-100 445,581 37 336.4 247.54 15,086 172 33.78 26.65 18.77
$200-$500 702,823 7,065 544.27 390.46 11,462 455 52.65 43.64 29.25
Enterprise 1,370,325 88 1,088.48 761.29 7,147 154 106.71 85.72 59.28

The Load Storm test is designed to simulate real users coming to the site, logging in and browsing the site bursting some of the caching mechanisms typically found on managed WordPress hosts.

Pantheon overall did very well. The only issue was in the $200-500 price range it started to show signs of load. It wasn't a huge issue and the plan still earned an honorable mention in the price tier.

Load Impact

Plan Tier Requests Errors Data Transferred (GB) Peak Average Load Time (Seconds) Peak Average Bandwidth (Mbps)
Peak Average Requests/Sec
$25-50 426896 0 23.73 0.159 401 880
$51-100 847397 0 47.26 0.196 773 1650
$200-$500 1257940 0 69.92 0.173 1020 2240
Enterprise 2046874 0 113.47 0.181 1660 3650

The Load Impact test makes sure static caching is effective so that if a page gets a lot of traffic the site will keep responding without issue.

Pantheon aced the LoadImpact tests across every price tier. Zero errors on all four tests and maintained a sub 200 millisecond response time average. Couldn't ask for a better performance.

Uptime

Plan Tier UptimeRobot StatusCake
$25-50 100 100
$51-100 100 100
$200-$500 99.97 100
Enterprise 100 100

Not a whole lot to say here other than excellent. 100% uptime across the board except one monitor out of two showing 99.97%.

WebPageTest / WPPerformanceTester

Plan Tier $25-50 $51-100 $200-$500 Enterprise
Dulles 0.355 0.354 0.353 0.374
Denver 1.57 1.493 1.236 1.322
LA 0.797 0.804 0.804 0.712
London 0.448 0.446 0.45 0.474
Frankfurt 0.369 0.356 0.375 0.428
Rose Hill, Mauritius 1.093 1.074 1.088 0.912
Singapore 0.433 0.449 0.045 0.426
Mumbai 0.692 0.64 0.662 0.522
Japan 0.419 0.39 0.393 0.366
Sydney 0.528 0.503 0.516 0.497
Brazil 0.405 0.408 0.4 0.384

The WPT tests were pretty phenomenal, I rarely comment on these because it's rare a company stands out so much. Once you left the US, Pantheon was often the fastest amongst their peers. Their global coverage appears to be top notch out of the box.

Plan Tier PHP Bench WP Bench
$25-50 9.914 163.2120124
$51-100 9.229 276.2430939
$200-$500 8.723 294.55081
Enterprise 8.708 235.9046945

The WPPerformanceTester results were pretty consistent. Since it's supposed to be the same platform that scales up, the results looking similar seems to make sense. The slight improvement in PHP bench scores as you scale is nice, but without a lot of repeated testing, unclear if that's by design or just a happy coincidence.

Conclusion

Another year, another fabulous performance from Pantheon. Not only are they the highest rated company on Review Signal's reviews - they consistently earn top tier performance awards in our testing. They stepped up their game from last year and earned an extra Top Tier performance award and an honorable mention. Their global response times were maybe the most impressive feature. They earned fastest average response time in 3/4 tiers on Load Impact and 2/4 on Load Storm, along with often having the fastest response time in their price tiers on WebPageTest. Overall, it was a pretty impressive performance from Pantheon this year.

Loading...

Interested in seeing which web hosting companies people love (and hate!)? Click here and find out how your web host stacks up.