Tag Archives: news

Post Mortem of the EIG Outage (August 2, 2013) That Affected BlueHost, HostGator, JustHost and HostMonster

I first wrote about EIG's major outage as it was occurring and had to speculate on a few things before I had the data to support those guesses. This post is a more complete picture of what happened.

Recap

EIG had a major outage on August 2, 2013 that lasted for many hours because core switches in their Provo, Utah datacenter failed. This failure caused customers of BlueHost, HostGator, JustHost and HostMonster to be taken offline.

I speculated as to what would occur after the outage. How would the brands of the affected companies be perceived after such a catastrophic failure? I looked for a comparable event: the GoDaddy DNS outage in September 2012. What I observed from that event was a very quick return to normal volumes of messages and sentiment. GoDaddy regressed to the mean. 

GoDaddy

The charts I used in my original post were lacking. I didn't have time to really collect and analyze all the data, especially sentiment. I could eyeball the historical data and see the ratings bounced back to their original levels but it wasn't a granular look.

godaddy_dns_outage_full

This chart shows the actual outage, tweet volume and sentiment. It's immediately clear that negative sentiment has a huge spike. I also suspect that a lot of the positive messages are actually mis-categorized; Review Signal isn't perfect and things like sarcasm are one of the hardest things for the sentiment analysis algorithms to categorize. The unusual volume lasts three days and then quickly drops back to a normal looking pattern with perhaps a slightly higher baseline volume. The actual rating goes back to hovering around 50%, which GoDaddy's long-term graph hovers around as well.godaddy_chart

Let's get back to the EIG outage and the affected brands. I am only going to talk about two of the brands, BlueHost and HostGator, in this post because on a granular level, the other two, HostMonster and JustHost, didn't have enough data. The brands without enough data will take more time to develop a clear picture about the effects of the outage.

BlueHost

bluehost_sentiment

I was wrong. So far at least. BlueHost had an overall rating of 57% before August 2. It hasn't broken 50% since the outage. BlueHost did not, or has not yet, regressed back to the mean. What's interesting is that the volume of tweets about BlueHost's outage was more than double in quantity to the similar GoDaddy outage, but they both quickly dropped back to normal volume within days of the event.

I will explore this a bit more, but to do that I need to show you the other brand.

 HostGator

hostgator_sentiment

HostGator's outage looks almost identical to GoDaddy's outage. Around 1000 negative messages on the day of the outage and back to normal within days. HostGator appears to have regressed to the mean as quickly as GoDaddy, its rating has been over 60% two days, which are pre-crash levels, where its average rating was 62%.  HostGator behaved exactly as I predicted.

Weird Conclusions and Speculations

Why hasn't BlueHost regress to the mean? One explanation, which I was alerted to by a kind reader (Thanks Linda!), is that not all of HostGator's customers were in the Provo, UT data center. So the outage may have disproportionately affected BlueHost customers compared to HostGator customers. BlueHost is also the larger hosting company by number of customers, although not domain count.

That explanation may explain the volume difference, but I don't think it explains the regression to the mean for one brand and not the other. Presumably the affected customers of both brands should be equally upset. Those lingering feelings should last equally long for both groups of customers.

I can't explain why we haven't seen BlueHost regress, but I can point out a few differences between this outage and the GoDaddy comparison which may be factors. One important factor is duration. GoDaddy's outage lasted 4-5 hours according to reports. The EIG outage lasted from the morning of August 2 until 9 PM. They were reporting 'intermittent instability' into August 3 according to their official website.

I could speculate that the combination of severity, duration and size of the affected brand has caused some sort of more permanent brand damage to BlueHost, but I think that's premature. BlueHost hasn't regressed yet, but I still think it will eventually. A company that large, with such a huge brand and marketing infrastructure will probably recover. I will be watching BlueHost carefully for the next few weeks or months along with the smaller brands to see if it happens. If it doesn't, this will be an interesting case study in branding, communication and perhaps social media.

 

Thank you for reading and if you have any ideas, feedback or suggestions please leave them in the comments below.

Service Interrupted: A Look at the EIG (BlueHost, HostGator, HostMonster, JustHost) Outage through Twitter

I woke up today and quickly found out that one of the major players in the hosting space was having a massive outage.  According to their own blog:

During the morning of August 2, 2013, Endurance International Group’s data center in Provo, UT experienced unexpected issues that impacted customers of bluehost, HostGator, HostMonster and JustHost. Company websites and some phone services were affected as well.

That sounds bad. Really bad. But how bad? Let's take a look at the data:

tweets_per_day_by_company

 

It's pretty clear that today was an outlier. A major outlier for all the affected companies.

Our data collection system here at Review Signal collected over 35,000 tweets today alone about these four companies. That is roughly 14 times the normal amount.

Interestingly enough, there are some very understanding customers out there too, it wasn't all negative.

hostgator_positive

 

How has it affected their rankings?

I must first note that most messages don't make it through our spam filtering systems for a variety of reasons. So despite there being over 35,000 tweets, we did not get 35,000 new reviews. Many of the messages were not up to our quality standards, eg. retweets, spam, duplicate messages and news. If you are interested in learning more about how we calculate scores and what kinds of messages count see our How It Works section.

 

BlueHost

I am not sure why, but BlueHost was impacted a lot more than it's bigger brother HostGator. BlueHost has 1.9 million domains on their server. They also received over 15,000 tweets about them today (50% more than HostGator).

BlueHost was rated at 57% (Overall Rating) from over two years worth of data collected. Today they dropped 8% to 49%. There were over 1,500 negative reviews today (Note: Our data was calculated early to write this article, the day isn't fully over yet).

HostGator

HostGator is the largest of the bunch and has 2.15 million domains under management. They seemed to have fared the storm better than their brothers with less tweets about them in absolute number and relative to their size.

HostGator was rated at 62% (Overall Rating) and dropped 5% to 57%. HostGator received approximately 700 negative reviews today.

HostMonster and JustHost

These are the babies of the bunch, HostMonster has 'only' 700,000 domainso and JustHost has barely over 350,000.

HostMonster went from a 56% (Overall Rating) to 48%, which is a 8% decline. JustHost dropped from 46% to 41%, a total of 5%.

Conclusion

Today was a pretty awful day for all the companies above but some were affected more than others. I don't have any answer as to why that might be. There are many plausible theories such as perhaps there were more BlueHost customers in the Provo, UT data center than the other companies. But without further information, it's only speculation. UPDATE: I was told BlueHost actually has more customers than HostGator, even if HostGator customers have more domainers. A simple explanation as to why BlueHost was impacted more.

What I can say is a major screw up definitely impacts a company's reputation. But large companies seem to regress to the mean.

GoDaddy is a good comparison. They had a major DNS outage around September 11-12. It left a noticeable dip on the overall rating but it seemed to bounce back. February's dip is the super bowl effect that brings a lot of attention to them (more negative than positive, but attention nonetheless). The long-term volume of tweets also doesn't appear to be affected after a few days.

godaddy_chart

godaddy_dns_outage

If we use GoDaddy as a benchmark, these companies will probably be back to their usual levels of service within a week, but today and the next couple days will leave a very long term impact on their rating at Review Signal.

Imagining a Better Review Site

I was a pretty unsatisfied consumer. There are so many review sites and so few that I actually trust. Other than my few trusted review sites which covered very specific topics, I was lost and confused about who to trust for information. Asking friends only went so far and people often give conflicting opinions. I searched what a lot of people were saying but there was no easy way to find any sort of consensus. There were always questions, and I often felt like I didn't have enough information to make a good decision.

It was a year and a half ago, I had just finished my Master's thesis about using Twitter data to predict box office sales for movies based on word of mouth. Then, it occurred to me, we don't write reviews, we talk with friends. Reviews are embedded into our every day conversations, and every single person on earth does it. We talk about what we like and dislike but it doesn't happen to be written in 'review' format. What if I could collect that data and make sense of it all?

I am proud to introduce Review Signal: a new kind of review site based on what people really think.

Our goals are simple:

  • Be transparent about where each opinion comes from and source it to the original location
  • Define very clearly how things are rated in an easy to understand way

Review Signal has been launched covering our first market: web hosting reviews. We've gathered a list of some of the most popular web hosting companies and collected over 100,000 opinions expressed about them. We've analyzed them and even broken down some important and hard-to-measure values such as how satisfied people are with the support offered by a company.

Visit Web Hosting Reviews at Review Signal

Web hosting is just the start. We would love to review everything, but we're also realistic about what we can do and how fast we can do it.  We will expand to other niches as fast as we can while still maintaining our strict quality standards.  If there is some market you're unhappy with and think we can make a difference, we would love to hear from you!  We also look forward to any and all feedback, suggestions, ideas and criticisms.  Contact us.

Kevin Ohashi

Founder