Review Signal Ranking Algorithm Update

In a recent article, The Rise and Fall of A Small Orange, it became quite apparent that our ranking algorithm here at Review Signal needed an update. Review Signal launched on September 25, 2012 which was almost 3.5 years ago. At launch, we had data from as early as 2011, which means this site's data is up to 5 years old today. It wasn't an issue back then because the oldest data would be at most, two years old and still relevant.

Today, our older data isn't really as relevant as it once was. A Small Orange exposed that weakness. It was an issue I knew I would have to deal with eventually, but nobody has really made the system fail until now. Since writing about The Rise and Fall of A Small Orange, I've been working hard to figure out a good way to update the ranking algorithm.

The solution I have come up with is a decay function. Older reviews will be worth a fraction of their more recent counterparts.

(1/(ABS(TIMESTAMPDIFF(YEAR, NOW(), max(timestamp)))+1))

This is the mathematical formula that Review Signal will now be using to calculate the value of a review.

An English explanation would be that for every year old the review is, it becomes worth one divided by years old. A one year or less old review would be worth 1/1 or 1.00. A two year old review would be worth 1/2 or 0.5. A 3 year old review would be worth 1/3 or .33 and so on.

This allows old reviews to still be a part of a company's rankings, but with a strong bias towards more recent reviews so that if a company starts performing poorly, it will decline faster in the rankings.

Checkout the full chart of how these changes affect the rankings and ratings of every published company.

Perhaps the most interesting column is how the Overall Ranking changed because of this algorithm update which I have included below. A Small Orange has the biggest change by a wide margin. HostWay lost a lot as well, but it was already at the bottom and the difference between 36% (old) and 27% (new) isn't very meaningful when you only fall two ranking spots.

HostMonster, Arvixe, HostGator, JustHost, BlueHost and some other EIG brands falling a bit more isn't surprising. It does highlight how old reviews were keeping them slightly higher than they should be, but none were ranked particularly well.

WebSynthesis dropping was a bit of a surprise. Still a decent rating at 62%, but a pretty substantial of 7 ranking places which dropped it from 10th to 17th.

On the other end, there is a lot less change upwards. However, Pagely got a nice little boost which jumped it 8 places upwards to 12th.

Then there's MochaHost, which has the dubious honor of jumping up one slot, to not be the absolute worst company we track, which is now Arvixe.

The following two tabs change content below.

• Bio
• Latest Posts

Kevin Ohashi

Founder at Review Signal

Kevin Ohashi is the geek-in-charge at Review Signal. He is passionate about making data meaningful for consumers. Kevin is based in Washington, DC.

Latest posts by Kevin Ohashi (see all)

• Analyzing Digital Ocean’s First Major Move with Cloudways - February 28, 2023
• Removing old companies - June 28, 2021
• WordPress & WooCommerce Hosting Performance Benchmarks 2021 - May 27, 2021