Review Signal
  1. home
  2. How It Works

How It Works

Ratings

All of our ratings are calculated from unique people. If someone writes multiple messages about a company we only count one message per type (positive/negative). We think there is room for improving this method such as recognizing groups of messages and outcomes of a series of events. However, we haven't been able to fully discuss those ideas and implement them yet.

Updated: March 3, 2016 (blog post): All of our rankings now use a decay function as described in the blog post. The formula weights reviews by age. For every year the value is multipled by (1 / years old). So 1 year old review is 1/1 or 1.00 (full value). A two year old review is 1/2 or 0.5, three years is 1/3 or 0.33, etc. This means old reviews have some value, but newer reviews will be worth more in calculating a company's ratings. All the algorithms below are the same except multiplied by this time decay coefficient.

Overall Rating

Overall Rating is calculated from the simple formula:

# Unique Positive Messages / (# Unique Positive Messages + # Unique Negative Messages)

Simply put, this an approval rating. How many people like or dislike a company.

Price/Support/Uptime Rating

These ratings are calculated exactly the same way as Overall Rating except it uses a subset of data talking about each respective aspect of the service.

Spam Filtering

Our spam filters are very aggressive, our biggest concern is bad data getting used. That means some legitimate comments will get mismarked as spam. We continually try to improve the accuracy of our spam filter and make sure as much good data gets through while keeping out the spam. We err on the side of safety (marking it spam) to make sure our data is as pristine as possible.

Most Helpful Ranking

The most helpful algorithm uses a Wilson Score Interval ( WARNING: lots of math) to rank the most helpful comments.

Classification

We take classifying messages very seriously. We established guidelines we apply to all messages when evaluating their meaning. We trained our systems to try and obey these rules as best as possible. Sometimes they make mistakes. If you see a mistake please flag it for review so we can fix it and improve our system.

Relevancy

Is the message talking about the company/product/service?

"Company is just made my life awesome with their product!" Relevant.

"Just watched the game at Company Stadium and it was awesome." Irrelevant.


Is the message talking about something relevant to the operation?

"Service just made my life so easy." Relevant.

"Just left happy hour at Company." Irrelevant.


What counts as a Positive Review?

  • Favorable Remark About the Company/Product/Service
  • Financial support: buying or using Company/Product/Service
  • Recommendation: suggesting someone buy, use or look at a Company/Product/Service

What counts as a Negative Review?

  • Unfavorable Remark About the Company/Product/Service
  • Problems/Errors with the Company/Product/Service
  • Complaints about the Company/Product/Service
  • Actively recommending against a Company/Product/Service

Final Issues

Can the message be interpreted in a context independent manner?

"I can't believe Company Supports Issue." It's unclear whether this is good or bad from the message alone. It might be understood with extra context, but since it's missing from the message, it cannot be used as a review.


Manners/Etiquette Issues:

"Thank you Company." Giving thanks can be used for etiquette, positive and negative (sarcasm) statements. We ignore them unless there is more information in the review to indicate intent.


Spokespeople/Employees:

All messages from people financially or otherwise connected to the company will not be counted in the reviews.

How We Make Money

Our plan is to make money from referring customers to good (hopefully!) companies. We receive commissions for signing up customers to a service. We try and make a deal with all the listed companies so it doesn't matter who you choose to use. That said, we will never compromise our rankings and display based on any sort of advertising or referral deal. We may also use/try other methods like advertising.

Our first and foremost goal is to provide you, the user, the best information available. We try to be as transparent as possible about how we make money and how this site works so that you understand what we do and why.

Advertising

Ads may be placed on the site in some categories. Currently, there are no advertisements on Review Signal. We would rather be ad-free as long as other less intrusive options cover our costs. However, as we expand some advertising might be necessary for some categories.

Affiliate Programs

This is our favorite method of earning money because it doesn't intrude on our users' experience. If we have an affiliate deal setup with a company and you click from our site to theirs and sign up we earn a commission. With affiliate deals setup we make money by helping you find the company that best matches your needs. It's a win, win, win scenario for you, us and the company you choose.

Data

Our data is collected from public sources only. We are not interested in taking your personal conversations/email/private content in any way, shape or form.

Our primary source(s) for data are:

  • Twitter

We also want to respect your right to privacy, if you have a message listed here and wish for it to be removed please contact us and leave a way for us to contact you to verify your identity so we can remove it. You may only remove messages that you wrote.

Company Listings

We receive a lot of interest in becoming listed on Review Signal. One of the limitations to our system is it needs a lot of data to be accurate. So many companies simply aren't a good fit for our systems.

Reasons your company may not be listed

  • Too new
  • Not enough data
  • English isn't the primary language

Things that won't impact your listing eligibility

  • Having an affiliate program
  • How much your affiliate program pays
  • How old your company is
  • How many reviews you have on other websites
  • Trying to just bribe us. If you feel like throwing your money away, give it to a worthy cause: EFF

You are welcome to Contact Us and ask about listing.

If you look promising, we may add you to our tracking system. But even that doesn't guarantee a listing at any point in the future. It simply means you might become eligible in the future as we collect more data.

If you're still interested in getting hosting company related updates and opportunities to participate on Review Signal please sign up for our Web Hosting Company Newsletter.