Designing a better scoring algorithm
A straight average gives the same weight to all reviews, even though your business policies, catalog, shipping options, and other things may have changed over time. A review that’s three years old isn’t as valuable as a review that was posted two days ago. And this approach can work both ways, ensuring a retailer isn’t punished for issues it has remedied, while not allowing a company to depend on outdated but positive reviews.
We wanted to do something better:
Design a system that removes flaws from a ratings average but still be clear and open about it.
Our goals for the ResellerRatings algorithm:
- Make sure ratings reflect current consumer voices
- Track retailer histories without punishing them for resolved issues or rewarding them for outdated positive feedback
- Eliminate troughs and peaks, ensuring a smooth and easy-to-understand reflection of all reviews
- Help merchants take an end-to-end approach to customer service
Ratings and Time
While our algorithm weighs recent reviews more heavily, it also ensures that improving a store score is not as easy as accumulating new positive reviews. Likewise, a few recent low scores will not hurt retailers unfairly. The ResellerRatings algorithm considers both time and review count across the spectrum of a full year. Data more than a year old also impacts the score, but to a much lesser extent.
So how should you read reviews?
Snapshot aggregates are useful sometimes, but smart buyers should always look deeper. ResellerRatings tries to equip you, the consumer, with a variety of tools in order to better understand the retail environment.
We suggest you always look beyond the score. Look for more context. A retailer may have a high rating, but no reviews less than eight months old. Or a store might have a number of bad reviews posted recently, but more positive scores historically. Everything needs to be taken contextually even though the score is designed to be a current, snapshot and indicator of how well your experience can be. Benchmark data, which includes returnability, pricing, and other factors, are also helpful for detail and context.
How is a retailer’s timeline calculated into the overall score?
We place a higher value on reviews posted within the last year. Reviews posted within the last three months are particularly valuable.
As a retailer, are reviews that are more than a year old still reflected in my overall score?
Yes, reviews more than a year old still factor into your overall score. Generally speaking, it’s beneficial to have a long trail of customer feedback.
Can I still see the average data of reviews?
Yes, you can see monthly averages over in your rating history. You can also view a ratings history for all the benchmarks we collect.
How often do you change the algorithm?
We rarely adjust the algorithm, but when we do it’s always done based on feedback data.
If I have a few recent negative reviews, but generally good historical reviews, how will this impact my score?
If you have a high number of reviews over the past year, negative reviews will affect you less. But the same is true of positive reviews—a few good ones won’t undo a year of negative feedback. If you don't have enough content in very recent timeframe, ResellerRatings averages the review count until we get enough to meet a minimum threshold. This ensures no extreme dips or peaks in the data.
Verified Reviews vs Non Verified
ResellerRatings believes in enabling an open network for collecting user-generated content. That is, you don't have to be a paying member to collect reviews, though being a paying member affords you additional review acquisition capabilities.
The vast majority of reviews on ResellerRatings are verified. You can see which reviews are verified by looking for the green verification badge on the review. Merchants can't pick and choose which reviews go on their public-facing storefront page, as they are required to adhere to our strict collection guidelines. A verified review is effectively a post fulfillment cycle. That means it comes from someone who has paid for a product or service from the merchant, and has had the service completed or the product delivered to them. Verified reviews have been sourced with extra data including an invoice, order ID, or other consumer information, and has gone through both an algorithmic and manual anti-fraud process.
Verified & Authentic
These are also known as “unsolicited” reviews. These reviews come to ResellerRatings through an organic process in which the consumer leaves a review directly on the site. All reviews posted on ResellerRatings, including organic reviews, are subject to a variety of anti-fraud measures.
If a retailer is a member of the ResellerRatings merchant program and adheres to the ethics and code of conduct rules, that retailer is allowed to solicit anonymous reviews. Many retailers have very strong privacy rules in place for protecting consumers, such as not allowing their identifiable information to be shared or posted in a review. We also work to protect consumers, and balance the need to promote customer voices while ensuring their privacy is protected.
Retailers who do not participate in the merchant program cannot solicit anonymous reviews. Unsolicited reviews for non-member merchants are identified by the generic term “Shopper” or “User” in place of a traditional username.
Removing Reviews and Maintaining Review Integrity
We believe in bettering the consumer shopping experience. To help achieve that goal, we developed a two-way street between shoppers and sellers. Sellers can engage with reviewers by responding privately or posting a public reply. Sellers are also given 48 hours to review new posts in order to prioritize them for follow-up and flag content for certain terms.
These terms include legal notifications for removal, personally identifiable information, etc. Reviews that are marked as fraudulent or as being violation of our policies are made inaccessible to the merchant and removed. Our team evaluates each of these flagged reviews on a case-by-case basis.
When a review is removed, we attempt to notify the author and provide them an opportunity to modify the review so it can be resubmitted.
Where did my review go!!?
Terms for Review Removal
|| 1. Review does not comply with applicable laws and regulations.
||2. Review contains vulgar, profane, obscene or offensive content.
||3. Review does not pertain to the business itself (i.e., product or shipping review).
||4. Review contains personally identifiable information ("PII") that can be used to uniquely identify, contact, or location a single person.
||5. Review has been posted multiple times or more than once by the same person.
||6. Reviewer is not an actual customer (limited to reviews posted within the past twelve (12) months).
||7. Reviewer is a current or former employee, or otherwise related to the company being reviewed.
||8. Review contains confidential, written communications between any parties.
||9. Review contains references to other businesses or competitors.
||10. Review posted one calendar year after transaction date it describes.
||11. Star rating does not match the user's sentiment (e.g., a negative star rating accompanied by a positive user comment).
Advertising with ResellerRatings
Our advertising policy is pretty simple. Merchant Members have claim to their storefront listing and can promote various offers or calls to action on their storefront.
Merchants who have and have maintained exceptional ratings generally have an oportunity to invoke better advertsing tool sets and features like Elite Merchant Seals.
While this feature is always evolving there are several places where a client can advertise and feature reviews. Clients are free to feature any type of review on their page.
Clients can also push other content on their page such as their facebook wall and other social media connections.
My company isn't listed, can you please add it to the site?
As a passive aggregator of consumer opinion, ResellerRatings provides the technology that powers the site, while our registered users provide 100% of the evaluations and are responsible for adding new companies to the site. The best way to get your company listed is to let your customers know about the site and encourage them to submit evaluations for your company. Some merchants go about that by putting a logo graphic link on their final order confirmation screen or home page, or perhaps a text message in their order email confirmations along the lines of, "Thanks for your purchase, be sure to evaluate us at ResellerRatings.com".
Can I offer a customer goods, services, or monies, to alter a negative review to a positive review or to remove their review?
No, you may not do this. While we would never stand in the way of a merchant working to resolve customer complaints and we encourage you to correct any errors and to satisfy your customers whenever possible, you may not request that a customer alter or remove their review in exchange for monetary consideration of any kind. Customers are always able to edit their reviews at any time should they choose to do so. If this policy is violated, some possible actions include: posting your solicitations to your company's reviews page, or revoking any or all features of your complimentary ResellerRatings merchant account.
Can I hold a contest or offer discounts to encourage my customers to submit reviews to ResellerRatings.com?
Although we encourage all merchants to use our exit-survey program to solicit reviews from your customers at the point of sale, we do allow merchants to solicit reviews in exchange for contests or other incentives as long as the merchant does not specifically solicit positive or good reviews, and does not require positive or good reviews in order for the customer to qualify for the merchant's incentive. The merchant must also only solicit reviews from existing customers, not prospective buyers. Merchants must wait 7 days post-transaction before soliciting a review from a customer, because the customer must have had an opportunity to receive their merchandise before writing a review. Merchants that violate these rules will have a note posted to their reviews page explaining the details of the disallowed solicitation and any reviews submitted in connection with a disallowed solicitation will be promptly removed.
My company only has a few evaluations. How can I encourage my customers to post evaluations?.
The #1 way to increase reviews is to use our exit survey feature. This feature displays a pop-up survey window to your customers at the point of sale, asking them to respond to a few questions about their ordering experience. We then followup with the customer later for their complete review.
I would like to post a rebuttal comment in response to a customer's complaint or praise. Can I do so?
Yes, merchants are welcome to post rebuttal comments. Use our merchant account request form (found on your company's detailed page at ResellerRatings) so we can grant you permission to post rebuttal comments for your company, and then visit your company's comments page to reply to an existing comment.
I created one or more fake accounts and submitted one or more fake ratings for my own company or for another company at ResellerRatings, and now there is an "admin note" on my company's reviews page letting the public know about my fraudulent actions. What can I do about this?
There is an evaluation/comment for my company that I think violates your policies. What procedure do I go through to get it removed?
ResellerRatings rated our company unfairly. Please explain.
We do not rate companies. Our site users are 100% responsible for evaluating 100% of the companies listed at ResellerRatings.com. The people who own and operate ResellerRatings.com do not submit ANY evaluations for ANY merchants. We only maintain the technology and software that powers the site. Further, users who post evaluations are 100% legally liable for the content of their comments.
My company received a new "Very Dissatisfied" survey response and it lowered my rating much more than a "Very Satisfied" response. What's wrong with your system.
Nothing is wrong with our system. This phemonenon is best illustrated using a test scores analogy:
In school, you could be going along earning test grades of 95, 95, 95, 95, then bam, you earn a 0 on a test and your 95 average suddenly blows out to 76. Then, you earn a 100 on a test but your average only goes up from a 76 to an 80. The 0 that you earned had a much greater impact on your average then the "100" that you earned. This is a simple fact of the arithmetic mean.
I want my company removed from the site.
We did not add your company to the site (one of our registered users did) and therefore we do not remove companies from the site.
I would like to have more than one account to modify my reseller information and post replies to comments.
Please make an additional request for a reseller admin account here.
Do merchants have control over which reviews they want to appear on ResellerRatings?
Absolutely not! One hundred percent of reviews submitted to ResellerRatings.com, from users with verified email addresses, will appear on our site within 48 hours unless the review violates one of our published policies. In those rare cases, we always contact the reviewer to bring their review into compliance with our policies. If you see a message board post or other online discussion claiming that merchants can manipulate reviews in any way, please let us know about it so we can address the concern. Merchants cannot and have never been able to control which reviews appear on ResellerRatings.
I want to evaluate a company. How do I go about it?
Go to ResellerRatings.com's homepage. Find the company you wish to evaluate, either by searching for its name in the search box or by browsing our list of companies. Click on the "Rate this Store" button on the company's reviews page. If you have an account, enter your user name and password. When the form appears, follow the instructions. If you don't have an account, click on the "register" button and follow the instructions to get a free account. After you register, you will receive an email to confirm your account. After your account is confirmed (you only need to confirm your account once), you will be presented with the evaluation form.
Why isn’t the review I posted showing up on the site?
Reviews are posted within 48 hours of submission. If you don't see your review after 48 hours, make sure that you have activated your account by clicking on the link in the email. In the unlikely event that your review violated one or more of our policies, it may have been deactivated until you can bring it into compliance with our policies. ResellerRatings will contact you in that case. Please feel free to contact us if you still have any questions about missing reviews.
I evaluated a company in the past but my opinion has since changed. How do I update my previous evaluation?
If you submitted an evaluation for a company on or after March 11, 2002, you can change your answers to the survey questions and edit your comments. Simply go to that company's "Rate This Store" button and the system will display your evaluation for you to edit.
ResellerRatings rated a company unfairly. Please explain.
We do not rate companies. Our site users are 100% responsible for evaluating all of the companies listed at ResellerRatings.com. They are also liable for their comments. The ResellerRatings team does not submit any evaluations for any companies.
Why don’t some companies have an overall score?
In order to ensure statistical accuracy, a company's overall score is not tabulated until that company has received 20 or more evaluations.
My evaluation was removed. Why is that?
All reviews must comply with our published policies. As a general rule, evaluations are rarely removed, and we always contact reviewers whenever possible to allow them to bring their reviews into compliance with our policies.
How do I know that the companies haven't submitted evaluations for themselves?
We use a variety of techniques to prevent fraud, including algorithmic processes and the manual review of selected evaluations. We also require email verification and customer invoices from users. And, of course, any fraudulent evaluations are removed.
I think ResellerRatings is a valuable service. Can I link to the site on my homepage?
Yes! Feel free to link to ResellerRatings.
Do you take kickbacks from retailers to give them better ratings?
No. We do not evaluate any retailers on ResellerRatings.com; only our registered users can submit evaluations. ResellerRatings was created to help buyers make educated purchasing decisions, and we strive to maintain the highest ethical standards in that endeavor.