Rating Quality Updates

Sunday, December 8th, 2013
By Arthur Lui

UPDATE: Changes #1 and #2 have been rolled back. After hearing reasonable feedback from concerned members of the community, the choice was clear that this wasn’t the best method.

The mission of the Table Tennis Database is to provide a platform for giving ratings and reviews that are:

  1. Accurate
  2. Useful for people’s research

There are several reasons why reviews and ratings aren’t always accurate (whether accidentally or deliberately), and we’re taking steps to uphold this mission and maintain quality as much as possible. Some of the changes that we’ve made include:

  1. Users cannot give an Overall score below 5. Does anything really deserve lower than a 5/10? Let’s be reasonable; every item is useful to a particular type of player, even if it doesn’t suit your needs. Very low scores unfairly affect the average numbers drastically.
  2. Past Overall ratings below 5 have been changed to unrated (as if “overall” was left blank).
    1. We had debated changing those low ratings to 5 (the lowest allowable rating), but decided against the idea of changing a user’s ratings without permission. If your rating was affected, you can always login and update the rating.
    2. All users with affected ratings have been notified by e-mail and can update the blank “overall” ratings with a more fair score.
  3. Users cannot give a Speed score below 4 for a rubber or blade. Only a freshly cooked pancake could produce speed scores lower than 4, otherwise it’s not physically possible to be that slow.
  4. Users cannot give a Control score below 3.
  5. Already many user accounts have been shut down as suspicious ratings and reviews were detected, and hundreds of ratings have been deleted. A sophisticated algorithm is in place to detect malicious ratings and they get deleted one-by-one. There is also a sophisticated keyword detector for preventing spam.

In order for the Table Tennis Database to continue to be a great resource, we will continue our commitment to protecting rating quality.

Do you have more suggestions for maintaining the quality of the ratings and reviews?

Facebook Like

Comments

4 Responses to “Rating Quality Updates”

  1. Van Savell says:

    I must admit this decision baffles me. If the rating scale is 5 to 10 then that is acceptable to have the lowest tating as a 5. But to have a scale of 1 to 10 and automatically disallow all ratings under 5? I rated two rubbers below a 5 and I stillthink that they are very poor rubbers and deserve their rating. What this decision does is inflate all ratings so that I cannot tell any difference since everything is rated 7 or above. A true bell shaped curve would suggest that statistically very few rubbers would be rated 10 or 1 and most rubbers would get a 5 rating. The current decision does not allow me to distinquish as well anymore. Heck, I use the descriptions more than anything because numbers are so all over the place. You can do what you like with the website-its yours. But if one could query my ratings you would see my ratings are not malacious or suspicious. I would argue that while most rubbers are useful to some types of players that some rubbers are absolutely failed experiments and future consumers should know what they are buying.

    UN:F [1.7.3_972]
    Rating: 0 (from 0 votes)
    • Arthur Lui says:

      Hi Van,

      Thanks for your response. After hearing a lot of feedback from reviewers whose ratings were impacted, I definitely see that this change wasn’t ideal, and there will be some changes (though the specifics are still being debated).

      The reason for this move was to prevent the malicious ratings that were clearly untrue, but I see that we don’t want to block “fair” ratings where a low overall score is justified either.

      I’d love to hear some suggestions on how we, as a community, can maintain accurate and fair ratings and reviews.

      While it is odd to disallow ratings with a scale that goes from 0-10, “overall” was a special case because of the reason you mentioned. As long as there are no quality control issues (each sheet is different from the next, or it wears out very quickly), I’m not convinced that any rubber deserves a very low overall rating. Everyone has a different playing style, and rubber X may not work for specific players, but may be perfect for other playing styles.

      A pattern I’m seeing is that many players are giving very low “overall” ratings because the rubber didn’t perform as “expected”, or didn’t fit their style, yet had no issues that make it a “bad” rubber. Other players loved it.

      What are some ways we can ensure that ratings are fair?

      The site will continue to evolve and improve as long as we can keep a dialogue going on how to refine the algorithms and systems in place, possibly with more human intervention.

      What are your thoughts?

      UA:F [1.7.3_972]
      Rating: 0 (from 0 votes)
  2. Eduardo says:

    The scores should always be accompanied by a written explanation. You can tell the difference between malicious or novice and well intentioned or experienced. The first 2 should be deleted or edited.

    UN:F [1.7.3_972]
    Rating: 0 (from 0 votes)
  3. Neil says:

    For this rule : Users cannot give a Speed score below 4 for a rubber or blade. Only a freshly cooked pancake could produce speed scores lower than 4, otherwise it’s not physically possible to be that slow.

    Some rubbers are developed to reduce the speed of the incoming ball and produce a low speed e.g Super block / Best Anti etc. or OX rubbers. In this case why not rate the speed lower than 5? Also I have a question , For the long pips how does the spin rating system work? if the higher the spinnier or higher the more reversal (less spin)? Maybe a consideration may be made to change the spin variable to long pips and anti spin rubbers!

    Regards

    UN:F [1.7.3_972]
    Rating: 0 (from 0 votes)

Leave a Reply

Learn about the author, Arthur Lui.