• Become a Tester
  • Sign in
  • The Testlio Advantage
    • Why We Are Different

      See what makes Testlio the leading choice for enterprises.

    • Our Solutions

      A breakdown of our core QA services and capabilities.

    • The Testlio Network

      Learn about our curated community of expert testers.

    • Our Platform

      Dive into the technology behind Testlio’s testing engine.

    • Why Crowdsourced Testing?

      Discover how our managed model drives quality at scale.

  • Our Solutions
    • By Capability
      • Manual Testing
      • Test Automation
      • Payments Testing
      • AI Testing
      • Functional Testing
      • Regression Testing
      • Accessibility Testing
      • Localization Testing
      • Customer Journey Testing
      • Usability Testing
    • By Technology
      • Mobile App Testing
      • Web Testing
      • Location Testing
      • Stream Testing
      • Device Testing
      • Voice Testing
    • By Industry
      • Commerce & Retail
      • Finance & Banking
      • Health & Wellness
      • Media & Entertainment
      • Learning & Education
      • Mobility & Travel
      • Software & Services
    • By Job Function
      • Engineering
      • QA Teams
      • Product Teams
  • Resources
    • Blog

      Insights, trends, and expert perspectives on modern software testing.

    • Webinars & Events

      Live and on-demand sessions with QA leaders and product experts.

    • Case Studies

      Real-world examples of how Testlio helps teams deliver quality at scale.

Contact sales
Contact sales
  • TestLion Pride

New Tester Rating System And What It Means For Testers!

Kristel Kruustük , Kristel is Testlio's co-founder.
November 11th, 2015

 

We’re excited to announce a new rating system for our testers!

 

Why?

Many testers were being unfairly rated. For example, there could be a tester who submits numerous high value bugs with a great bug report, but maybe they didn’t follow instructions for a few issues. Instead of ending up with a test cycle rating of 4.8 which they would’ve probably deserved, they end up with a 3.7.

 

We felt this was incredibly unfair. What matters is the value you provided to the project and customers. Now we have introduced a weighted system, meaning factors that we find valuable will have different weights based on their importance.

What’s New

As a tester, your testing process won’t be any different. You’ll continue to report high quality issues that deliver value. However what you’re probably curious about is what happens behind the scenes. In other words, what makes your score.

Bug Depth Redefined

Previously you were rated on how many high priority issues you reported. We listened and agree that this was the wrong approach on how to rate our testers. Sometimes there are projects where no high priority issues exist. Testers ended up avoiding these projects because they knew no high priority issues would be in there and therefore not get a great rating.

Now your rating is compared to the relative value you bring to the test cycle. So for example, if you are in a project and you submit 3 medium severity bugs (assuming they’re all high quality) in a project where no high severity bugs exist, then you will receive maximum score in the “issue depth” portion of your rating formula.

Bug Report Quality

Something else we decided to try to make more concrete is the Bug Report Quality. Each of your issues will be individually rated on how clear the bug reports are. And then your Bug Report Quality score will be the average of all of the approved issues you submitted in the test cycle.

If you’re looking to keep your bug quality score high, please follow these tips which we rate on:

  • Clear title (information: as little as possible as much as needed)
  • Clear and valid steps (one should be able to get the point reading it the first time)
  • Valid proof (logs, screenshot, video) that match the issue report
  • Valid estimation of severity
  • Label, Feature, Environment and other requirements are met

Cover all of these and your Bug Report Quality will be max points.

New Feedback Screen

Previously you were shown your feedback given to you per test cycle.

You’ll be shown each comment written on how you can improve your approved issues. Don’t worry, you can still see the feedback you receive per test cycle as well!

Please note as well that we decided to remove the ability for a tester to see their rating per test cycle. At Testlio we’re strong supporters of improving our tester community through constructive feedback, not by arbitrary ratings and grades. You’ll still be able to see your overall rating, but the focus should be on the feedback you receive from a test lead on how to improve.

What’s Next?

If you’re a good tester, you should be consistently seeing higher and fairer ratings. In which case, congratulations and good job!

On our side, we’re going to be continuing to monitor how the rating system is working and making sure all is going well. We know this is a very delicate part of your user experience, and we’re fully attentive on making sure that it rolls out smoothly.

 

We would love to hear your thoughts in the comments below!

You may also like

  • Culture Meet a Test Lead: Tuyet Nhu
  • Culture Continuing Testlio’s Steadfast Support of Ukraine
  • Culture Meet a Test Lead: Delfina
  • Culture A Day in the Life of a TestLion: Renata Amatore, Director of DX Usability Testing
  • Culture Ignite social impact strategy uses equity to democratize tech
  • LinkedIn
Company
  • About Testlio
  • Leadership Team
  • News
  • Partnerships
  • Careers
  • Become a Tester
  • Platform Login
  • Contact Us
Resources
  • Blog
  • Webinars & Events
  • Case Studies
Legal
  • Notices
  • Privacy Policy
  • Terms of Use
  • Modern Slavery Policy

Subscribe
to our newsletter