Overview of Block Utility (BU) Rating
‌

For every BU rating task, you must

• Understand the query (release the task if you can't!).

• Assign a utility rating to ALL results.

• Assign a PQ rating to the landing page of the result block if a PQ slider is present.

• Identify duplicates (See Section 29.0).

• Flag Foreign Language (F), Porn (P), and Didn't Load (D) results.
You will develop your own workflow as you master BU rating. You may decide to identify duplicates first or assign PQ ratings before utility ratings.
Remember: If you can't understand the query and user intent, please release the task. For this reason, understanding the query is a good first step.
All blocks in BU rating have a utility slider. Please assign a utility rating for all results. Please assign an OT rating for:

• Unsolicited Foreign Language (F) results

• Unsolicited Porn (P) results

• Didn't Load (D) results

• Any other kind of useless result
Some results don't have a PQ slider. If a result block has no PQ rating slider, you do not have to give a PQ rating.
If there is a PQ slider, please assign a PQ rating based on the landing page. If you are unable to evaluate the PQ of a

Foreign Language or Didn't Load result, you should assign a PQ rating of Medium.
« Previous Next »