An assessor is a person who, on the instructions of a search engine developer, evaluates how closely a document matches a request. Most often assessors are not familiar with information retrieval professionally. Their qualifications approach the knowledge of average Internet users. One of the first work of assessors began to use Google in 2003. Yandex assessors appeared in 2006.
Assessor work order
Initially, an automatic search takes place, and the search engine generates a sequence of sites according to their relevance calculated by the robot. Then the assessor, who can at this moment be located anywhere in the world, using a special program and ordinary logic, evaluates the result. The rating scale is developed by each search engine independently. It most often contains six or more items. The work of assessors is characterized by an error of approximately 5%.
The assessor sends the result of his activity to the headquarters of the search engine, where, based on a joint analysis of the search robot data and the assessor’s information, the final document rating is calculated. The result, which the assessor helps to achieve, is a significant increase in the objectivity of the evaluation of documents, the exclusion from the rating of resources erroneously included in the rating list based on formal attributes.
All search engines have their own staff of assessors, which exposes to constant rotation. The most suitable assessor is a user who has an average level of Internet knowledge. The purpose of the work that he performs is to improve the functioning of search engines so that they answer the user's questions as accurately as possible.
Assessors evaluating search results complete assignments. Their content is a keyword, a link, and instructions for assessing whether a link matches a given word. The given word according to the instructions that the assessor must adhere to is an action of the type “pass”, “do” or “find out”.
The assessor must decide whether the keyword corresponds to a specific action taken by the user (making a purchase, watching a movie, listening to music) or some data that interests him.
Types of assessor assessments at Yandex
Yandex assessors consistently give two types of assessments:
1. Preliminary assessment. Does this document refer to pornography and does it contain malicious code. If the answer is yes, then the evaluation of the document is terminated.
2. Assessment of relevance (compliance). This assessment is non-quantitative. The assessor gives his assessment by assigning the document to any category:
- “Vital” - if it is an official site or an official answer to a question.
- “Useful” - a document containing data that exactly matches the search query.
- "Relevant +" - a document that matches the search query.
- “Relevant-” - a document that does not quite exactly meet the search query.
- “Irrelevant” - a document that does not match the search query.
- "Spam" - a document with signs of black optimization (attempts to deceive the search engine).
- "Not about that" is a category that is designed to separate similar concepts for a robot, but fundamentally different concepts for humans. So, for the search query “Leo Tolstoy,” the search system should not produce documents about complete people and animals as results.
The value of assessors
The work of assessors helps in assessing the degree of accuracy of the search and training the search robot. Assessors are not able to influence the positions taken by a particular site.
Website accessors are guided by clear instructions in their work. It has become a fairly large and complex document and is constantly updated with new requirements.