Assessor is who?

Assessor - a person who, on the instructions of the developers of the search engine, performs an assessment of how well the document found corresponds to the query. Most often, assessors are not familiar with information retrieval professionally. Their qualification is close to the knowledge of average Internet users. One of the first work of assessors began to use Google in 2003. The Yandex Assessors appeared in 2006.

The order of the assessor's work

Initially, an automatic search is performed, and the search engine generates a sequence of sites according to their relevance, calculated by the robot. Then the assessor, who at this moment can be located anywhere in the world using a special program and conventional logic, evaluates the result. A rating scale is developed by each search engine independently. It often contains six or more items. The work of the assessors differs by an error of approximately 5%.

The evaluator sends the results of his activity to the search engine's headquarters, where the final rating of the document is calculated based on the joint analysis of the search robot's data and the assessor's information. The result that the assessor helps to achieve is a significant increase in the objectivity of the evaluation of documents, exclusion from the rating of resources erroneously included in the rating list on the basis of formal characteristics.

All search engines have their own staff of assessors, who constantly rotates. The most suitable assessor is a user with an average level of knowledge of the Internet. The purpose of the work he performs is to improve the functioning of search engines so that they answer the user's questions as accurately as possible.

The assessors evaluating the results of the search perform tasks. Their content is a keyword, a link and an instruction to evaluate the correspondence of a link to a given word. The specified word according to the instruction to be followed by the assessor is an action of the form "go", "make" or "learn".

The assessor must decide whether the keyword corresponds to a certain action taken by the user (making a purchase, watching a movie, listening to music) or some data that interests him.

Types of assessments assessor in "Yandex"

The Yandex's assessors give two types of evaluations:

1. Preliminary assessment. Does this document refer to pornography and does it contain malicious code. If the answer is "yes," the evaluation of the document is terminated.

2. Relevance assessment. This estimate is non-quantitative. The assessor gives his assessment by referring the document to any category:

• "Vital" - if it's an official website or an official answer to a question.
• "Useful" is a document containing data that exactly matches the search query.
• "Relevant +" - a document that meets the search query.
• "Relevant" - a document that does not exactly match the search query.
• "Irrelevant" is a document that does not match the search query.
• "Spam" - a document with signs of black optimization (attempts to deceive the search engine).
• "Not about that" is a category that is designed to separate robots that are similar for a robot, but fundamentally different for a person concepts. So, on the search query "Leo Tolstoy", the search engine should not produce documents about complete people and animals as results.

The value of assessments

The work of assessors helps in assessing the degree of accuracy of search and training of the search robot. The assessors are not able to influence the positions occupied by a particular site.

Assessor sites in their work are guided by clear instructions. It has become a rather large and complex document and is constantly being updated with new requirements.