Find the average precision of a retrieval system - Information retrieval exercise

Find average precision of a retrieval system, information retrieval system evaluation, how to find the average precision of a search engine, information retrieval systems solved exercises

Question:

An information retrieval system give us the following result (10 documents) where the green colored are relevant ones and the red colored are non-relevant. The numbers indicate the rank of the document. Find the average precision of the given system.

1

2

3

4

5

6

7

8

9

10

 

Solution:

Precision = Fraction of retrieved documents that are relevant.

Precision = No. of relevant items retrieved/No. of retrieved items

Precision@k = Fraction of retrieved documents that are relevant in the top k documents.

Precision@k = No. of relevant docs in the top 5 results / k

Precision@5 = 3/5 = 0.6

Precision values @k is calculated and shown in the table below for all documents.

Rank	1	2	3	4	5	6	7	8	9	10
Precision	1.0	0.5	0.33	0.5	0.6	0.5	0.43	0.5	0.44	0.5

 

We need to compute Precision@k for each of the k values in the ranked document list. Then we can calculate the average precision as follows;

Average precision = average of precision@k

Here, rel(k) is 1 if the document is relevant, otherwise 0. Hence, we can add the relevant documents precision@k and calculate the average precision as follows;

Average precision = (1.0 + 0.5 + 0.6 + 0.5 + 0.5)/5 = 0.62

Average precision for the retrieved documents = 0.62

 

************

 

How to calculate the average precision of a search result?

What is average precision?

What is precision@k?

How to calculate precision@k? 

How to evaluate an information retrieval system?