Joseph, Minnu Helen and Ravana, Sri Devi (2024) Reliable Information Retrieval Systems Performance Evaluation: A Review. IEEE Access, 12. pp. 51740-51751. ISSN 2169-3536, DOI https://doi.org/10.1109/ACCESS.2024.3377239.
Full text not available from this repository.Abstract
With the progressive and availability of various search tools, interest in the evaluation of information retrieval based on user perspective has grown tremendously among researchers. The Information Retrieval System Evaluation is done through Cranfield-paradigm in which the test collections provide the foundation of the evaluation process. The test collections consist of a document corpus, topics, and a set of relevant judgments. The relevant judgments are the documents which retrieved from the test collections based on the topics. The accuracy of the evaluation process is based on the number of relevant documents in the relevance judgment set, called qrels. This paper presents a comprehensive study, which discusses the various ways to improve the number of relevant documents in the qrels to improve the quality of qrels and through that increase the accuracy of the evaluation process. Different ways in which each methodology was performed to retrieve more relevant documents were categorized, described, and analyzed, resulting in an inclusive flow of these methodologies.
| Item Type: | Article |
|---|---|
| Funders: | Ministry of Higher Education Malaysia via Fundamental Research Grant Scheme |
| Uncontrolled Keywords: | Information retrieval; Costs; System performance; Reliability; XML; Web sites; Testing; Document handling; Performance evaluation; Document similarity; human accessors; information retrieval; information systems evaluation; pooling; topics |
| Subjects: | Q Science > QA Mathematics > QA75 Electronic computers. Computer science |
| Divisions: | Faculty of Computer Science & Information Technology |
| Depositing User: | Ms. Juhaida Abd Rahim |
| Date Deposited: | 13 Nov 2024 08:39 |
| Last Modified: | 13 Nov 2024 08:39 |
| URI: | http://eprints.um.edu.my/id/eprint/45868 |
Actions (login required)
 |
View Item |
