Muwanei, Sinyinda and Ravana, Sri Devi and Hoo, Wai Lam and Kunda, Douglas (2021) The predictions of performance metrics in information retrieval: An experimental study. Malaysian Journal of Computer Science (SI). pp. 35-54. ISSN 0127-9084, DOI https://doi.org/10.22452/mjcs.sp2021no2.3.
Full text not available from this repository.Abstract
Information retrieval systems are widely used by people from all walks of life to meet diverse user needs. Hence, the ability of these retrieval systems to return the relevant information in response to user queries has been a matter of concern to the information retrieval research community. To address this concern, evaluations of these retrieval systems is extremely critical and the most popular way is the approach that employs test collections. This approach has been the popular evaluation approach in information retrieval for several decades. However, one of the limitations of this evaluation approach concerns the costly creation of relevance judgments. In recent research, this limitation was addressed by predicting performance metrics at the high cut-off depths of documents by using performance metrics computed at low cut-off depths. However, the challenge the research community is faced with is how to predict the precision and the non-cumulative gain performance metrics at the high cut-off depths of documents while using other performance metrics computed at the low cut-off depths of at most 30 documents. This study addresses this challenge by investigating the predictability of performance metrics and proposing two approaches that predict the precision and the non-cumulative discounted gain performance metrics. This study has shown that there exist dataset shifts in the performance metrics computed from different test collections. Furthermore, the proposed approaches have demonstrated better results of the ranked correlations of the predictions of performance metrics than existing research.
| Item Type: | Article |
|---|---|
| Funders: | Fundamental Research Grant Scheme (FRGS)[FP006-2020], Ministry of Education, Malaysia |
| Uncontrolled Keywords: | Performance metrics;Correlation;Prediction;Evaluation;Retrieval;High cost;Low cost |
| Subjects: | Q Science > QA Mathematics > QA75 Electronic computers. Computer science |
| Divisions: | Faculty of Computer Science & Information Technology |
| Depositing User: | Ms Zaharah Ramly |
| Date Deposited: | 26 Oct 2022 02:32 |
| Last Modified: | 26 Oct 2022 02:32 |
| URI: | http://eprints.um.edu.my/id/eprint/35384 |
Actions (login required)
 |
View Item |
