Muwanei, Sinyinda and Ravana, Sri Devi and Hoo, Wai Lam and Kunda, Douglas (2022) Prediction of the high-cost normalised discounted cumulative gain (nDCG) measure in information retrieval evaluation. Information Research-An International Electronic Journal, 27 (2). ISSN 1368-1613, DOI https://doi.org/10.47989/irpaper928.
Full text not available from this repository.Abstract
Introduction. Information retrieval systems are vital to meeting daily information needs of users. The effectiveness of these systems has often been evaluated using the test collections approach, despite the high evaluation costs of this approach. Recent methods have been proposed that reduce evaluation costs through the prediction of information retrieval performance measures at the higher cut-off depths using other measures computed at the lower cut-off depths. The purpose of this paper is to propose two methods that addresses the challenge of accurately predicting the normalised discounted cumulative gain (nDCG) measure. Method. Data from selected test collections of the Text REtrieval Conference was used. The proposed methods employ the gradient boosting and linear regression models trained with topic scores of measures partitioned by TREC Tracks. Analysis. To evaluate the proposed methods, the coefficient of determination, Kendall's tau and Spearman correlations were used. Results. The proposed methods provide better predictions of the nDCG measure at the higher cut-off depths while using other measures computed at the lower cut-off depths. Conclusions. These proposed methods have shown improvement in the predictions of the nDCG measure while reducing the evaluation costs.
| Item Type: | Article |
|---|---|
| Funders: | University of Malaya RU Grant (Grant No: GPF098C-2020), Fundamental Research Grant Scheme (FRGS) by Ministry of Higher Education, Malaysia (Grant No: FP006-2020) |
| Uncontrolled Keywords: | Information retrieval systems |
| Subjects: | Z Bibliography. Library Science. Information Resources > Libraries > Library science. Information science |
| Divisions: | Faculty of Computer Science & Information Technology > Department of Information System |
| Depositing User: | Ms. Juhaida Abd Rahim |
| Date Deposited: | 19 Oct 2023 03:37 |
| Last Modified: | 19 Oct 2023 03:37 |
| URI: | http://eprints.um.edu.my/id/eprint/41961 |
Actions (login required)
 |
View Item |
