Please use this identifier to cite or link to this item:
https://hdl.handle.net/20.500.11851/2656
Full metadata record
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Gupta, S. | - |
dc.contributor.author | Kutlu, Mücahid | - |
dc.contributor.author | Khetan, V. | - |
dc.contributor.author | Lease, M. | - |
dc.date.accessioned | 2019-12-25T14:01:59Z | |
dc.date.available | 2019-12-25T14:01:59Z | |
dc.date.issued | 2019 | |
dc.identifier.citation | Gupta, S., Kutlu, M., Khetan, V., and Lease, M. (2019, April). Correlation, Prediction and Ranking of Evaluation Metrics in Information Retrieval. In European Conference on Information Retrieval (pp. 636-651). Springer, Cham. | en_US |
dc.identifier.isbn | 9.78303E+12 | |
dc.identifier.issn | 3029743 | |
dc.identifier.uri | https://link.springer.com/chapter/10.1007%2F978-3-030-15712-8_41 | - |
dc.identifier.uri | https://hdl.handle.net/20.500.11851/2656 | - |
dc.description | 41st European Conference on Information Retrieval, ECIR ( 2019: Cologne; Germany ) | |
dc.description.abstract | Given limited time and space, IR studies often report few evaluation metrics which must be carefully selected. To inform such selection, we first quantify correlation between 23 popular IR metrics on 8 TREC test collections. Next, we investigate prediction of unreported metrics: given 1–3 metrics, we assess the best predictors for 10 others. We show that accurate prediction of MAP, P@10, and RBP can be achieved using 2–3 other metrics. We further explore whether high-cost evaluation measures can be predicted using low-cost measures. We show RBP(p = 0.95) at cutoff depth 1000 can be accurately predicted given measures computed at depth 30. Lastly, we present a novel model for ranking evaluation metrics based on covariance, enabling selection of a set of metrics that are most informative and distinctive. A greedy-forward approach is guaranteed to yield sub-modular results, while an iterative-backward method is empirically found to achieve the best results. © Springer Nature Switzerland AG 2019. | en_US |
dc.description.sponsorship | Acknowledgements. This work was made possible by NPRP grant# NPRP 7-1313-1-245 from the Qatar National Research Fund (a member of the Qatar Foundation). The statements made herein are solely the responsibility of the authors. | |
dc.language.iso | en | en_US |
dc.publisher | Springer Verlag | en_US |
dc.relation.ispartof | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) | en_US |
dc.rights | info:eu-repo/semantics/closedAccess | en_US |
dc.subject | Information retrieval | en_US |
dc.subject | search engines | en_US |
dc.subject | relevance assessments | en_US |
dc.title | Correlation, prediction and ranking of evaluation metrics in information retrieval | en_US |
dc.type | Conference Object | en_US |
dc.department | Faculties, Faculty of Engineering, Department of Computer Engineering | en_US |
dc.department | Fakülteler, Mühendislik Fakültesi, Bilgisayar Mühendisliği Bölümü | tr_TR |
dc.identifier.volume | 11437 | |
dc.identifier.startpage | 636 | |
dc.identifier.endpage | 651 | |
dc.authorid | 0000-0002-4102-803X | - |
dc.identifier.scopus | 2-s2.0-85064873308 | en_US |
dc.institutionauthor | Kutlu, Mücahid | - |
dc.identifier.doi | 10.1007/978-3-030-15712-8_41 | - |
dc.relation.publicationcategory | Konferans Öğesi - Uluslararası - Kurum Öğretim Elemanı | en_US |
dc.identifier.scopusquality | Q2 | - |
item.fulltext | No Fulltext | - |
item.grantfulltext | none | - |
item.languageiso639-1 | en | - |
item.openairetype | Conference Object | - |
item.openairecristype | http://purl.org/coar/resource_type/c_18cf | - |
item.cerifentitytype | Publications | - |
crisitem.author.dept | 02.3. Department of Computer Engineering | - |
Appears in Collections: | Bilgisayar Mühendisliği Bölümü / Department of Computer Engineering Scopus İndeksli Yayınlar Koleksiyonu / Scopus Indexed Publications Collection |
CORE Recommender
SCOPUSTM
Citations
1
checked on Sep 7, 2024
Page view(s)
108
checked on Sep 2, 2024
Google ScholarTM
Check
Altmetric
Items in GCRIS Repository are protected by copyright, with all rights reserved, unless otherwise indicated.