Please use this identifier to cite or link to this item:
https://hdl.handle.net/20.500.11851/2655
Full metadata record
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Rahman, M. Mustafizur | - |
dc.contributor.author | Kutlu, Mücahid | - |
dc.contributor.author | Lease, Matthew | - |
dc.date.accessioned | 2019-12-25T14:01:59Z | |
dc.date.available | 2019-12-25T14:01:59Z | |
dc.date.issued | 2019 | |
dc.identifier.citation | Rahman, M. M., Kutlu, M., and Lease, M. (2019, May). Constructing Test Collections using Multi-armed Bandits and Active Learning. In The World Wide Web Conference (pp. 3158-3164). ACM. | en_US |
dc.identifier.isbn | 9.78145E+12 | |
dc.identifier.uri | https://dl.acm.org/citation.cfm?doid=3308558.3313675 | - |
dc.identifier.uri | https://hdl.handle.net/20.500.11851/2655 | - |
dc.description | The Web Conference 2019 - Proceedings of the World Wide Web Conference (2019: San Francisco; United States ) | |
dc.description.abstract | While test collections provide the cornerstone of system-based evaluation in information retrieval, human relevance judging has become prohibitively expensive as collections have grown ever larger. Consequently, intelligently deciding which documents to judge has become increasingly important. We propose a two-phase approach to intelligent judging across topics which does not require document rankings from a shared task. In the first phase, we dynamically select the next topic to judge via a multi-armed bandit method. In the second phase, we employ active learning to select which document to judge next for that topic. Experiments on three TREC collections (varying scarcity of relevant documents) achieve ? ? 0.90 correlation for P@10 ranking and find 90% of the relevant documents at 48% of the original budget. To support reproducibility and follow-on work, we have shared our code online1. © 2019 IW3C2 (International World Wide Web Conference Committee), published under Creative Commons CC-BY 4.0 License. | en_US |
dc.description.sponsorship | NPRP grant from the Qatar National Research Fund [NPRP 7-1313-1-245] | |
dc.language.iso | en | en_US |
dc.publisher | Association for Computing Machinery, Inc | en_US |
dc.rights | info:eu-repo/semantics/closedAccess | en_US |
dc.subject | Information retrieval | en_US |
dc.subject | search engines | en_US |
dc.subject | relevance assessments | en_US |
dc.title | Constructing Test Collections Using Multi-Armed Bandits and Active Learning | en_US |
dc.type | Conference Object | en_US |
dc.department | Faculties, Faculty of Engineering, Department of Computer Engineering | en_US |
dc.department | Fakülteler, Mühendislik Fakültesi, Bilgisayar Mühendisliği Bölümü | tr_TR |
dc.identifier.startpage | 3158 | |
dc.identifier.endpage | 3164 | |
dc.authorid | 0000-0002-4102-803X | - |
dc.identifier.wos | WOS:000483508403033 | en_US |
dc.identifier.scopus | 2-s2.0-85066910999 | en_US |
dc.institutionauthor | Kutlu, Mücahid | - |
dc.identifier.doi | 10.1145/3308558.3313675 | - |
dc.relation.publicationcategory | Konferans Öğesi - Uluslararası - Kurum Öğretim Elemanı | en_US |
item.openairetype | Conference Object | - |
item.languageiso639-1 | en | - |
item.grantfulltext | none | - |
item.fulltext | No Fulltext | - |
item.openairecristype | http://purl.org/coar/resource_type/c_18cf | - |
item.cerifentitytype | Publications | - |
crisitem.author.dept | 02.3. Department of Computer Engineering | - |
Appears in Collections: | Bilgisayar Mühendisliği Bölümü / Department of Computer Engineering Scopus İndeksli Yayınlar Koleksiyonu / Scopus Indexed Publications Collection WoS İndeksli Yayınlar Koleksiyonu / WoS Indexed Publications Collection |
CORE Recommender
SCOPUSTM
Citations
4
checked on Dec 21, 2024
WEB OF SCIENCETM
Citations
10
checked on Dec 21, 2024
Page view(s)
60
checked on Dec 23, 2024
Google ScholarTM
Check
Altmetric
Items in GCRIS Repository are protected by copyright, with all rights reserved, unless otherwise indicated.