Please use this identifier to cite or link to this item: https://hdl.handle.net/20.500.11851/2655
Full metadata record
DC FieldValueLanguage
dc.contributor.authorRahman, M. Mustafizur-
dc.contributor.authorKutlu, Mücahid-
dc.contributor.authorLease, Matthew-
dc.date.accessioned2019-12-25T14:01:59Z
dc.date.available2019-12-25T14:01:59Z
dc.date.issued2019
dc.identifier.citationRahman, M. M., Kutlu, M., and Lease, M. (2019, May). Constructing Test Collections using Multi-armed Bandits and Active Learning. In The World Wide Web Conference (pp. 3158-3164). ACM.en_US
dc.identifier.isbn9.78145E+12
dc.identifier.urihttps://dl.acm.org/citation.cfm?doid=3308558.3313675-
dc.identifier.urihttps://hdl.handle.net/20.500.11851/2655-
dc.descriptionThe Web Conference 2019 - Proceedings of the World Wide Web Conference (2019: San Francisco; United States )
dc.description.abstractWhile test collections provide the cornerstone of system-based evaluation in information retrieval, human relevance judging has become prohibitively expensive as collections have grown ever larger. Consequently, intelligently deciding which documents to judge has become increasingly important. We propose a two-phase approach to intelligent judging across topics which does not require document rankings from a shared task. In the first phase, we dynamically select the next topic to judge via a multi-armed bandit method. In the second phase, we employ active learning to select which document to judge next for that topic. Experiments on three TREC collections (varying scarcity of relevant documents) achieve ? ? 0.90 correlation for P@10 ranking and find 90% of the relevant documents at 48% of the original budget. To support reproducibility and follow-on work, we have shared our code online1. © 2019 IW3C2 (International World Wide Web Conference Committee), published under Creative Commons CC-BY 4.0 License.en_US
dc.description.sponsorshipNPRP grant from the Qatar National Research Fund [NPRP 7-1313-1-245]
dc.language.isoenen_US
dc.publisher Association for Computing Machinery, Incen_US
dc.rightsinfo:eu-repo/semantics/closedAccessen_US
dc.subjectInformation retrieval en_US
dc.subject search engines en_US
dc.subjectrelevance assessmentsen_US
dc.titleConstructing Test Collections using Multi-armed Bandits and Active Learningen_US
dc.typeConference Objecten_US
dc.departmentFaculties, Faculty of Engineering, Department of Computer Engineeringen_US
dc.departmentFakülteler, Mühendislik Fakültesi, Bilgisayar Mühendisliği Bölümütr_TR
dc.identifier.startpage3158
dc.identifier.endpage3164
dc.authorid0000-0002-4102-803X-
dc.identifier.wosWOS:000483508403033en_US
dc.identifier.scopus2-s2.0-85066910999en_US
dc.institutionauthorKutlu, Mücahid-
dc.identifier.doi10.1145/3308558.3313675-
dc.relation.publicationcategoryKonferans Öğesi - Uluslararası - Kurum Öğretim Elemanıen_US
item.openairecristypehttp://purl.org/coar/resource_type/c_18cf-
item.grantfulltextnone-
item.fulltextNo Fulltext-
item.openairetypeConference Object-
item.cerifentitytypePublications-
item.languageiso639-1en-
crisitem.author.dept02.3. Department of Computer Engineering-
Appears in Collections:Bilgisayar Mühendisliği Bölümü / Department of Computer Engineering
Scopus İndeksli Yayınlar Koleksiyonu / Scopus Indexed Publications Collection
WoS İndeksli Yayınlar Koleksiyonu / WoS Indexed Publications Collection
Show simple item record



CORE Recommender

SCOPUSTM   
Citations

4
checked on Mar 23, 2024

WEB OF SCIENCETM
Citations

9
checked on Mar 23, 2024

Page view(s)

28
checked on Mar 25, 2024

Google ScholarTM

Check




Altmetric


Items in GCRIS Repository are protected by copyright, with all rights reserved, unless otherwise indicated.