Please use this identifier to cite or link to this item: https://hdl.handle.net/20.500.11851/4037
Title: Building Test Collections using Bandit Techniques: A Reproducibility Study
Authors: Altun, Bahadır
Kutlu, Mücahid
Keywords: Bandit methods
evaluation
information retrieval
test collections
Issue Date: Oct-2020
Publisher: Association for Computing Machinery
Source: Altun, B., and Kutlu, M. (2020, October). Building Test Collections using Bandit Techniques: A Reproducibility Study. In Proceedings of the 29th ACM International Conference on Information and Knowledge Management (pp. 1953-1956).
Abstract: The high cost of constructing test collections led many researchers to develop intelligent document selection methods to find relevant documents with fewer judgments than the standard pooling method requires. In this paper, we conduct a comprehensive set of experiments to evaluate six bandit-based document selection methods, in terms of evaluation reliability, fairness, and reusability of the resultant test collections. In our experiments, the best performing method varies across test collections, showing the importance of using diverse test collections for an accurate performance analysis. Our experiments with six test collections also show that Move-To-Front is the most robust method among the ones we investigate. © 2020 ACM.
URI: https://hdl.handle.net/20.500.11851/4037
https://dl.acm.org/doi/10.1145/3340531.3412121
ISBN: 978-145036859-9
Appears in Collections:Bilgisayar Mühendisliği Bölümü / Department of Computer Engineering
Scopus İndeksli Yayınlar Koleksiyonu / Scopus Indexed Publications Collection
WoS İndeksli Yayınlar Koleksiyonu / WoS Indexed Publications Collection

Show full item record

CORE Recommender

Page view(s)

92
checked on Feb 6, 2023

Google ScholarTM

Check

Altmetric


Items in GCRIS Repository are protected by copyright, with all rights reserved, unless otherwise indicated.