On the Impact of Predicate Complexity in Crowdsourced Classification Tasks

On the Impact of Predicate Complexity in Crowdsourced Classification Tasks

Bibliographic Details
Parent link:Web Search and Data Mining, WSDM 2021.— 2021.— [P. 67-75]
Corporate Authors: Национальный исследовательский Томский политехнический университет Институт социально-гуманитарных технологий Кафедра экономики Международная научно-образовательная лаборатория технологий улучшения благополучия пожилых людей, Национальный исследовательский Томский политехнический университет Школа базовой инженерной подготовки Отделение социально-гуманитарных наук
Other Authors: Ramirez J. Jorge, Baez M. Marcos, Casati F. Fabio, Cernuzzi L. Luca, Benatallah B. Boualem, Taran Е. А. Ekaterina Aleksandrovna, Malanina V. A. Veronika Anatolievna
Summary:Title screen
This paper explores and offers guidance on a specific and relevant problem in task design for crowdsourcing: how to formulate a complex question used to classify a set of items. In micro-task markets, classification is still among the most popular tasks. We situate our work in the context of information retrieval and multi-predicate classification, i.e., classifying a set of items based on a set of conditions. Our experiments cover a wide range of tasks and domains, and also consider crowd workers alone and in tandem with machine learning classifiers. We provide empirical evidence into how the resulting classification performance is affected by different predicate formulation strategies, emphasizing the importance of predicate formulation as a task design dimension in crowdsourcing.
Language:English
Published: 2021
Subjects:
электронный ресурс
труды учёных ТПУ
crowdsourcing
task design
predicate complexity
краудсорсинг
Online Access:https://doi.org/10.1145/3437963.3441831
Format: Electronic Book Chapter
KOHA link:https://koha.lib.tpu.ru/cgi-bin/koha/opac-detail.pl?biblionumber=665083

Similar Items

Similar Items