On the Impact of Predicate Complexity in Crowdsourced Classification Tasks

書誌詳細
Parent link:Web Search and Data Mining, WSDM 2021.— 2021.— [P. 67-75]
共著者: Национальный исследовательский Томский политехнический университет Институт социально-гуманитарных технологий Кафедра экономики Международная научно-образовательная лаборатория технологий улучшения благополучия пожилых людей, Национальный исследовательский Томский политехнический университет Школа базовой инженерной подготовки Отделение социально-гуманитарных наук
その他の著者: Ramirez J. Jorge, Baez M. Marcos, Casati F. Fabio, Cernuzzi L. Luca, Benatallah B. Boualem, Taran Е. А. Ekaterina Aleksandrovna, Malanina V. A. Veronika Anatolievna
要約:Title screen
This paper explores and offers guidance on a specific and relevant problem in task design for crowdsourcing: how to formulate a complex question used to classify a set of items. In micro-task markets, classification is still among the most popular tasks. We situate our work in the context of information retrieval and multi-predicate classification, i.e., classifying a set of items based on a set of conditions. Our experiments cover a wide range of tasks and domains, and also consider crowd workers alone and in tandem with machine learning classifiers. We provide empirical evidence into how the resulting classification performance is affected by different predicate formulation strategies, emphasizing the importance of predicate formulation as a task design dimension in crowdsourcing.
言語:英語
出版事項: 2021
主題:
オンライン・アクセス:https://doi.org/10.1145/3437963.3441831
フォーマット: 電子媒体 図書の章
KOHA link:https://koha.lib.tpu.ru/cgi-bin/koha/opac-detail.pl?biblionumber=665083

MARC

LEADER 00000naa0a2200000 4500
001 665083
005 20250815112716.0
035 |a (RuTPU)RU\TPU\network\36282 
035 |a RU\TPU\network\19756 
090 |a 665083 
100 |a 20210810d2021 k y0engy50 ca 
101 0 |a eng 
102 |a US 
135 |a drcn ---uucaa 
181 0 |a i  
182 0 |a b 
200 1 |a On the Impact of Predicate Complexity in Crowdsourced Classification Tasks  |f J. Ramirez, M. Baez, F. Casati [et al.] 
203 |a Text  |c electronic 
300 |a Title screen 
320 |a [References: 53 tit.] 
330 |a This paper explores and offers guidance on a specific and relevant problem in task design for crowdsourcing: how to formulate a complex question used to classify a set of items. In micro-task markets, classification is still among the most popular tasks. We situate our work in the context of information retrieval and multi-predicate classification, i.e., classifying a set of items based on a set of conditions. Our experiments cover a wide range of tasks and domains, and also consider crowd workers alone and in tandem with machine learning classifiers. We provide empirical evidence into how the resulting classification performance is affected by different predicate formulation strategies, emphasizing the importance of predicate formulation as a task design dimension in crowdsourcing. 
463 |t Web Search and Data Mining, WSDM 2021  |o Proceedings of the 14th ACM International Conference, Virtual Event, Jerusalem, Israel, March 8-12, 2021  |v [P. 67-75]  |d 2021 
610 1 |a электронный ресурс 
610 1 |a труды учёных ТПУ 
610 1 |a crowdsourcing 
610 1 |a task design 
610 1 |a predicate complexity 
610 1 |a краудсорсинг 
701 1 |a Ramirez  |b J.  |g Jorge 
701 1 |a Baez  |b M.  |g Marcos 
701 1 |a Casati  |b F.  |c Italian economist and Professor at the University of Trento (Italy)  |c Professor of Tomsk Polytechnic University, candidate of technical Sciences  |f 1971-  |g Fabio  |3 (RuTPU)RU\TPU\pers\39820 
701 1 |a Cernuzzi  |b L.  |g Luca 
701 1 |a Benatallah  |b B.  |g Boualem 
701 1 |a Taran  |b Е. А.  |c Economist  |c Senior Lecturer of Tomsk Polytechnic University  |f 1981-  |g Ekaterina Aleksandrovna  |3 (RuTPU)RU\TPU\pers\35026  |9 18317 
701 1 |a Malanina  |b V. A.  |c economist  |c Associate Professor of Tomsk Polytechnic University, Candidate of economic sciences  |f 1977-  |g Veronika Anatolievna  |3 (RuTPU)RU\TPU\pers\34896  |9 18214 
712 0 2 |a Национальный исследовательский Томский политехнический университет  |b Институт социально-гуманитарных технологий  |b Кафедра экономики  |b Международная научно-образовательная лаборатория технологий улучшения благополучия пожилых людей  |3 (RuTPU)RU\TPU\col\21257 
712 0 2 |a Национальный исследовательский Томский политехнический университет  |b Школа базовой инженерной подготовки  |b Отделение социально-гуманитарных наук  |3 (RuTPU)RU\TPU\col\23512 
801 2 |a RU  |b 63413507  |c 20210810  |g RCR 
856 4 |u https://doi.org/10.1145/3437963.3441831 
942 |c CF