BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//wp-events-plugin.com//7.4.0.1//EN
TZID:Europe/Paris
X-WR-TIMEZONE:Europe/Paris
BEGIN:VEVENT
UID:413@isdm.umontpellier.fr
DTSTART;TZID=Europe/Paris:20240912T140000
DTEND;TZID=Europe/Paris:20240912T140000
DTSTAMP:20260828T120503Z
URL:https://isdm.umontpellier.fr/events/label-ambiguity-in-crowdsourcing-f
 or-classification-and-expert-feedback-2/
SUMMARY:Label ambiguity in crowdsourcing for classification and expert feed
 back
DESCRIPTION:Room 02.124\, Building 5\, St Priest campus\n\nMachine Learning
  in Montpellier\, Theory &amp\; Practice\n\nWhile classification datasets 
 are composed of more and more data\, the need for human expertise to label
  them is still present. Crowdsourcing platforms are a way to gather expert
  feedback at a low cost. However\, the quality of these labels is not alwa
 ys guaranteed. In this thesis\, we focus on the problem of label ambiguity
  in crowdsourcing. Label ambiguity has mostly two sources: the worker’s 
 ability and the task’s difficulty. We first present a new indicator\, th
 e WAUM (Weighted Area Under the Magin)\, to detect ambiguous tasks given t
 o workers. Based on the existing AUM in the classical supervised setting\,
  this lets us explore large datasets while focusing on tasks that might re
 quire more relevant expertise or should be discarded from the actual datas
 et. We then present a new open-source python library\, PeerAnnot\, that we
  developed to handle crowdsourced datasets in image classification. Finall
 y\, we present a case study on the Pl@ntNet dataset\, where we evaluate th
 e current state of the platform’s label aggregation strategy and propose
  ways to improve it. This setting\, with a large number of tasks\, experts
  and classes\, is highly challenging for current crowdsourcing aggregation
  strategies. We report consistently better performance against competitors
  and propose a new aggregation strategy that could be used in the future t
 o improve the quality of the Pl@ntNet dataset. We also release this large 
 dataset of expert feedback that could be used to improve the quality of th
 e current aggregation methods and provide a new benchmark\,\,\n\nMachine L
 earning in Montpellier\, Theory &amp\; Practice
ATTACH;FMTTYPE=image/jpeg:https://isdm.umontpellier.fr/wp-content/uploads/
 2026/06/ml-mtp-gC78d5.png
CATEGORIES:ML MTP
LOCATION:Room 02.124 Building 5\, St Priest Campus\, Montpellier\, 34000\, 
 France
X-APPLE-STRUCTURED-LOCATION;VALUE=URI;X-ADDRESS=St Priest Campus\, Montpell
 ier\, 34000\, France;X-APPLE-RADIUS=100;X-TITLE=Room 02.124 Building 5:geo
 :0,0
END:VEVENT
BEGIN:VTIMEZONE
TZID:Europe/Paris
X-LIC-LOCATION:Europe/Paris
BEGIN:DAYLIGHT
DTSTART:20240331T030000
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
END:DAYLIGHT
END:VTIMEZONE
END:VCALENDAR