How Do You All Do Data Labelling/annotation?

Hi! First – please forgive me if this is a stupid question / solved problem, but I’m sort of new to this space, and curious. How have you all dealt with creating labelled datasets for your use cases?

E.g

  • what tool(s) did you use? I’ve looked into a few like Prolific (not free), Label studio (free), and I’ve looked at a few other websites
  • how did you approach recruiting participants/data annotators? e.g. did you work with a company like Outlier, or did you recruit contractors, or maybe you brought them on full-time?
  • Building on that, how did you handle collaboration and consensus if you used multiple annotators for the same row/task? or more broadly, quality control?

Seems like hard problems to me…would appreciate any insight or advice you have from your experiences! Thanks so much!

submitted by /u/Advanced-Park1031
[link] [comments]

Leave a Reply

Your email address will not be published. Required fields are marked *