years ago (before the current generative AI wave) I’d seen this person start a website for crowdsourced image annotations, I thought that was a great idea so I tried to support by becoming a user, when I had spare moments I’d go annotate. Killed a lot of time doing that during pandemic lockdowns etc. There around 300,000 polygonal outlines here accumulated over many years. to view them you must search for specific labels ; there’s a few hundred listed in the system and a backlog of new label requests hidden from public view. there is an export feature<\/p>\n
https:\/\/imagemonkey.io<\/a><\/p>\n example .. roads\/pavements in street scenes (“rework” mode will show you outlines, you can also go to “dataset->explore” to browse or export)<\/p>\n https:\/\/imagemonkey.io\/annotate?mode=browse&view=unified&query=road%7Cpavement&search_option=rework<\/a><\/p>\n it’s also possible to get the annotations out in batches via a python API<\/p>\n https:\/\/github.com\/ImageMonkey\/imagemonkey-libs\/blob\/master\/python\/snippets\/export.py<\/a><\/p>\n i’m worried the owner might get disheartened from a sense of futility (so few contributors, and now there are really powerful foundation models available including image to text)<\/p>\n but I figure “every little helps”, it would be useful to get this data out into a format or location where it can feed back into training, maybe even if it’s obscure and not yet in training sets it could be used for benchmarking or testing other models<\/p>\n When the site was started the author imagined a tool for automatically fine-tuning some vision nets for specific labels, I’d wanted to broaden it to become more general. the label list did grow and there’s probably a couple of hundred more that would make sense to make ‘live’<\/p>\n There’s also an aspect that these generative AI models get accused of theft, so the more deliberate voluntary data there is out there the better. I’d guess that you could mix image annotations somehow into the pretraining data for multimodal models, right? I’m also aware that you can reduce the number of images needed to train image-generators if you have polygonal annotations aswell as image\/descriptions-text pairs.<\/p>\n Just before the diffusion craze kicked off I’d had some attempts at trying to train small vision nets myself from scratch (rtx3080) but could only get so far. When stable diffusion came out I figured my own attemtps to train things were futile.<\/p>\n Here’s a thread where I documented my training attempt for the site owner<\/p>\n https:\/\/github.com\/ImageMonkey\/imagemonkey-core\/issues\/300<\/a> – in here you’ll see some visualisations of the annotations (the usual color coded overlays)<\/p>\n I think these labels today could be generalised by using an NLP model to turn the labels into vector embeddings (cluster similar labels or train image to embedding, etc)<\/p>\n The annotations would probably want to be converted to some better known format that could be loaded into other tools. they are available in his json format.<\/p>\n can anyone advise on how to get this effort fed back into some kind of visible community benefit?<\/p>\n<\/div>\n submitted by \/u\/dobkeratops <\/a>
[link]<\/a><\/span> [comments]<\/a><\/span><\/p>
