Hi, everyone.
I’ve been looking into dictionary data for a language-learning app I’m building, and I’ve run into something surprising.
There are tons of dictionary apps, browser extensions, reading tools, and language-learning apps that support many languages.
But when I actually try to find good open dictionary data myself, it’s surprisingly difficult.
Even for something as common as English -> Chinese, it’s hard to find a dataset that has all of these:
- headwords
- translations
- part of speech
- multiple senses
- example sentences
- a clear license that allows reuse
I’ve found things like:
- Wiktionary / Wiktextract / Kaikki
- WordNet / Open Multilingual WordNet
- ECDICT for English–Chinese
- CC-CEDICT for Chinese–English
- Tatoeba for bilingual example sentences
- FreeDict
- various language-specific projects
But they all seem to cover different pieces of the puzzle, and the quality and coverage vary quite a bit.
Meanwhile, I regularly see relatively small dictionary apps or browser extensions claiming to support English, Chinese, Japanese, Korean, French, German, Spanish, Italian, Russian, Arabic, etc.
So I’m genuinely curious:
Where does all of that dictionary data usually come from?
I’m especially interested in hearing from anyone who has actually built a dictionary app.
What data sources did you use, and how did you handle licensing?
Btw, I know Cambridge, Oxford, and other major dictionaries offer paid APIs or data licensing, but they’re quite expensive. I assume many smaller dictionary apps probably use their own databases or combine multiple data sources somehow.
Any pointers would be greatly appreciated. Thanks!!!
submitted by /u/SnooPears7307
[link] [comments]