Hi, everyone.
I’ve been looking into dictionary data for a language-learning app I’m building, and I’ve run into something surprising.<\/p>\n
There are tons of dictionary apps, browser extensions, reading tools, and language-learning apps that support many languages.<\/p>\n
But when I actually try to find good open dictionary data myself, it’s surprisingly difficult.<\/p>\n
Even for something as common as English -> Chinese, it’s hard to find a dataset that has all of these:<\/p>\n
- \n
- headwords<\/li>\n
- translations<\/li>\n
- part of speech<\/li>\n
- multiple senses<\/li>\n
- example sentences<\/li>\n
- a clear license that allows reuse<\/li>\n<\/ul>\n
I’ve found things like:<\/p>\n
- \n
- Wiktionary \/ Wiktextract \/ Kaikki<\/li>\n
- WordNet \/ Open Multilingual WordNet<\/li>\n
- ECDICT for English\u2013Chinese<\/li>\n
- CC-CEDICT for Chinese\u2013English<\/li>\n
- Tatoeba for bilingual example sentences<\/li>\n
- FreeDict<\/li>\n
- various language-specific projects<\/li>\n<\/ul>\n
But they all seem to cover different pieces of the puzzle, and the quality and coverage vary quite a bit.<\/p>\n
Meanwhile, I regularly see relatively small dictionary apps or browser extensions claiming to support English, Chinese, Japanese, Korean, French, German, Spanish, Italian, Russian, Arabic, etc.<\/p>\n
So I’m genuinely curious:
Where does all of that dictionary data usually come from?<\/p>\nI’m especially interested in hearing from anyone who has actually built a dictionary app.<\/p>\n
What data sources did you use, and how did you handle licensing?<\/p>\n
Btw, I know Cambridge, Oxford, and other major dictionaries offer paid APIs or data licensing, but they’re quite expensive. I assume many smaller dictionary apps probably use their own databases or combine multiple data sources somehow.<\/p>\n
Any pointers would be greatly appreciated. Thanks!!!<\/p>\n<\/div>\n
submitted by \/u\/SnooPears7307 <\/a>
[link]<\/a><\/span> [comments]<\/a><\/span><\/p>
