Dear data community,<\/p>\n
I am a researcher currently looking for datasets and inspiration for my work. My research focuses on AI agents within organizations, and my goal is to develop a system where agents can oversee data pipelines, generate lineage, and propose improvements.<\/p>\n
Ideally, I am looking for datasets that are either raw or require cleaning, so they can better support data governance use cases (e.g., defining ER models, data quality rules, lineage, etc.). One idea I explored was using data from crypto exchanges, since they are freely available. However, these datasets are typically already well-structured, require minimal cleaning, and do not easily lend themselves to modeling complex governance scenarios (e.g., ER modeling, data ownership, data quality issues).<\/p>\n
Additionally, I would like to build a simple machine learning component on top of the data, mainly for completeness and demonstration purposes.<\/p>\n
That said, I am finding it quite challenging to identify \u201crealistic\u201d and sufficiently complex datasets that meet these criteria.<\/p>\n
I would greatly appreciate any suggestions or pointers to relevant data sources.<\/p>\n<\/div>\n
submitted by \/u\/Vegetable_Fishing <\/a>
[link]<\/a><\/span> [comments]<\/a><\/span><\/p>
