I\u2019m working on a data integration problem in the railway\/infrastructure domain and would really appreciate some input from people with experience in data engineering or system design.<\/p>\n
We are integrating data from multiple railway companies. The challenge is that they often describe the same physical asset differently.<\/p>\n
Both refer to essentially the same real-world object (track), but:<\/p>\n
– naming differs<\/p>\n
– structure and attributes may differ<\/p>\n
– IDs are not shared across systems<\/p>\n
What we want to achieve:<\/p>\n
– Automatically detect that these refer to the same type of object<\/p>\n
– Map them to a unified model (something like an ontology layer)<\/p>\n
– Ideally also match actual instances across systems (entity resolution)<\/p>\n
What is the best-practice architecture for this kind of problem?<\/p>\n
How much can realistically be automated vs. manually mapped?<\/p>\n
Thanks a lot!<\/p>\n<\/div>\n
submitted by \/u\/theophil93 <\/a>
[link]<\/a><\/span> [comments]<\/a><\/span><\/p>
