I’m the founder of StockFit API – SEC sourced clean fundamentals for all US Companies (delisted or not) – all Point-In-Time data perfect for back testing.<\/p>\n
One of my subscribers has pushed me to investigate what it would take to widen the coverage to the pre-XBRL time.<\/p>\n
If you don’t know what that means:<\/strong> Before that time, there where 2 distinct data format time frames that you will have to prepare for if you want that data too: FDS is kinda ok to parse, but comes with many exceptions across filers. You can easily get to a coverage of 90%+<\/p>\n The nightmare starts in 2001. There really is no structural mandate whatsoever that you can rely on! Everything has to be tested empirically and your parser needs to be able to handle everything, validate as much as it can, and provide an observation layer for you to ensure data integrity.<\/p>\n This is probably the most challenging piece in my entire stack to get right. But in the end, I would be able to claim that I serve 30+ years of historical fundamentals, and that is absolutely worth it.<\/p>\n This effort is NOT finished yet. Until now, I’m serving 20+ years of fundamentals that are of very high quality\/accuracy. Getting that next batch to that same bar is something I’m trying to get to now.<\/p>\n But I’ll absolutely take one step at a time to get there. Otherwise this will get ugly very quickly. And there is nothing worse than serving wrong data.<\/p>\n I’m curious if anyone has done this pre-XBRL parsing before? Any lessons you can share after having gone through that? <\/p>\n And from the consumer side: How much do you care in your use case about fundamentals that are pre-2009? <\/p>\n<\/div>\n submitted by \/u\/Either_Door_5500 <\/a>
In 2009, XBRL was mandated my the SEC, which from that time on provides a structured data format to extract the actual fundamentals a company reported.<\/p>\n
1. Pre 2001: FDS (Financial Data Schedule)<\/strong> – a semi-structured way data was reported, similar to XML
2. 2001 – 2009: No structure at all<\/strong> – a full mix of HTML\/ASCII text\/tables <\/p>\n
[link]<\/a><\/span> [comments]<\/a><\/span><\/p>
