Every dataset repo has its own README style – some list sources, others list fields, almost none explain the extraction process. I’m thinking scraped data deserves its own metadata standard: crawl date, frequency, robots.txt compliance, schema history, coverage ratio. But no one seems to agree on how deep to go. How would you design a reproducible, lightweight standard for scraped data documentation something between bare minimum CSV and academic paper appendix?
submitted by /u/Vivid_Stock5288
[link] [comments]
