MIT technology review magazine <\/a> data from January 1997 to October 2024. I started scrapping from 1890 but looks like posts from years < 1997 aren’t posted so I’ve excluded them from the dataset (I’ve metadata about these issues though, which includes the cover image, title and link to the pdf file for that issue).<\/p>\n Format:<\/strong><\/p>\n { title: “Issue Title”, date: “2024 January”, hero: “cover image url”, pdfLink: “link to pdf file”, posts: [{ title: “Post Title”, date: “Article publishing date”, topic: “Policy”, headerImg: “image url for article hero img”, authors: [{ name: “Author name”, link: “Link to author profile”, }], body: “<p>Article content goes here<\/p>”, }] } <\/p>\n All files are stored in folders named by year.<\/p>\n Useage<\/strong>: I actually scrapped this data for myself to generate epub and pdf files with less clutter and better readability on mobile\/kindle devices. I’m currently scrapping all the popular magazines like economist, newyorker, atlantic, vanity fair etc without a solid usecase other then generating epubs\/pdfs. You can generate epubs\/html or combine it with other data to use in some LLM projects.<\/p>\n Download link: Google Drive<\/a><\/p>\n<\/div>\n submitted by \/u\/waqarHocain <\/a>
[link]<\/a><\/span> [comments]<\/a><\/span><\/p>
