I’m working on a research project and need to build a dataset from recent social media data — mainly text like posts, comments, captions, etc.
I’ve been checking out different platforms and this is what I’ve found so far:
- YouTube → comments are available through the official API
- Reddit → seems possible, but I’m a bit confused about the current access/API situation
- X/Twitter → official access looks paid
- Instagram → came across Apify and a few similar scraping services
- Facebook → same, there are some third-party options
I also checked Apify just to get an idea of the cost. Depending on the platform/actor, it seems to range from relatively cheap to pretty expensive. But I don’t have a fixed dataset size yet, so I’m not really committed to Apify or any particular platform.
Basically, I’m trying to figure out:
- What’s the easiest way to collect recent public data from these platforms?
- Are there any good free/low-cost alternatives?
- Is Apify actually worth using for something like this?
- Are there any APIs/tools/datasets I might be missing?
- If you’ve built a similar dataset recently, what approach did you use?
The data would only be from public sources, and this is for research/academic use.
Would really appreciate some pointers from people who have actually dealt with this recently.
submitted by /u/AIMill
[link] [comments]