Best Way To Collect Recent Social Media Data For A Research Dataset?

I’m working on a research project and need to build a dataset from recent social media data — mainly text like posts, comments, captions, etc.

I’ve been checking out different platforms and this is what I’ve found so far:

  • YouTube → comments are available through the official API
  • Reddit → seems possible, but I’m a bit confused about the current access/API situation
  • X/Twitter → official access looks paid
  • Instagram → came across Apify and a few similar scraping services
  • Facebook → same, there are some third-party options

I also checked Apify just to get an idea of the cost. Depending on the platform/actor, it seems to range from relatively cheap to pretty expensive. But I don’t have a fixed dataset size yet, so I’m not really committed to Apify or any particular platform.

Basically, I’m trying to figure out:

  • What’s the easiest way to collect recent public data from these platforms?
  • Are there any good free/low-cost alternatives?
  • Is Apify actually worth using for something like this?
  • Are there any APIs/tools/datasets I might be missing?
  • If you’ve built a similar dataset recently, what approach did you use?

The data would only be from public sources, and this is for research/academic use.

Would really appreciate some pointers from people who have actually dealt with this recently.

submitted by /u/AIMill
[link] [comments]

Leave a Reply

Your email address will not be published. Required fields are marked *