Looking For Advice On Scaling SEC Data App (10 Rps Limit)

I’ve built a financial app that pulls company financials from the SEC—nearly verbatim (a few tags can be missing)—covering the XBRL era (2009/2010 to present). I’m launching a site to show detailed quarterly and annual statements.

Constraint: The SEC allows ~10 requests/second per IP, so I’m worried I can only support a few hundred concurrent users if I fetch on demand.

Goal: Scale beyond that without blasting the SEC and without storing/downloading the entire corpus.

What’s the best approach to: • stay under ~10 rps to the SEC, • keep storage minimal, and • still serve fast, detailed statements to lots of users?

Any proven patterns (caching, precomputed aggregates, CDN, etc.) you’d recommend?

submitted by /u/Ok-Access5317
[link] [comments]

Leave a Reply

Your email address will not be published. Required fields are marked *