Where Can I Find APIs (or Legal Ways To Scrape) All Physics Research Papers, Recent And Historical?

I’m working on a personal tool that needs access to a large dataset of research papers, preferably focused on physics (but ideally spanning all fields eventually).

I’m looking for any APIs (official or public) that provide access to:

• Recent and old research papers
• Metadata (title, authors,, etc.)
• PDFs if possible

Are there any known APIs or sources I can legally use?

I’m also open to scraping, but want to know what the legal implications are, especially if I just want this data for personal research.

Any advice appreciated 🙂 especially from academics or data engineers who’ve built something similar!