I’ve been working on a modernized Steam dataset that goes beyond the typical CSV dump approach. My third data science project, and my first serious one that I’ve published on Zenodo. I’m a systems engineer, so I take a bit of a different approach and have extensive documentation.<\/p>\n
Would love a star on the repo if you’re so inclined or get use from it! https:\/\/github.com\/vintagedon\/steam-dataset-2025<\/a><\/p>\n After collecting data on 263,890 applications from Steam’s official API (including games, DLC, software, and tools), I built a multi-modal database system designed for actual data science workflows. Both as an exercise, a way to ‘show my work’ and also to prep for my own paper on the dataset.<\/p>\n What makes this different:<\/p>\n Architecture-first approach: Instead of flat CSV files, this uses PostgreSQL 16 for normalized relational data, Neo4j for publisher\/developer relationship graphs, and pgvector for semantic search on game descriptions. The goal was to make it analytically-native from the start.<\/p>\n Comprehensive coverage: 263K applications compared to the 27K in the popular 2019 Kaggle dataset. Includes rich HTML descriptions with embedded media, international pricing, detailed metadata, and Steam’s full application catalog as of January 2025.<\/p>\n Semantic search ready: Game descriptions are vectorized using sentence-transformers, enabling queries like “find games similar to Baldur’s Gate 3” based on actual content similarity rather than just tags.<\/p>\n Use cases: – NLP projects using game descriptions (avg 270 words) – Price prediction models with international market data – Semantic search and recommendation systems – Time-series analysis of gaming trends<\/p>\n Data quality notes: – ~56% API success rate (Steam delists games, regional restrictions, content type diversity) – Conservative rate limiting (1.5s delays) for sustainable collection – All data from official Steam Web API only (no third-party scrapers) – Comprehensive error handling and retry logic<\/p>\n The dataset is fully documented with setup guides, analysis examples, and architectural decision rationale. Built using Python 3.12+, all collection and processing code included.<\/p>\n Repository: https:\/\/github.com\/vintagedon\/steam-dataset-2025<\/a><\/p>\n Zenodo Release: https:\/\/zenodo.org\/records\/17266923<\/a><\/p>\n Quick stats: – 263,890 total applications – ~150K successful detailed records – International pricing across 40+ currencies – 50+ metadata fields per game – Vector embeddings for 100K+ descriptions<\/p>\n This is an active project \u2013 still refining collection strategies and adding analytical examples. Open to feedback on what analysis would be most useful to include.<\/p>\n Technical stack: Python, PostgreSQL 16, Neo4j, pgvector, sentence-transformers, official Steam Web API<\/p>\n<\/div>\n submitted by \/u\/vintagedon <\/a>
[link]<\/a><\/span> [comments]<\/a><\/span><\/p>
