Sharing a dataset I’ve been building: daily LLM inference outputs on stock market forecasting, captured before outcomes were known, so predictions can’t be reconstructed with hindsight.<\/p>\n
What’s in it: 90+ days of runs (Feb 17 \u2013 May 19, 2026, ongoing) for Gemini 2.5 Flash with Google Search grounding, temperature 0.2 Multi-model coverage: 2.5 Pro, 2.5 Flash Lite, and 3 Flash Preview also included Per-run: 10-trading-day price lookahead, sentiment, confidence score, full reasoning trace, cited search snippets ~3,655 rows total, 211MB, fully documented schema with a Colab quickstart notebook for hydrating ground truth yourself<\/p>\n
Why it might be useful: most LLM benchmark datasets test on static, already-resolved questions. This one is structured so ground truth genuinely didn’t exist at generation time \u2014 useful for studying calibration (ECE), hallucination patterns, and confidence-vs-accuracy relationships under real uncertainty instead of retrospective fitting.<\/p>\n
Note on compliance: realized prices and news text aren’t redistributed (licensing reasons) \u2014 there’s a hydration script to populate those fields yourself with your own data source, or you can just inspect pre-computed outcome comparisons and results on the companion site (glassballai.com\/results).<\/p>\n
Note Evaluation: Some tickers have very low run counts due to interrupted tracking or individual tracking runs that are not part of the fixed set of tracked stocks. They are included for full transparency and factor into the global metrics, but their individual ticker-level stats should be ignored due to high variance.<\/p>\n
Published on Hugging Face under CC-BY-NC-4.0: huggingface.co\/datasets\/louidev\/glassballai<\/p>\n
Happy to answer questions about the collection methodology or the metrics computed on top of it.<\/p>\n<\/div>\n
submitted by \/u\/aufgeblobt <\/a>
[link]<\/a><\/span> [comments]<\/a><\/span><\/p>
