# r\/datasets<\/a> – ScrapeGraphAI 100k Post<\/p>\n Announcing ScrapeGraphAI 100k<\/strong> – a dataset of 100,000 real-world structured extraction examples from the open-source ScrapeGraphAI library:<\/p>\n https:\/\/huggingface.co\/datasets\/scrapegraphai\/scrapegraphai-100k<\/a><\/p>\n What’s Inside:<\/strong><\/p>\n This is raw production data – not synthetic, not toy problems. Derived from 9 million PostHog events collected from real users of ScrapeGraphAI during Q2-Q3 2025.<\/p>\n Every example includes:<\/p>\n – `prompt`: Actual user instructions sent to the LLM<\/p>\n – `schema`: JSON schema defining expected output structure<\/p>\n – `response`: What the LLM actually returned<\/p>\n – `content`: Source web content (markdown)<\/p>\n – `llm_model`: Which model was used (89% gpt-4o-mini)<\/p>\n – `source`: Source URL<\/p>\n – `execution_time`: Real timing data<\/p>\n – `response_is_valid`: Ground truth validation (avg 93% valid)<\/p>\n Schema Complexity Metrics:<\/strong><\/p>\n – `schema_depth`: Nesting levels (typically 2-4, max ~7)<\/p>\n – `schema_keys`: Number of fields (typically 5-15, max 40+)<\/p>\n – `schema_elements`: Total structural pieces<\/p>\n – `schema_cyclomatic_complexity`: Branching complexity from `oneOf`, `anyOf`, etc.<\/p>\n – `schema_complexity_score`: Weighted aggregate difficulty metric<\/p>\n All metrics based on [SLOT: Structuring the Output of LLMs](https:\/\/arxiv.org\/abs\/2505.04016v1<\/a>)<\/p>\n Data Quality<\/strong>:<\/p>\n – Heavily balanced<\/strong>: Cleaned from 9M raw events to 100k diverse examples<\/p>\n – Real-world distribution<\/strong>: Includes simple extractions and gnarly complex schemas<\/p>\n – Validation annotations<\/strong>: `response_is_valid` field tells you when LLMs fail<\/p>\n – Complexity correlation<\/strong>: More complex schemas = lower validation rates (thresholds identified)<\/p>\n Key Findings<\/strong>:<\/p>\n – 93% average validation rate across all schemas<\/p>\n – Complex schemas cause noticeable degradation (non-linear drop-off)<\/p>\n – Response size heavily correlates with execution time<\/p>\n – 90% of schemas have <20 keys and depth <5<\/p>\n – Top 10% contain the truly difficult extraction tasks<\/p>\n Use Cases:<\/strong><\/p>\n – Fine-tuning models for structured data extraction<\/p>\n – Analyzing LLM failure patterns on complex schemas<\/p>\n – Understanding real-world schema complexity distribution<\/p>\n – Benchmarking extraction accuracy and speed<\/p>\n – Training models that handle edge cases better<\/p>\n – Studying correlation between schema complexity and output validity<\/p>\n The Real Story:<\/strong><\/p>\n This dataset reflects actual open-source usage patterns – not pre-filtered or curated. You see the mess:<\/p>\n – Schema duplication (some schemas used millions of times)<\/p>\n – Diverse complexity levels (from simple price extraction to full articles)<\/p>\n – Real failure cases (7% of responses don’t match their schemas)<\/p>\n – Validation is syntactic only (semantically wrong but valid JSON passes)<\/p>\n Load It:<\/strong><\/p>\n This is the kind of dataset that’s actually useful for ML work – messy, real, and representative of actual problems people solve.<\/p>\n<\/div>\n submitted by \/u\/Electrical-Signal858 <\/a> from datasets import load_dataset dataset = load_dataset(\"scrapegraphai\/sgai-100k\") <\/code><\/pre>\n
[link]<\/a><\/span> [comments]<\/a><\/span><\/p>
