Centre for Net Zero’s Faraday dataset (OpenSynth) is excellent and effectively
unusable casually — it’s ~6GB of parquet with the load profiles stored as
delimited strings. So I aggregated it and published the result.
150 cohorts, each with a mean daily total, a 24-hour load shape, and deciles
where the cell was thick enough:
– baseline (no solar/battery/EV/heat pump)
– property type, EPC band, and the cross-tab
– month
– tariff type (standard / Economy 7 / smart / automated)
– heat pump: with vs without, unmatched, matched on property+EPC, by month,
and by tariff
– EV: with vs without, and by tariff
– LSOA k-means cluster
Cross-check, which is the reason to trust any of it: baseline comes out at
9.737 kWh/day and a median of 8.31. SERL Statistics Report 1 — separate
source, ~13,000 real metered GB homes — publishes a mean of 9.8 and a median
of 8.2. Two moments of the distribution, two unrelated datasets, ~1% apart.
Limitations, up front:
– Synthetic. Faraday is a generative model trained on ~1bn smart meter
readings from Octopus customers, who over-index on smart tariffs and LCT.
– No household identifier, so the deciles are over household-DAYS, not
households. That spread is wider than the between-home spread — treat it
as an upper bound. It’s recorded in `distributionOver` in the JSON.
– EPC band moves consumption by <0.1% within a property type, including for
heat pump households where insulation should dominate. I read that as the
model being under-conditioned on EPC rather than a finding about housing,
and I’ve built nothing on those cuts. Published anyway — a null result is
still a result.
– `cluster_label` is a k-means grouping of LSOAs on socio-demographic
features, NOT geography. Published as clusters, never as regions. Each
cluster’s country mix (from LSOA code prefixes) is included so you can join
your own geography.
– Cohorts under 500 profiles are dropped rather than published thin.
CSV and JSON, CDLA-Permissive-2.0 (same as the source), no registration.
Derivation script is in the repo and reproduces the file exactly.
https://www.energycosting.co.uk/data/uk-domestic-electricity
All credit to Centre for Net Zero for Faraday — I’ve only aggregated it.
submitted by /u/aussiesteveau
[link] [comments]