i everyone!

I built and curated a small, high-density synthetic dataset focused on academic research methodology in Spanish to help address the lack of native, non-translated academic datasets for LLM fine-tuning and alignment.

Key Specs

• Repository: Januka2009/GPT5.6_SOL_INVESTIGACION on Hugging Face.
• Format & Size: 400 multi-turn conversations in JSONL/Parquet.
• Domain Balance: Stratified 90/10 train/test split across 4 core areas: Health, Social Sciences, STEM, and Economics.
• License: Creative Commons Attribution 4.0 (CC BY 4.0).
• Quality Check: Audited with an LLM-as-a-judge workflow (4.99/5 score) to strip conversational AI fluff and verify methodological accuracy.

It is ideal as seed data for LoRA/SFT experiments, academic tone alignment, or domain evaluation.

Any feedback, suggestions for new disciplines, or pull requests are greatly appreciated!

I recently digitized nearly two years of handwritten price records from a betel leaf market in MaLeTha Village, Sagaing Region, Myanmar.
The dataset contains around 1,800 daily records from April 2024 to March 2026, covering three betel leaf grades and prices in MMK/viss.
What makes it interesting is that I also added Myanmar-specific calendar features, including:
1. Myanmar Buddhist calendar dates
2. Seasons
3. Buddhist Sabbath (Uposatha) proximity
4. Major festival periods

Some interesting patterns appeared in the data, especially strong seasonal differences in prices.
The dataset is available on Hugging Face and can be loaded directly with the datasets library.
I also wrote about the process, the findings, and the limitations here:
Read the full write-up on Medium
Dataset: aungthuhein-dev/betel-leaf

I’d be interested to see what people could build with this — forecasting models, visualization, agricultural research, or something I haven’t thought of.

We’ve released an open dataset for multilingual PII/PHI extraction in medical text covering 22 Indic languages.

Dataset: https://huggingface.co/datasets/Sidharth1743/indicphi

The dataset currently contains approximately 25,000 synthetic medical records covering around 40 PII/PHI entity types.

The goal is to investigate how well multilingual models can extract sensitive entities from medical text in languages that have relatively limited NLP resources.

We’re currently working on the next stage: creating a human-annotated gold set that can be used to evaluate the synthetic data and trained models.

That’s actually where we’re looking for community input.

We’re trying to determine the best methodology for:

• selecting a representative subset of the 25K records
• balancing the 22 languages
• balancing different PII/PHI entity types
• human span-level annotation
• inter-annotator agreement
• adjudication
• measuring synthetic → human generalization

We’re not planning to have humans annotate all 25K records. The goal is to construct a smaller but carefully sampled human gold standard.

The associated model is here:

https://huggingface.co/sach3v/indicphi-muril-pii

I’d especially appreciate feedback from people who have built or evaluated multilingual datasets:

How would you sample the human gold set from the synthetic corpus?

And what would you consider necessary for making the resulting benchmark reproducible and useful to other researchers?

I kept getting this wrong on my own site, so i read every operator’s docs and wrote it down.

The thing that surprised me is that “block AI” isnt one decision anymore. openai runs gptbot for training, oai-searchbot for search indexing and chatgpt-user for when someone asks about your page directly. google, anthropic, amazon and apple all split theirs the same way. block the wrong one and you get the opposite of what you wanted, and nothing tells you.

So every entry says what happens if you disallow it, not just what it is. one is marked undocumented because bytedance publishes nothing and i didnt want to pretend otherwise.

json: https://utilix.app/crawlers.json
repo: https://github.com/utilixdev1/web-crawler-tokens

DISCLAIMER: utilix.app is my site. the data is cc0, use it however you want.

Hello guys,

My open source about robotics datasets reached 25 stars. It looks like low but its important for me.

I appreciate your stars.

https://github.com/Calibra-Robotics/Calibra

I recently worked on a Python project to collect and analyze 2026 FIFA World Cup data from FIFA, Sofascore, and Wikipedia.

There is no downloadable dataset yet. The repository contains the collection and analysis code, so the data can be generated by running the pipeline.

The project was built to handle three different source types:

• Wikipedia pages with server-rendered HTML.
• Sofascore data exposed through JSON endpoints.
• FIFA data accessed through browser automation and a session-authenticated API.

After extraction, the pipeline cleans and normalizes the responses for team- and player-level analysis using Python and Pandas.

The analysis currently explores metrics including:

• Shot-conversion rate.
• Expected goals.
• Passing accuracy.
• Pressing.
• Ball recovery.
• Team and player performance.

Repository:
https://github.com/ZenRows/scraping-world-cup-data

The accompanying ZenRows write-up explains the collection approach and analysis in more detail:

https://www.zenrows.com/blog/scrape-2026-fifa-world-cup-data-python

Data retrieval date: July 21, 2026.

Full disclosure: this project and the accompanying analysis are associated with ZenRows.

Disclosure: I built and maintain this dataset. It’s free, no signup, nothing to buy.

It’s an open species database, now available as free downloads. Covers plants, aquarium fish and reptiles/exotic pets.

What’s in it:

– 410k species with full taxonomy

– Common names in Chinese and English

– 2.7M trait records like light, temperature and care level

CSV or Darwin Core, CC-BY.

Heads up: the traits are AI-extracted. We’re bringing in a few PhDs in these fields to review and correct them, but that’s still in progress, so there will be mistakes for now. If you know a group well, I’d honestly love for you to poke holes in it.

What would you use something like this for?

Download page (my site): https://tropicals.cn/en/datasets

I am about to post a free audio dataset this weekend and I was curious if its the kind of thing people might be interested in. I can’t find anything like it currently available.

I took 750 male and 750 female audio samples from VCTK for each of their two mics and 3000 samples from the AMI headset microphones (gender inferred by pitch) for a total of 6k samples.

Those 6k samples were then run through 25 codecs commonly used in telecommunications and audio recording. Think Opus, MP3, etc. Permuteated a few options like DTX comfort noise off, adaptive and set. SILK disabled or enabled on Opus etc. Also ran 7 tandem encodings to mirror real channel transmissions. Works out to each single sample being available in 41 encodings for 1:1 comparison isolating the effects of the codecs themselves.

Same 6k samples also went through various audio processing conditions like reverb, echo, band pass filter, pitch shifting, autotune, time stretching, babble, and several kinds of additive noise. Including those permutations adds another 49 conditions.

Thinking about running MFA on the original samples to generate 6k textgrids as well.

All encodings come with source metadata, encoding parameters, function calls, libraries and versions used. Should be fully reproduceable.

Anyway curious about what people think or if I’m wasting my time uploading it all.

Hey all — I’m with a startup Harell Data which is a cloud platform to access to proprietary data sets for training PLM’s.

We’re early and want to make sure we’re solving a real problem before we go further, so we’re looking for a couple of researchers who actually train or fine-tune pLMs to try it hands-on and tell us straight.
In exchange we’ll set you up with complimentary trial access – we just want honest researcher feedback.

Current data partners:

• A-Alpha Bio — antibody-antigen binding affinity data, including the SEPIA datasets ·
• Adaptive Biotechnologies — TCR-pMHC / peptide-HLA binding data (TCR αβ + peptide-HLA sequences)

If that’s you, comment or DM me and we will set up a time to discuss. (we only have a couple of complementary licenses so will need to make sure it’s the correct research domain)

Need a legitimate and generalized dataset that contains details on general machinery and stats about it, example – general problems that arise regularly, machine failure at particular temperature and pressure valves etc, related to petroleum and refinery industries.

Disclosure: I built this and I run it. It is mine.

OrchestraFinder indexes US orchestras and their upcoming concerts in a normalized form: orchestra, concert, date, venue, works performed, composer, conductor, soloists. As of today that is 357 orchestras across all fifty states, 919 composers, 26,615 works and 4,623 upcoming concerts, and it grows daily.

The interesting part as data is that orchestral programming has never been aggregated nationally. Each orchestra publishes its season in its own format on its own site. Questions like ‘which living composers are actually being programmed, and where’ have not had an empirical answer.

Happy to answer questions about the ingestion, the normalization, update cadence, or anything else.

You can type stuff like:

cash – long_term_debt > 0

AVERAGE(fcf, 5) > 1b

revenue INCREASED IN 3 OF LAST 4 QUARTERS

gross_margin NEVER BELOW 40% OVER 5 YEARS

TOP(revenue, 20) AND net_income / equity > 20%

industry = “Pharmaceutical Preparations”

or whatever else you think of.

you can run these separately or all together, plain language ish or excel style formula

Runs across about 4,900 US filers. Annual, or quarterly. It’s at sequelseccom, free, no signup no ads no nothing. just launched so let me know if its got issues which im sure it does, i hope its helpful

Limitations, up front: undimensioned XBRL only, so no segment breakdowns (no iPhone revenue). No footnote text. No prices, so nothing price-derived like P/E. Foreign filers stay in their own currency, no conversions or anything

If you’re choosing data for a finance project , getting a CSV is only the first step. What does “close” mean? Which date does a dividend filter use? Is a fundamental ratio current or a snapshot?

One source worth evalauting is London Strategic Edge, which offers free access to market data with your own API key. Available data includes price candles, company fundamentals, dividends, splits, financial statements and economc series. Check coverage and history for your particular instruments before committing to it.

Disclosure: I maintain an unofficial, MIT-licensed adapter for this API. I’m not affiliated with the provider and take no payment.

Here is a compact data quality checklist from working on that adapter:

For a concrete starting exercise, I’d inspect IBM s daily prices and dividend events over a short historical window. Compare the close with a second source using the same session definition, inspect the dividend dates and check for splits before calculating returns.

If you already use an AI assistant with MCP tools ,this is a sample request you can adapt:

Inspect IBM daily candles, dividends, and splits for June 2026. Show the returned fields and dates, report any truncation, and explain the session and adjustment caveats. Flag anything the data does not establish.

My adapter, lse-data-mcp, lets an asssistant do that initial inspection. It runs locally, uses your own key and exposes 15 read-only tools. It cannot place trades and does not cache or persist market-data responses. It is still 0.x beta.

For the actual research dataset, use the provider ‘s SDK/API or exports directly, record extraction parameters and package versions and retain inputs where the licence permits. The MCP adapter returns bounded samples, with a maximum of 5000 rows per call and a separate response-size cap.

Free access does not mean open data: the adapter’s MIT licence covers its code, not the underlying data. The provider’s terms restrict redistribution, so check them before sharing a dataset or replication package.

I’ve been building The Economic Atlas as a way to make economic data easier to explore and compare.

The Compare tool lets you compare and rank economies across indicators such as GDP, inflation, unemployment, interest rates, government debt and trade, with plenty of historical data available as well.

The data is sourced from official institutions and the site is designed to be data-focused, without news or commentary.

I’m particularly interested in feedback from people who regularly use economic data, what would make a tool like this genuinely useful to you?

Disclosure: We created and published this dataset.

New Open-Access Benchmark: Hierarchical segmentation of vascular fibrosis in computational pathology.

The Core Challenge:
Segmenting the vascular wall is well-defined (baseline UNet DSC ≈ 0.80; inter-expert DSC ≈ 0.95). However, segmenting the intramural fibrotic microstructure remains an open problem. Due to its diffuse and heterogeneous nature, our baseline MONAI UNet achieves a DSC of only ~0.12 on fibrosis, while human inter-expert agreement averages ~0.61 DSC.

Dataset Specifications:
🔬 Scale: 705 high-resolution micrographs (1534×780 px, 0.252 μm/px), Picro-Mallory stain.
🏷️ Annotations: ROI + dual independent expert masks (vascular wall + fibrosis).
⚠️ Hierarchical Constraint: Fibrosis masks must be strictly spatially contained within the vascular wall.
🛡️ Robust Benchmarking: No color normalization applied; native aspect ratios preserved; strict animal-level 5-fold CV splits provided to prevent data leakage.

📄 Read the Data Descriptor: https://doi.org/10.1038/s41597-026-08214-y
🗂️ Access the Dataset: https://doi.org/10.6084/m9.figshare.31386748

A lot of AI discussion focuses on model size.

I think the bigger long-term moat may be owned specialist datasets.

A good domain dataset does more than add examples. It encodes:

• the edge cases
• the failure modes
• the validation rules
• the domain-specific structure
• the feedback loop that keeps improving it

That changes the economics.

Instead of paying frontier-model prices forever for every narrow task, you can use a smaller specialist model trained on data you actually own and understand — then reserve frontier models for the hard cases.

The interesting part is not “synthetic data” by itself. It is building a measured dataset-engineering process that can repeatedly find holes, fill them, verify the new data, and prove the capability gain on held-out evals.

Models will keep changing.

A high-quality proprietary domain dataset can compound for years.

huggingface.co/collections/CompilingThings/mql5-code-generation

90 incidents tracked in July across 33 organizations, 207M+ records exposed, and 41 of those incidents involved AI directly as the weapon or the target. A rogue commercial AI agent hit multiple enterprises in a single week and reused stolen credentials across four downstream services before anyone caught the identity switch.

None of that shows up to a traditional perimeter tool — the traffic looks like a signed, credentialed agent making legitimate API calls at machine speed. Firewalls and DLP were built to watch humans and static services, not autonomous callers that chain tools and pivot in seconds.

Curious how other teams are actually handling this right now: is anyone giving AI agents a distinct, revocable identity separate from the service accounts they inherit? Or is it still “the SOC catches it after the fact” for most orgs?

Disclosure: I run AerialDataset, the provider of this data.

We license real-world drone captures that retain overlapping photographs of the same environment alongside the available photogrammetric outputs. This is a commercial catalog, not an open-data release.

One inspection example is an Andel capture containing 61 original photographs, an orthomosaic, point-cloud files, a mesh, camera/capture data, and scene metadata.

The main distinction is that the photographs belong to the same physical scene and can be inspected alongside its reconstructed spatial products, rather than being unrelated aerial images.

Contents vary by scene. Please do not assume that every capture includes calibrated camera poses, annotations, independently validated accuracy, or every reconstruction output.

Catalog and scene previews:

https://aerialdataset.com/

Inspection package:

https://huggingface.co/datasets/avenian/aerial-drone-photogrammetry-samples

The inspection package requires a Hugging Face account and acceptance of evaluation terms. It is not licensed for model training, redistribution, or production use; those require a separate agreement.

Happy to answer questions here about the actual files, available metadata, and collection selection.

Centre for Net Zero’s Faraday dataset (OpenSynth) is excellent and effectively

unusable casually — it’s ~6GB of parquet with the load profiles stored as

delimited strings. So I aggregated it and published the result.

150 cohorts, each with a mean daily total, a 24-hour load shape, and deciles

where the cell was thick enough:

– baseline (no solar/battery/EV/heat pump)

– property type, EPC band, and the cross-tab

– month

– tariff type (standard / Economy 7 / smart / automated)

– heat pump: with vs without, unmatched, matched on property+EPC, by month,

and by tariff

– EV: with vs without, and by tariff

– LSOA k-means cluster

Cross-check, which is the reason to trust any of it: baseline comes out at

9.737 kWh/day and a median of 8.31. SERL Statistics Report 1 — separate

source, ~13,000 real metered GB homes — publishes a mean of 9.8 and a median

of 8.2. Two moments of the distribution, two unrelated datasets, ~1% apart.

Limitations, up front:

– Synthetic. Faraday is a generative model trained on ~1bn smart meter

readings from Octopus customers, who over-index on smart tariffs and LCT.

– No household identifier, so the deciles are over household-DAYS, not

households. That spread is wider than the between-home spread — treat it

as an upper bound. It’s recorded in `distributionOver` in the JSON.

– EPC band moves consumption by <0.1% within a property type, including for

heat pump households where insulation should dominate. I read that as the

model being under-conditioned on EPC rather than a finding about housing,

and I’ve built nothing on those cuts. Published anyway — a null result is

still a result.

– `cluster_label` is a k-means grouping of LSOAs on socio-demographic

features, NOT geography. Published as clusters, never as regions. Each

cluster’s country mix (from LSOA code prefixes) is included so you can join

your own geography.

– Cohorts under 500 profiles are dropped rather than published thin.

CSV and JSON, CDLA-Permissive-2.0 (same as the source), no registration.

Derivation script is in the repo and reproduces the file exactly.

https://www.energycosting.co.uk/data/uk-domestic-electricity

All credit to Centre for Net Zero for Faraday — I’ve only aggregated it.

hey folks, im building this with frens: Mostly Right

Describe a dataset, agents find sources and wire up ingestion, cleaning, and joins. You get parquet + an api, with scheduled refreshes.

Happy for any feedback!

I’ve just released SFHQ-VirtualID, a synthetic face dataset family built for identity-level machine unlearning. Everything is generated, so there are no real faces in it, and both releases have DOIs.

The problem I kept running into: in most face datasets used for unlearning, a person’s images are spread across splits, so “forgot the person” and “forgot some images” end up confounded. Here, each identity_id maps to exactly one split and contributes both train and holdout images (675 retain / 75 forget, 15-step protocol, MUFAC-aligned holdouts).

What ships:

– Bench: 67,500 balanced + 36,064 imbalanced 224×224 aligned crops. Uniform and seeded-Poisson forget schedules, plus a 5:1 long-tail popularity gradient for long-tail forgetting tests.

– Raw: 75,000 1024² portraits (100 per identity) with per-candidate prompt metadata for pose, expression, lighting, setting and camera.

I did not filter candidates on ArcFace identity similarity. The 0.40/0.45 thresholds are config defaults that ship as recorded columns (arcface_similarity, laplacian_variance, detection_confidence) rather than enforced filters, because silently dropping borderline-similar candidates hides exactly the confound that could explain a model’s apparent unlearning. Only the quality gate is enforced (detection plus Laplacian sharpness ≥ 80), and the 123 rejected candidates are documented in the manifest.

Reproducibility: 15-shard Slurm array on UoL’s Aire HPC (~120 GPU-hours), InstantID + Juggernaut-XL-v9 + ControlNet, pinned environment (torch 2.6/cu124, diffusers 0.39.0, insightface 1.0.1, pinned antelopev2 revision), RELEASE_MANIFEST.json and SHA-256 checksums.

Links:

Code · Hugging Face repos [Raw] [Bench] · Project Overview · Bench DOI 10.5281/zenodo.21877893 · Raw DOI 10.5281/zenodo.21879130

Caveats: the dataset is synthetic, so it’s a proxy for real-face benchmarks; demographic balance is inherited from the seed selection; and the two releases are related (Bench crops derive from the Raw candidates), so they aren’t independent test sets.

Happy to answer questions, and I’d genuinely like the split design stress-tested, since that’s the part I’d most want criticised.