I use “Heretic” library on models to liberate them from their safeguards, but while checking their “uncensoredness”, I found they can hallucinate a lot. You know, it’s basically like a child who’s now allowed to use the F word once and he says “Fred” instead of the actual thing.

So I think if the models train on valid uncensored data (specially if they start Grokking) the results can improve. So I am using for these types of datasets to test my theory.

I’ve been working on a dataset for a research project on how China is portrayed in the media. It currently contains just over 10,000 news articles from both Chinese and Western news outlets.

Each article is classified by topic and by the way China is portrayed (e.g. positive, negative, threat, Xi-centered, neutral, etc.). The dataset was originally created for academic research, but I’m now wondering whether it could also have commercial value.

I’m not trying to sell it here, just looking for advice. Has anyone here ever licensed or sold a specialized dataset like this? Who would actually be interested in buying it? AI companies, media intelligence firms, universities, think tanks…? Or are datasets like this generally expected to be open source?

I’d really appreciate hearing from anyone who has experience commercializing niche datasets or knows how this market works.

Hey builders! 🤖

I’ve launched a project called x402 Data Hub (https://x402datahub.io). It is designed to solve a major problem for autonomous AI agents: accessing premium real-time data without expensive monthly subscriptions.

We offer 54 structured JSON datasets across:

– SaaS & AI API Pricing (LLM cost comparison, cloud GPU indexes)

– Live multi-chain Gas metrics (Base/Arbitrum)

– Tech jobs & developer market rates

– Expat living data (rent index, digital nomad visas, tax rules)

Zero API keys or email registrations required. Access feeds instantly using HTTP 402 pay-per-request protocol ($0.01 USDC on Base or Arbitrum).

🎁 We offer 50 free daily requests per dataset for developer testing.

Read the agent-friendly specification: https://x402datahub.io/llms.txt

Let me know what datasets you would like to see added next!

Hi all,

I’ve been working on a financial research project that involves analyzing company filings and disclosures for A-shares (Shanghai/Shenzhen) and HKEx listed entities.

Coming from a Western market background, the biggest pain points I’ve noticed are the language barrier, disparate filing locations, and the lack of structured APIs formatted for LLMs/RAG.

For those who cover APEx or emerging markets:

1. What tools or data providers are you currently using for CN/HK stock filings?
2. How do you handle language translation and footnote extraction in your pipeline?

Would love to exchange ideas with anyone working on similar Asia-Pacific equity pipelines!

Quick background: public companies have to file everything with the SEC. Who’s buying their own stock, what hedge funds hold, who just crossed 5% with an activist stake. It’s all public and free on sec.gov. It’s also served as XML that makes you want to quit programming, and every filer formats it differently.

Edgrapi does that parsing once and hands it back as clean JSON. One call, one URL. That’s the product.

Month one taught me things I didn’t expect.

The data lies if you read it literally. Michael Burry’s latest fund filing is 66% Palantir and Nvidia. Except they’re puts, bearish bets. A naive parser reports him as massively long two stocks he’s actually betting against. Half the fund-tracker tools out there get this wrong, and getting it right turned out to be most of the value.

Postgres quietly broke every login for a week. Worked perfectly on my laptop, failed in production. A floating-point column was rounding my 10-digit timestamps, so every login token was born already expired by about three hours. SQLite stored it fine, Postgres didn’t. I only caught it because I logged the raw stored value instead of the error message.

Every API call took 2.6 seconds, no matter what. Cheap call, heavy call, same 2.6s. That flatness was the tell: it wasn’t the work, it was nine database round-trips to a server in another region. Collapsed it to two. It’s 0.7s now.

We are building an edge-AI audio noise-reduction system on an ESP32-S3.

Our architecture uses a lightweight GRUNet (~59k parameters) to output a dynamic gain mask on a 44-band Mel-spectrogram.

​I need gigabytes of audio to train the model. Does anyone have recommendations for the best open-source datasets for:

1> ​Clean, isolated human speech.

2> ​Diverse ambient background noise (traffic, crowds, machinery, etc.).

​Also, any tips or open-source scripts for artificially mixing these at different Signal-to-Noise Ratios (SNRs) before generating the 16kHz Mel-spectrograms would be hugely appreciated!

Disclosure: I built this, it’s my project, and paid tiers exist. There’s a free tier and everything shown below is viewable without signing up.

What it is: structured sponsorship records extracted from public podcast RSS show notes. One row per (brand, episode):

brand (canonically resolved) | show | episode | publish date | promo code | promo URL + registrable domain | sponsor type (paid / affiliate / house ad) | confidence | confidence tier | first seen | last seen | the verbatim sentence the claim came from

Sample rows straight out of the DB:

– AG1 on Huberman Lab, 2026-07-27, evidence: “AG1: https://drinkag1.com/huberman“

– Visible on Good Hang with Amy Poehler, code HANG, 2 episodes, 14-day span

– Saily on Machtwechsel (German news podcast), code “Machtwechsel”, 3 episodes over 18 days

Method, since this sub cares about it: LLM extraction over the show-notes text, then a deterministic brand-resolution layer on top. Domain evidence merges entities first (drinkag1.com and athleticgreens.com collapse into one AG1 entity), exact normalized-name match second, and anything that is merely name-similar goes to an adjudication queue and is never auto-merged. That last rule is what keeps Dove the soap separate from Dove the chocolate. Every record retains its source sentence so any claim can be audited by hand.

Honest limits, up front:

– Show notes only. Ads that exist purely in audio and never appear in the notes are invisible to this. Transcript coverage is not built yet.

– The corpus is small right now: 314 episodes across 93 shows, US + DE + FR. It grows daily but this is not a historical archive.

– I am deliberately not publishing an accuracy percentage. I ran a held-out evaluation, then used its failures to fix the extractor, which burns that holdout. Any number I quoted today would be inflated. A fresh untouched holdout is the next task. Until then every record carries a confidence tier and only the CONFIRMED tier is presented as fact.

– No spend or impression estimates. This answers who advertises where, not how much they paid.

Free tier is 200 requests/month, paid is $49/$199/$499. Keys are not self-serve yet, so the page is an early-access list rather than a checkout.

Two things I would actually like this sub’s read on: is a per-record evidence string useful to you, or is it dead weight next to a confidence score? And what would you want joined onto this that is missing (show category, audience estimates, historical backfill)?

https://podintel.github.io/?src=datasets

Disclosure: I’m the developer and founder of StaticCreation. This is my own product.

Hey. Been working on this for over a year and just went live.

What it is: StaticCreation is an API marketplace for UK public sector data. Instead of scraping Companies House, DVSA, Land Registry, EPC, and a dozen other sources separately, you get one API key that works across everything.

The problem I kept hitting: Every time I built something that needed UK data, I’d spend weeks writing scrapers, handling rate limits, parsing inconsistent formats, and maintaining pipelines that break every time a government site changes. I figured other devs were doing the same thing.

What’s in there:

1.2 billion records across 16 datasets

Property: EPC certificates, Land Registry transactions, planning applications

Vehicles: DVLA data, full MOT history (832m test records), MIB insurance checks

Business: Companies House profiles, officers, charges, filings

Plus energy data, school/Ofsted ratings, crime stats, food data, fuel prices, trademarks, and more

How it works:

Buy tokens once (starts at £1.50 for 100), spend them per call. No subscriptions, no separate plans per dataset. One balance works across every endpoint.

Combined endpoints are what I’m most proud of, instead of calling 5 APIs yourself and joining the data, one call to Property Intelligence returns EPC + crime + schools + energy + price history for any UK address. Same idea for vehicles, companies, etc.

Stack: FastAPI backend, PostgreSQL, self-hosted on dedicated hardware in the UK. Everything runs through my own ETL pipelines, no reselling third-party APIs.

What I’d love feedback on:

Is the pricing clear enough?

Are there UK datasets you’d want that I’m missing?

Would you actually use something like this?

Site: https://staticcreation.co.uk

Happy to answer any questions about the build, the data partnerships, or the tech.

QUESTION.

Is there a good way to watermark data assets before sharing with potential buyers?

We regularly share data samples with customers for evaluation, with clear licence terms on usage scope. But I worry those terms are practically unenforceable. Someone could generate synthetic data from a sample even though the licence restricts use to evaluation only.

Has anyone found effective ways to tag or watermark files before sharing? Metadata tagging is one option, but are there any deeper level solutions (steganographic watermarking, fingerprinting, etc)?

To keep it simple, let’s say we only talking about CSV files.
But this applies to video, audio, PDF, and archives too if you got any experience.

Disclosure: this is my own project — I built and maintain it.

Source (the data itself): https://github.com/markselby9/readtheplaces.com — one directory per book under /books, each containing book.json, waypoints.json and source.txt.

Browsable version: https://readtheplaces.com

Scale: 392 novels, 2,105 places, 298 cities.

Per-place schema (waypoints.json), one real record, abridged:

{ "id": "westminster-doorstep", "name": "Clarissa's house, Westminster", "progressLabel": "10:00", "character": "clarissa", "coords": [-0.1275, 51.4993], "placeCertainty": "inferred", "certaintyNote": "Woolf never gives an address. The Dalloways live in Westminster within earshot of Big Ben; scholars place the house around Dean's Yard. Sited here as a considered guess, not a fact.", "quoteAnchor": "Mrs. Dalloway said she would buy the flowers herself.", "passage": "...", "sources": [...] } 

The field worth arguing about is placeCertainty. Geocoding fiction is mostly a disambiguation problem: many places are described rather than named (the abbey in The Name of the Rose is a northern Italian abbey Eco never names), and the named ones collide constantly. So each record carries what the resolution was based on, and inferred sitings say so in plain English instead of sitting on the map looking like facts. Filter to placeCertainty != "inferred" and you get a much smaller, much harder subset.

Waypoints are ordered by narrative progression rather than geography, so it’s usable for route/sequence work as well as point work.

How it was built, honestly: candidate mentions are extracted from the text by an LLM pass, then resolved against gazetteer data and checked by hand. Recall on minor mentions is therefore better than precision, and coverage skews heavily to 19th–20th century English-language fiction. Treat it as a curated dataset with a machine-assisted first pass, not a gold standard. It is not synthetic — every record points at a real passage in a real book.

Licence is CC BY-SA 4.0. Corrections are PRs against the JSON files, or there’s an issue template if you’d rather just report one.

Hi everyone. I’ve been building a historical research dataset for low-float/small-cap stocks and I’m running into a much bigger data-coverage problem than I expected.

The general universe I’m looking at is:

• Stocks with fewer than roughly 20M shares in float.
• Lower-market-cap names.
• At least 1M shares of daily volume.
• Dates from 2023 through 2025.
• Premarket, regular-session, and ideally postmarket one-minute data.
• Delisted, renamed, merged, and inactive tickers included.

So far, I’ve been able to build a fairly large list of eligible ticker-days and recover point-in-time information for a lot of old and delisted names. I’ve tested data from Alpaca SIP and Massive/Polygon, but the historical minute data is much more incomplete than I expected.

The issue is not just whether a ticker exists in the database. I need enough complete data to reconstruct:

• Premarket high and volume.
• Regular-session volume and VWAP.
• Intraday highs and lows.
• Technical setups around PMH breaks and pullbacks.
• Whether a theoretical entry reached a stop or 2R target first.
• Missing trading minutes versus minutes where no trades actually occurred.

In my broad initial universe, only around 1,600 out of roughly 293,000 eligible ticker-days had completely stored paths under the original strict definition. I’ve improved coverage using multiple providers, but there are still thousands of older, renamed, or archive-risk ticker-days where the data is partial or completely missing.

I’ve also found that mixing feeds can create problems. For example, bars from a direct-feed composite may look valid on their own but have very different volume from consolidated SIP data because off-exchange and FINRA TRF prints may be missing. Since my study uses volume, VWAP, liquidity, and dollar-volume features, I can’t just patch missing days with a different feed unless the data is reasonably comparable.

Has anyone built something similar and found a provider that offers:

• Historical one-minute bars going back several years.
• Full premarket and postmarket coverage.
• Raw/unadjusted price and volume.
• Consolidated SIP or TAQ-derived data, including FINRA/TRF volume.
• Delisted and inactive securities.
• Historical ticker changes and point-in-time symbol mapping.
• A flat-file or bulk-download option so I can save the data once and reuse it.
• Coverage where an empty response can be distinguished from a genuinely inactive/no-trade day.

I’m open to a paid provider or one-time historical download. I’ve looked at Alpaca, Massive/Polygon, Databento, and IBKR so far. Databento’s direct feeds are interesting, but the feeds visible to my account were missing several exchanges/reporting facilities, so the volume was not comparable to SIP. IBKR also appears to filter some historical trades and may not retain data for securities that are no longer trading.

I’d especially appreciate feedback from anyone who has used AlgoSeek, Tick Data, NYSE TAQ, LSEG Tick History, or another provider for old low-float and delisted stocks.

I’m not looking for real-time alerts or a trading platform. I’m trying to build one clean historical research dataset that I can download, validate, and keep locally.

Anyone looking for US egocentric data?

We have 375 data collectors.

Hello everyone,

I am a final-year undergraduate student in Computer Science and Engineering (CSE) at Daffodil International University, Bangladesh.

I am currently working on my undergraduate thesis titled:

“Deepfake Image Detection Using Spatial-Frequency Feature Fusion and Explainable Deep Learning.”

I am looking for publicly available human face image datasets for academic research purposes only.

If possible, I would appreciate datasets containing different face poses (front, left-profile, and right-profile), although frontal face datasets are also perfectly acceptable.

I am specifically looking for publicly available datasets or images that can be legally used for academic research.

Specifically, I need three categories of images:

📁 1. AI-generated Human Faces

• GAN-generated faces
• Diffusion-generated faces
• Other synthetic human faces

📁 2. Deepfake Human Faces

• Face swap
• Face manipulation
• Deepfake images extracted from public datasets

📁 3. Real Human Faces

Natural human face photographs

Different ages, genders, and lighting conditions

Front, left-profile, and right-profile faces

If anyone wishes to share publicly distributable images or datasets, I have also created a shared Google Drive folder for convenience.

I am not requesting copyrighted or private images. I am only looking for publicly available datasets or resources that are legally shareable for academic research.

Google Drive:
https://drive.google.com/drive/folders/1U_XL41UdusIKukb3TAREZOeeu_uSwM3b?usp=drive_link

If you know any public datasets, GitHub repositories, Hugging Face datasets, or other reliable resources, I would greatly appreciate your recommendations.

If you already have a suitable dataset, you are also welcome to upload publicly shareable images directly to the appropriate folder in the shared drive.

The collected data will be used strictly for academic research and educational purposes.

I would be happy to acknowledge contributors in my thesis if their publicly shareable dataset or resource significantly supports this research.

If you have worked on deepfake detection or know of any useful public datasets, I would greatly appreciate your suggestions.

Thank you very much for your time and support!

Three datasets aggregated from public FAA releases (the raw ones ship as an MS Access export and awkward portal dumps) into analysis-ready CSVs with per-column documentation:

Wildlife strikes on civil aircraft, 1990–2026 — 347,575 reports: by year, airport (452, ICAO-coded), and species. 2025 set the all-time record (24,458 reports). Fun divergence: the species planes hit most (doves, swallows) almost never damage them (~1.5%), while deer damage the aircraft in ~82% of reported strikes. https://www.kaggle.com/datasets/himaxym/faa-wildlife-strikes-us

Laser strikes on aircraft, 2021–2025 — 54,722 reports with 243 crew injuries, by year, state, and reporting ATC facility (caveat documented: the “city” is the ATC facility’s location, not where the laser was fired). https://www.kaggle.com/datasets/himaxym/faa-laser-strikes-us

Drone (UAS) sightings reported by pilots, 2019–2026 — 12,566 reports by year, state, and city. NYC is #1 (584). https://www.kaggle.com/datasets/himaxym/faa-drone-sightings-us

Versioned copy with citable DOI (Zenodo, wildlife): https://doi.org/10.5281/zenodo.21347859

Original sources (US government work, public domain): – https://wildlife.faa.gov/ – https://www.faa.gov/about/initiatives/lasers – https://www.faa.gov/uas/resources/public_records/uas_sightings_report

Disclosure: I compiled and maintain these aggregates (and an interactive explorer at himaxym.com/safety). The compilation is CC BY 4.0 — use it for anything, attribution appreciated.

Hello everyone,
We’re developing an AI-powered app, Faseeh AI, that detects pronunciation mistakes in Quranic recitation and gives users precise, real-time feedback.
Our current model was trained on hundreds of hours of professional recitations and high-quality, clean audio from well-known reciters. The model performs well on similar input, but struggles with real-world users: different accents, non-native speakers, beginners, children, women, and anyone who doesn’t sound like a professional reciter.
To fix this, we need to train on diverse, real-world recitation data not studio-quality professional audio, but recordings that reflect how actual learners sound.
Specifically, we’re looking for:
– Recitation datasets from non-professional or everyday users
– Diverse demographics: male/female, kids/adults, beginner/intermediate
– Multiple accents and mother tongues (Malay, Indonesian, Urdu, English, Turkish, etc.)
– Any publicly available or research-use datasets we may have missed
We’ve already explored academic sources, but still not enough.
If you know of any dataset, research project, university study, or community effort collecting this type of audio, we would genuinely appreciate the lead.
We’re also open to ethical data collection partnerships if any researchers or institutions are working in this space.
Happy to share more about the project if helpful.
Thank you very much in advance.

I have a real conversation dataset of founder getting feedback from random people on their idea.

Valu of this conversation:

– Brainstorming on Idea

– Real human conversation

– same person with different person paired.

Hey ya’ll. I’ve come across this dataset that detects user engagement and emotions (e.g. boredom, confusion) from facial expressions and have been thinking of using it for our research project. For those who have used it, how was your experience?

Source: https://people.iith.ac.in/vineethnb/resources/daisee/index.html

I need a dataset with movie title, budget, total box office collection data.

I am working on a project, this would be really helpfull

hi everyone, im building a python screener / trade tracker for my portfolio. currently im using xfinlink for US data with eodhd and yfinance as fallback but i need high quality korean fundamentals data, which none of them seem to provide (xfinlink is US-only; eodhd and yfinance yes but inaccurate in many instances).

anyone running python/screener pipelines on korean securities and can share a reputable & reliable data vendor? cost is not really a concern for me. quality is. FYI I’ll be buying as retail so would appreciate recommendations that offer non-institutional plans. cheers.

A project I’ve been working on for a while and I’m excited to finally share!

The NDE Archive is a database of over 6,700 documented near-death experiences from recognized sources. One of the main reasons I built it is that existing sites are often hard to search through and accounts are mostly plain text with little filtering. Here you can actually search and filter experiences in meaningful ways, for example by demographics like sexual orientation or ethnicity, which opens up some really interesting comparisons.

These stories were also individually analyzed with Sonnet to surface patterns and statistics that are not easily visible when reading individual accounts.

The project is non-profit and was built out of curiosity for the subject and nothing else. If you’d like to support it, sharing is hugely helpful, and donations are welcome through the website.

https://ndearchive.com/

Disclosure: I did not build the original dataset, which was obtained from other recognized sources. I did the data collation and presentation on the web app.

Disclosure: I build these datasets and run the site they’re published on, so this is a self-promotion post per rule 1.

I’ve been compiling comparable car-cost data across 36 countries – the kind of cross-country tables that don’t really exist anywhere else (most sources are single-country). Two newest releases:

– Cost of getting a driving licence in 36 countries, next to average salary, as a share of a month’s pay. It’s 112% of a month’s net pay in the Netherlands, 94% in Japan, but 8% in Mexico City (where there’s no practical test).

– Public DC fast-charging prices in 30 countries (pay-as-you-go, per kWh and per 100km). The UK is the most expensive in the world at about $1.06/kWh, roughly 6x India.

Earlier sets in the same family: 5-year total cost of ownership, first-year car tax, and home EV charging – all 36 countries.

Sources & method: national statistics offices, driving-school associations, official fee schedules, GlobalPetrolPrices and published operator tariffs – every row carries its own source URL and date. Licensed CC-BY, free to reuse with attribution. Each CSV is linked on its study page here: https://carsmultiverse.com/research/

Happy to answer questions about any country’s numbers, or add columns people want.

Please give me if someone has the dataset

Hi everyone,

Sharing a new open dataset for people working on education AI, curriculum modeling, or LLM training.

K12-KGraph is a curriculum-aligned knowledge graph built from publicly available K-12 textbook materials. The current release covers math, physics, chemistry, and biology, and includes structured links between concepts, skills, experiments, exercises, textbook sections, chapters, and books.

The main idea is simple: for education LLMs, adding more practice questions is useful, but it often only teaches the model how to answer questions. A curriculum knowledge graph can also teach the model how topics are connected, which concepts should come first, and what knowledge may be missing when a student gets stuck.

The released resources include:

• A curriculum knowledge graph
• A benchmark for testing curriculum understanding
• A prepared training dataset generated from the graph
• The paper and construction method, so the same approach can be adapted to other textbook systems where content rights are clear

In the experiments, the graph-based training data performed better than the same amount of regular instruction or exercise-style data on education benchmarks. The useful takeaway is that structure matters: modeling the curriculum itself can improve education LLMs more efficiently than only scaling question banks.

Links:

Paper + Dataset: https://huggingface.co/papers/2605.09635