I recently created my first public dataset focused on cryptocurrency sentiment analysis and Bitcoin market forecasting. The dataset contains around 20,000 Reddit posts collected from major crypto communities between 2017 and 2025 using the PRAW API.

It includes:

• Reddit post metadata
• Cleaned text features
• Crypto-enhanced VADER sentiment
• Custom FinBERT sentiment scores
• Bitcoin prices and returns
• Binary BTC movement labels for 1h, 6h, 12h, and 24h horizons

The dataset was built for financial NLP, sentiment analysis, and forecasting research. I am still learning dataset engineering and would appreciate feedback, suggestions, or ideas for improvement.

Hi r/datasets — disclosure first: I sell a paid version of these on Gumroad ($34, 83% off launch). I’m posting the free 20-record samples here because they’re genuinely useful on their own and the mod rules ask self-promotion to be labeled.

What’s in the free samples:

20 niche datasets, each with 20 fully-enriched records as CSV + JSON. ~55,000 records total in the paid version (54,958 as of today). Topics:

• ai-tools, ai-agents, ai-prompts, ai-models-pricing (13 paid Llama 3.3 70B providers compared)
• public-apis, mcp-servers (2,971), developer-tools, vscode-extensions
• self-hosted-software, open-source-alternatives, no-code-lowcode
• design-resources, cybersecurity-tools
• npm-packages (top by weekly downloads), homebrew-formulae
• huggingface-models (top 4,000 by downloads), huggingface-datasets (2,600+)
• vector-db / RAG ecosystem, ai-agent-frameworks (1,324 records — grew 6.6x in 8 days)

Why I built them:

I kept needing structured, queryable lists of “all the X tools” for filterable directory builds. Awesome-lists and READMEs are great for browsing but useless for jq / SQL / search infrastructure. So I curate, normalize, validate (zero invalid records), enrich with stars/downloads/installs, and refresh.

Per-record fields are typed — categorizationTier rates each record 87-100% specific (vs vague “tool” labels). Open question for the sub: how do you handle tier-of-specificity in your own dataset categorization work? My current rubric is per-dataset config-driven but I’m curious what others do.

Free samples (CSV + JSON, MIT-style permissive): https://github.com/futdevpro/niche-datasets-free

Includes mega-sample.json (5 random records from each of the 20 datasets, 100 records total).

Paid version on Gumroad — $34 launch price (83% off $198 list), monthly refresh on AI Models Pricing because OpenRouter changes weekly, quarterly on the rest. Linked from the GitHub README if anyone wants the full thing.

Happy to answer questions about the catalog, methodology, or specific datasets.

I am unable to convince myself to use one method.
Some methods that i thought of were :

1. I use normalization for full training data of one subject across all features. In this method, i am introducing some kind of lookahead bias, and also this loses on some information which could have been valuable. And also when i want to use one model ( suppose regression with gradient descent) for the subjects combined, then I am unable to judge if this will be a good method.
2. A bad method was to not care about the subjects, and just normalize across full feature. but this just feels wrong to me.
3. I was reading about cross sectional normalization which ranks the subjects and does some kind of normalization. But i am unsure how that would be useful.
4. Another way i found was by using some rolling window, where i keep normalizing not over full data, but the past window data. This seems better but here also what choice of window should be done, and there are lot of questions.

And the bigger problem over all of these is the time series . I would lose quite a lot of information when i don’t consider these. ( although not all features have a big factor of this).

I’ve been building an open dataset of geopolitical and supply chain risk events scraped from public feeds (GDELT, ACLED, GDACS, NASA FIRMS, WHO DON) for the past few months. Around 70k events at this point. The thing that surprised me when I cross-checked against mainstream news coverage: only about a quarter of those events have any major-outlet article attached.

The other ~72% are silent. Flagged in at least one public feed but never picked up by major news. I’d assumed those would all be low-severity noise (small protests, minor weather flags, single-source rumors). They’re not. Roughly a quarter of the silent set is still rated critical or high severity by the source feed itself, which works out to ~14k events nobody covered. ACLED specifically dominates the silent set — local conflict events that don’t make English-language outlets.

The cross-check has obvious limits worth flagging up front: my “news coverage” is a Google News fetch (so paywalled or non-English coverage gets undercounted), and the severity is graded after the fact by an LLM step (so wrong angle on ambiguous events). Both are best-effort. But the headline gap — ~28% news overlap — is just a SQL join, not LLM-dependent. Events are geocoded by region, no PII. Actor names from ACLED are excluded per their license.

The deduplicated event/chokepoint/entity tables are up on Hugging Face and Kaggle as Parquet + a 10% CSV sample, CC-BY-NC-SA. Browsable map version is at tremorwatch.com if you want to poke at individual events first. Curious if anyone has tried something similar at this scale and how you’d refine the coverage definition (different news source mix, embedding-based fuzzy match, etc).

Disclosure: I built this — part of an early-stage startup (Volt AI). Dataset is free under CC-BY-NC-SA, no paid tier exists yet. Posting under r/datasets self-promo guidelines; happy to adjust format if mods prefer.

I’m releasing the NAICS Contagion Map, a dataset designed to bridge the gap between the physical economy (NAICS) and financial market taxonomies (GICS).

The goal was to map how volatility in upstream raw materials (Tier 4) systematically ripples down to consumer-facing products (Tier 1). This is particularly useful for anyone doing economic modelling, supply chain resilience analysis, or ESG/Risk research.

What’s inside the CSVs:

• 340+ NAICS Nodes: Each assigned a Contagion Score (1.0 – 10.0) based on upstream concentration (HHI) and structural importance.
• 1,100+ Topological Edges: Mapping the exact flow from Tier 4 (Commodities) -> Tier 3 (Extractors) -> Tier 2 (Processors) -> Tier 1 (Assemblers).
• NAICS to GICS Bridge: Each node is mapped to its financial sector equivalent.

Methodology: This is a derived dataset. The structural tiers and contagion scores were generated via a deterministic heuristic algorithm I built that analyzes industrial interdependencies. While the raw NAICS data is from Census/GICS registries, the relationship mapping (edges) and risk scoring are my original derivation.

Access GitHub Repo

Full Disclosure : I am the creator of this project. I’m sharing this as a free open-source intelligence drop for the community to play with. I’d love to get feedback on the edge logic or hear how you’re using the topology in your own models.

Sharing a dataset I’ve been maintaining since the MV Hondius hantavirus cluster started in early April.

Aggregated from primary public health sources: WHO Disease Outbreak News, CDC HAN advisories, ECDC bulletins, PAHO weekly reports, ProMED-mail, and national health ministries. Cron pulls every 30 minutes, normalizes case definitions per WHO DON600 framework, geocodes to city or province level where source data permits, dedupes against the archive.

Format: JSON
License: CC-BY-SA 4.0
Endpoint: https://hantaosint.com/api/v1/public.json
Dashboard: https://hantaosint.com
Methodology: https://hantaosint.com/methodology

Fields: case_id, date, country, region, virus_strain, confidence_level (confirmed/suspected/probable/monitoring), source, source_url, lat, lng

Confidence levels are kept separate rather than conflated, which most outbreak trackers don’t bother with. Historical outbreaks included for retrospective analysis: 1993 Four Corners, 2012 Yosemite, 2018-19 Epuyen.

Use cases I built it for: time-series modeling of cluster spread, retrospective comparison of hantavirus outbreaks, surveillance signal for travel medicine research.

Happy to add fields if researchers need additional structure. Open to feedback on the schema and source coverage.

https://zenodo.org/records/20130055
This curated dataset provides a comprehensive and high-fidelity record of the judicial executions carried out in the United States during the year 1999, which represents the historical peak of capital punishment activity in the modern era.
https://zenodo.org/records/20130055
https://github.com/lightbluetitan/us-capital-punishment-1999

I’m researching gaps in publicly available recurring data that people regularly need for analytics, ML, automation, monitoring, or business workflows.

I’m especially interested in data that is technically public but still difficult to use because it is:

• trapped in PDFs
• scattered across websites
• updated inconsistently
• available only through dashboards
• difficult to scrape
• missing historical archives
• lacking APIs
• poorly standardized

Examples could include:

• government notices
• procurement/tender data
• financial filings
• real-estate listings
• agriculture pricing
• shipping/logistics updates
• business registries
• market prices
• legal/regulatory documents
• municipality/city data

I’m trying to understand what robotics and embodied AI teams actually need when collecting real-world training data.

The use cases I keep hearing about are:

-robotic hand manipulation

-grasping and pick-and-place

-soft and fragile object handling

-tabletop tasks

-warehouse tasks

For teams working on imitation learning, VLA models, or robot manipulation, what is usually the biggest bottleneck?

-not enough real-world data

-task diversity

-camera and sensor consistency

-annotation quality

-hardware-specific data

I work with a small team connected to robotic visual data collection, but I’m mainly trying to understand what teams actually need before going too deep in the wrong direction.

Disclaimer, I’m a developer advocate at Bruin. I previously worked in data analyst and then data engineering roles for almost 10 years, and now at this job I finally have the freedom to play around with data just for fun. This community has always been my go to place to find cool datasets.

That’s why I’m excited to share this announcement with you but I promise to keep the promotional talk very minimal.

I’m sure many of you use AI agents to analyze data, build dashboards, and share them with friends and others. Bruin has a lot of open-source tools for data ingestion, transformation, orchestration, and visualization. Today we are announcing the general availability of Bruin Cloud which is the managed service of those free open-source tools.

I’m personally excited because as a dev advocate I’ve focused mainly on our open-source tools but managing and deploying them locally is sometimes an obstacle for someone that just wants to play around with data – so the free tier (no payment required) version of Bruin Cloud will give you enough credits to get started to run your pipelines but more importantly analyze your data using the AI data analyst and dashboard builder.

Check out the open-source tools: https://github.com/bruin-data

If interested, feel free to check Bruin Cloud too: https://cloud.getbruin.com/register

Basically title.

For context: I am building a startup and for demo purposes we want to setup a new demo tenant with fake business data. The closes thing I’ve found is the Microsoft Contoso dataset, but like many other options, its just a data set, not a hosted datawarehouse that keeps getting (preferably daily) updates.

Ideally I’d just plugin with sql db credentials and go to town with a read only user.

Does any1 know if something like this exists?

My project is to implement this ml model into a diary factory but im a fresher , please help. Thank you

im building gym app and i want real videos for it.

and is there any api with free development plan?

https://www.kaggle.com/datasets/akshankrithick/b2b-saas-account-health-dataset

Synthetic but realistic B2B SaaS dataset modeled after platforms like Datadog, HubSpot, and Amplitude. 50,000 customer accounts with 18 features covering product engagement, billing, and support metrics.

Three Prediction Tasks

1. Churn prediction (binary): Will this account cancel within 90 days? (~9.5% churn rate)
2. Revenue prediction (regression): What is this account’s next-month revenue?
3. Health segmentation (multiclass): Thriving / Stable / At Risk / Critical

I have Ulcerative Colitis and a nerd brain. I’ve been tracking my bowel movements for 10 years. I built myself a little dashboard to log every stool. So I have date, time, Bristol stool type, urgency, and any blood present (because UC).

Maybe I’m not the first person to do this, but if I am then there might be some use that the data could have?

Does anyone have any suggestions about what I could do with the data? Any kind of value for researchers?

I did skip tracking for about two years in the middle so it’s actually about 8 years worth of data but doing back 10 years.

Yesterday out of nowhere, the Pentagon released the first tranche of the PURSUE archive which has hundreds of declassified UAP records going back to the 1940s, sourced from the FBI, NASA, etc. The raw release is mostly scanned PDFs with no text layer, which makes the whole thing essentially un-queryable without an OCR pass.

Ran it through a parser to produce clean markdown and JSON. Free to download: https://pursue-release-01-parsed.vercel.app/

And I threw in a map and search for fun bc why not. The data is from the Dept of War (unaffiliated), the parsing came from Extend (I work here) and the site was generated with Codex (unaffiliated).

Hello,

Does anyone know how to download/have a link for the full National Public Database leak? I tried searching extensively on the clearnet and dark web but I can’t find anything other than 2 old Github repo’s with broken download links. I just want to explore the database and do some data analysis stuff on it, nothing bad 🙂

Any help would be greatly appreciated!

Hi,I built an app that preserves, encrypts, searches, reuses, and hands off the full work traces people create with Claude, Codex, Cursor, OpenClaw, and other AI agents.

Some technical details:

– AES-256-GCM encrypted local vault for transcripts, attachments, and state

– No DataMoat cloud vault or server-side transcript storage

– Vault keys and transcript data stay on the user’s machine

– Supported sources today include Claude CLI, Codex CLI/app local sessions, Claude Desktop local-agent sessions on macOS, OpenClaw, and Cursor agent transcripts

– Captures locally written thinking/reasoning blocks when the source tool stores them on disk

– Stores both raw source records and normalized searchable records

– Supports encrypted attachment blobs for supported images, PDFs, documents, and other files

– Password-based unlock with an scrypt verifier

– Optional TOTP authenticator support

– 24-word BIP39 recovery phrase and one-time recovery codes

– Secure Enclave-backed unlock path on supported Macs, with Touch ID in the packaged macOS app

– Packaged macOS app is signed and notarized; Linux source install is available; Windows ZIP builds are available but still unsigned

We believe every person and company should have the fundamental right to own their AI data and build their own data moat.

Source:

https://github.com/max-ng/datamoat

If you want to support the project, please consider starring the repo. Thank you!

Built a catalog of AI-generated impossible products

The database is more interesting than the site, so here it is as a dataset.

What’s in it:

• 126K English product names + AI-generated descriptions + images
• 35K manually categorized into 18 labels (Useless, Anti-Productivity, Quantum Junk, WTF, etc.)
• 28K scored by a custom “Crap-O-Meter”, a multi-step AI pipeline rating text coherence, image relevance, and creativity/absurdity on 0–10 scales

Three configs: full (everything), featured (manually curated 35K), evaluated (with scores)

CC BY 4.0. Use it for creative text generation, humor/absurdism research, or placeholder data that’s more interesting than Lorem Ipsum

Disclosure: I run the Live Events Standards Council, which is working on this problem. Sharing because the data gap itself is genuinely interesting and I’d love input from people who work in this space.

Something I haven’t seen discussed anywhere:

The US ticket refund insurance market is $2.01 billion annually. 13.6% CAGR projected through 2035. Every single policy in this market is currently priced as if every venue carries identical risk — because there is literally no venue-level risk data in existence anywhere.

No public chargeback rates by venue. No cancellation frequency by platform. No loss ratio transparency by ticketing provider. The FTC documented a ~10% chargeback rate in high-fraud ticketing contexts versus 0.6-1% e-commerce baseline — but that data isn’t broken down by venue, platform, or event type. Every underwriter is flying completely blind on risk differentiation.

This matters now because the DOJ-Live Nation settlement just opened a newly competitive market with 14,700+ independent venues and 15+ competing ticketing platforms — none of which have any certification, compliance data, or way for insurers to differentiate between them.

Analogous markets that built certification infrastructure — restaurant health grades, IIHS auto safety ratings, LEED building certification — documented 13-55% reductions in adverse events once a public quality signal existed. The mechanism is consistent: visible certification changes consumer selection behavior and gives operators incentive to comply.

We filed a public-interest submission in the Live Nation federal remedies proceeding making the actuarial case for why venue-level certification matters: https://liveeventscouncil.org/LESC-court-filing/

If anyone here works in insurance data, actuarial modeling, or regulatory datasets in adjacent industries — genuinely would love input on methodology for building the first venue-level risk dataset in this market. Open research volunteer role if anyone’s interested.

The graphic was interesting in that there seemed to be no rhyme or reason as to why one U.S. state might have a greater incidence of autism than another. But my question is, Is it possible to get autism incidence data from the CDC? Once T***p started his second term, the CDC data website was locked down. I don’t understand how stats for every state were available. (The ADDM data is site-specific, not state-specific.) Unless…*special ed* data is being used, which would most likely be readily available on a state-by-state basis.