I spent the last year gathering as much Usenet text data as possible from a large variety of resources and including from about 7000 live active servers as well to fill in gaps.

The end result is a searchable/filterable dataset acessible via web+api (free and paid options)

API : https://www.usenet-rewind.com/api-docs

Web: https://www.usenet-rewind.com

Binaries have been filtered out and range is from posts from 1981-Current and constantly populating. the site includes a histogram of distribution of message counts per year.

I welcome feedback or possible use cases. Built on Apache Solr 10, Mariadb 12 (rocksdb) and NVME disk hardware for fastest operation.

[self-promotion] [free] [paid]

Disclosure first: the ads come from a US data center job board I run, so this is my own data and a link to my own project. Nothing is paywalled or email-gated.

What it is. Once a month I freeze every live ad on the board at one recorded UTC instant and publish the aggregates. August 2026 is the first release:

• 1,341 live ads, 62 employers, 41 states, 254 city-state pairs
• 769 ads (57.3%) carry employer-stated pay
• Two CSVs: aggregates (live jobs by state, role family and employer, with shares) and pay bands (median advertised lower and upper bound by role and pay period)
• CC BY 4.0

Zenodo, version DOI: https://doi.org/10.5281/zenodo.22261858 — the series DOI https://doi.org/10.5281/zenodo.22261857 always resolves to the newest edition.

Original source, where the method and every edition live: https://datacenterjobhub.com/data-center-hiring-index/2026/08

Method. The population is every ad live and approved at the cutoff, 2026-09-02T10:15:54Z. Pay is employer-stated only: where a job feed supplied its own estimate the ad counts as not stating pay, and ads with a floor but no ceiling are excluded from the bands. Annual and hourly are reported separately and never converted. A role x pay-period group is published only at 8 or more ranges. “Median advertised lower bound” is the median of the ranges’ minimums, so it is a statistic about advertised ranges, not about earnings.

One cut that is not in the bundle yet. I ran the stated-pay rate by state against the twelve US jurisdictions that require a pay range in the advertisement itself. This was read at a later cutoff, 2026-09-03T15:10:37Z and 1,351 live ads, so it does not tie exactly to the frozen edition above:

• 86.1% of ads state pay in the 9 mandate states that have live ads; 51.8% in the other 31
• dropping the largest employer, which states pay in 174 of its 175 ads, widens that to 84.9% against 44.0%
• 13 employers run 5 or more ads on each side of the line. Compared with themselves: 93.8% in mandate states, 81.0% elsewhere. So it is not only employer mix
• of the eight largest data center markets, only California has a posting mandate. Virginia has none and states pay in 76% of 152 ads; Texas manages 41% of 340

That CSV goes into the September edition’s folder so it is citable rather than just asserted here.

Limits. One board’s ads, not the market: it skews to employers whose careers sites are ingested, and 62 employers is not the industry. Mandate status is as of 2026-01-30. 33 ads carry no state and sit outside the state figures. Every number above comes from a script in the repo.

Just uploaded the full 5.94 billion TikTok video dataset to Hugging Face. It’s fully open source:
https://huggingface.co/datasets/kuben-developer/tiktok-videos-4b

This dataset was collected using a TikTok mobile app reverse-engineering method I developed a few years ago. The method allowed me to extract billions of videos, profiles, comments and replies, hashtags, sounds, and more.

Full write-up and code here:
https://tiktok-api.seeksocial.io

Disclaimer: The TikTok app exposes 24 endpoints that can be accessed without a TikTok account, so the data itself is publicly accessible. But accessing it this way is probably still against TikTok’s ToS. Also, the full code is not free, I charge a small fee for access to it.

I’m researching how to build or fine-tune an audio-to-chord-recognition engine comparable in ambition to Song Master Pro / Auralis Sound Prism.
The goal is not basic major/minor chord detection. I need reliable recognition of dense harmonic material: jazz, soul, funk, neo-soul, Brazilian music, film music, and arrangements with chords such as maj9, 6/9, m9, m11, 13, altered dominants, slash chords/inversions, secondary dominants, modal interchange, suspensions, passing harmony, etc.
Most public datasets I’ve found seem too limited: either simplified chord labels, weak annotations, or repertoire that does not really cover sophisticated harmony. In particular, I need time-aligned audio + chord labels, ideally with beat/downbeat information and a rich, consistent chord vocabulary.
My questions:
Which open datasets are genuinely useful for this level of chord-recognition work?
Are there any commercial/licensable datasets with high-quality, detailed chord annotations that can legally be used to train a model and ship it in commercial software?
Is a dataset such as iReal Pro-style chord charts, Hooktheory, Ultimate Guitar, Chordify, or similar usable in any legitimate/licensable way — or are they generally not viable due to rights and annotation quality?
For a serious model, is the realistic route to combine public datasets with a privately licensed/hand-annotated corpus? If so, roughly how many accurately annotated tracks would be needed before it becomes meaningfully good at jazz-influenced harmony?
Are there papers, benchmarks, companies, or dataset vendors I should study before spending money?
I’m specifically looking for practical, legally usable data sources—not advice to scrape chord sites. Any experience from people who have trained MIR / chord-recognition models would be very valuable.

Hey all, I work at CompanyData. A while back we published free business datasets for a few major cities on Datahub: https://datahub.io/@companydatadotcom (i can also share a link from our own website, just let me know in comments)

Each dataset covers companies in one city, sourced from official government registries. Current fields per company: registered address, registration number, SIC industry code, revenue, employee count, and ownership information through headquarter linkages, so you can see which companies belong to the same group. The Berlin one turned out to be the most popular by far, which we didn’t expect.

No commercial angle here, downloads are free. We publish these because registry-sourced company data should be easier to get your hands on than it is.

What I’d really like feedback on is the fields themselves. Which ones actually matter for your work, and what’s missing? Founding date, legal form, ownership links, NACE instead of SIC, officer counts, website domains? We decide what goes into the next batch based on what people here say, so be critical. Also open to suggestions for the next city or country.

I recently completed a dataset containing 51,080 publicly funded broadband locations in Virginia.

The project involved combining multiple public datasets, including statewide address points, administrative boundaries, broadband funding records, and service qualification results into a single address dataset.

The largest challenge was not collecting the data but reconciling it. No single source contained all funded locations. Records had to be standardized, deduplicated, geographically validated, merged across multiple funding programs, and linked back to authoritative address points while preserving provenance.

The final dataset contains:

• ADDRESS
• CITY
• COUNTY
• STATE
• ZIP
• LATITUDE
• LONGITUDE
• SOURCE_COUNTY
• FUNDING_TYPE
• SOURCE_DATASET
• VATI_INCLUDED
• BEAD_INCLUDED
• CURRENT_APB_RESULT
• APB_V2_RESULT
• APB_can_apb_service_address
• APB_lookup_status
• APB_match_quality
• APB_lookup_type
• APB_available_bundle_count
• APB_available_bundle_names
• APB_lookup_timestamp

After constructing the dataset, each address was evaluated through the public All Points Broadband ordering system and classified as:

yes
no
unknown_no_suggestion
unknown_bad_resolution
unknown_error
not_checked

Several issues emerged during processing, including duplicate addresses appearing in multiple programs, inconsistent formatting across source datasets, addresses resolving to incorrect counties, and addresses resolving to entirely different states.

Rather than discarding problematic records, unresolved locations were retained and classified separately so that every funded location remained accounted for.

The project ultimately became an exercise in address matching, GIS integration, provenance tracking, and public infrastructure data management.

Primary data sources:

Virginia Geographic Information Network (VGIN)

https://vgin.vdem.virginia.gov

Virginia statewide address points

Virginia Administrative Boundary Dataset

Virginia Telecommunication Initiative (VATI) 2022

Virginia Telecommunication Initiative (VATI) 2023

Virginia Telecommunication Initiative (VATI) 2024

Virginia Broadband Equity, Access, and Deployment (BEAD) post challenge locations

Virginia Office of Broadband

https://www.dhcd.virginia.gov/broadband

Virginia Broadband Map

https://vast.virginia.gov

Northern Shenandoah Valley Regional Commission

https://www.nsvregion.org

Rappahannock Broadband Authority Project Resources

https://rappbroadband.org/about-the-project/

I’ve been building a normalized dataset and API from SEC ownership filings (Forms 3, 4 and 5, including amendments). It currently covers roughly 869,000 filings and more than 2.4 million transaction line items, including issuer and reporting-owner data, roles, transaction codes, timestamps, and price-quality information.

I’m not launching it as a public API yet. I’m looking for one person who already works with SEC ownership data and has a concrete research, backtesting, or data-engineering task they would be willing to run against it.

The pilot currently provides one filtered transaction endpoint with cursor pagination through an individual, revocable API key. There is no bulk export or separate filing/footnote endpoint yet.

The documentation is available here:

https://insiderfilings.info/api/v1/docs/

If this fits something you are already working on, please comment or DM me with a sentence or two about your use case. In return for access, I’m mainly looking for candid feedback on whether the API actually saves work, where the data model is unclear, and what blocks the workflow.

We’re an AI training data marketplace working with companies building robotics and physical AI systems. We’re currently looking for a data supplier, collector, or lab that has a large amount of egocentric video data and is willing to sell or license it.

Currently looking for 10,000 to 50,000 hours, with priority for OTS (off-the-shelf), already recorded material. This could be previously sold or unsold. We’re interested in various tasks and regions, in both residential and commercial settings.

We have the budget to purchase, and the timeline is 3-6 months.

If you’re able to produce those hours, let us know what your collection volumes per month look like. Happy to consider if your data and price are the right fit.

We’re also interested in various setups (IMU, stereo, hand tracking, annotation, narration, etc.).

Disclaimer per sub rules: I built and operate this — it is self-promotion, disclosed up front.

The dataset: real SEC EDGAR ownership-filing metadata — Form 4 (insider trades), 13F-HR (institutional holdings), Form 3 — for a sample of 23 major US tickers, last ~120 days (675 records currently). Original source is the SEC’s public EDGAR system (public domain): https://www.sec.gov

Fields per record: ticker, CIK, form type, filing date, accession number, primary document, and the SEC source URL.

Access: free demo API key at https://edgarfeed.onrender.com (no card).

Honest limits: metadata-level records only (not parsed line items yet); sample coverage, not the full market; this is a beta demand test, not a finished product. If you need full history, webhooks, or normalized transaction line items, that’s the planned paid roadmap ($0/$29/$79 — not yet verified).

Questions about the data or the approach are welcome.

42 years. 244 apartments. Four decades of rent, vacancies, improvements, preferential leases, and deregulations, as filed with the New York State Division of Housing and Community Renewal.

Doo heelppp guyssss plzzzz 😭😭

I built a tool for qualitative research and I need people to break it.

I’m a student with a background in ML/data science, and I’ve been building a tool that can analyze things like interview transcripts, focus groups, open-ended survey responses, reviews, etc. — basically qualitative text.

It does sentiment analysis, coding, and thematic analysis.

But here’s the problem: I don’t want to test it on fake ChatGPT-generated data.

I’m looking for researchers/students who have real qualitative data they’ve already collected and wouldn’t mind letting me run through the tool.

In return, I’ll send you the analysis/results completely free.

No subscription. No sales pitch. No “book a demo.” 😅

I’m mainly trying to figure out:

– Are the codes actually useful?

– Do the themes make sense?

– Does it save you time?

– Where does it completely screw up?

– Would you actually use something like this in your research?

You can anonymize/redact anything sensitive before sending it.

If you have a thesis, dissertation, research project, interview transcripts, focus group data, survey responses, or anything else qualitative sitting on your laptop, I’d genuinely love to test it.

Comment “interested” or DM me.

And yes, I’m specifically looking for people who are willing to tell me “this is terrible” if it is. 😂

Looking for an open-access dataset with firm-level AI workforce or AI skills data — global coverage, up to the present.

I’ve looked at Revelio Labs and Cognism, but all of them are paid licenses.

Is there anything open or free for academic use?

Thanks.

Hey everyone! I’ve been working on a little side project I wanted to share: latex-ocr, a standalone formula OCR model — you feed it an image of a math formula, it spits out the LaTeX source.

The main hook: it’s only 67M parameters, so it runs comfortably on a laptop CPU. No GPU, no 300M-parameter monster to load. It’s a CoCa-style model (contrastive captioner adapted for OCR), and despite the small size it beats the 107M UniMER-tiny baseline and gets pretty close to the 325M one on plain formulas.

The part I’m actually most proud of is the dataset. Real papers don’t just use plain symbols — you see mathbb{R}, mathcal{F}, mathfrak{g} everywhere, and existing OCR datasets basically ignore font styles, so models trained on them can’t read (or hallucinate) those macros. So I rebuilt ~1.3M formulas with a MathJax → SVG → PDF → PNG pipeline and injected font-style macros with semantic heuristics (number sets → mathbb, vectors → mathbf, differentials → mathrm). On that styled test set it clearly outperforms all the baselines — fair warning though, those baselines are zero-shot on styled data, so take that comparison with a grain of salt. The plain-split numbers are the like-for-like ones.

Everything is open: model weights and dataset on Hugging Face, training recipes included if you want to reproduce or fine-tune it yourself, MIT license. There’s also a FastAPI server and a Gradio web UI, so you can drag-and-drop an image and see the LaTeX with a rendered preview.

Repo: https://github.com/PadishahIII/latex-ocr Model: https://huggingface.co/PadishahIIIXXX/latex-ocr Dataset: https://huggingface.co/datasets/PadishahIIIXXX/latex-ocr-dataset

Happy to answer questions about the training setup, the data pipeline, or anything else. Would love feedback — especially if you try it on your own gnarly formulas and it breaks, that’s genuinely useful.

Disclaimer: this is my own open-source project.

I maintain a daily-refreshed dataset of Pakistani mutual fund NAVs, built from the public MUFAP data (the industry association at mufap.com.pk is the original source). It is served as a free no-key API.

Per fund you get the current NAV, full NAV history and computed returns, plus AMC and category metadata. JSON, MIT licensed, no PII.

https://github.com/saadsalmankhan/pakistan-mutual-funds-api

Happy to expose extra fields if they are useful for research.

A high-fidelity synthetic retail POS (Point of Sale) transaction dataset containing over 1,190,000+ synchronized master invoices and 4,190,000+ item line details across multiple relational schemas. Generated using a custom, highly-optimized Prolog simulation engine, this dataset is enterprise production-grade and perfect for database stress-testing, query optimization benchmarks, and complex retail machine learning models.

Disclosure per rule 1: I work at Schema Labs and one of the three models tested is ours

The thing that started this was annoyance rather than research. most datasets any of us actually work with show up with columns called V1 through V57, or metric_14, or some four character code from a system nobody maintains. meanwhile benchmarking mostly happens on clean data where the age column is called age, which quietly rewards a model for reading the header instead of the values.

so we took 20 numerical classification datasets off OpenML and ran each twice. once as published, once with every column name stripped. same splits otherwise.

mean ROC-AUC with names gone:

Schema-2: 0.9230
TabuLa-8B: 0.8658
ConTextTab: 0.8541

ours held flat, 0.9230 with names and 0.9230 without. the other two both gave up roughly 7 points once the headers went. flagging that these are our internal runs, not third party replicated.

one thing that doesn’t fit the story but is true: stripping names doesn’t strip position, and column order still carries signal on some of these. we didn’t control for it and probably should have.

all 20 are public on OpenML so anyone can rerun this against a model we didn’t cover. would like someone to get a different number. if you want the full per-dataset breakdown and the methodology, comment and I’ll send it over.

Looking for information about this company

I’ve been building a tool that takes a list of crypto wallet addresses and enriches them with labels you’d normally have to look up individually through block explorers and intelligence platforms.

The idea is pretty simple: give it a list of addresses and get back things like:

• Entity / owner — e.g. Binance: Hot Wallet, Coinbase: Cold Wallet, Deribit
• Category tags — Centralized Exchange, Hot/Cold Wallet, Mixer, Gambling, Sanctioned, High Transacting, etc.
• First-funder relationships and contract flags
• Multiple chains — Ethereum, BSC, Base, Arbitrum, Polygon, Avalanche and others
• Clean CSV/JSON output, with one row per network

I built it mainly because doing this manually for large address lists gets painful pretty quickly. The pipeline can currently process around 100k addresses in under an hour, depending on the chains and data involved.

The interesting part is that I initially thought the main value would be the speed/cost compared with existing intelligence platforms, but I’m not completely sure what people would actually care about most.

I’m curious for anyone working with blockchain data, AML/compliance, fraud investigations or crypto analytics:

• Would bulk wallet labeling actually solve a meaningful problem for you?
• Which labels would be most useful?
• Would you prefer an API, CSV upload, or something else?
• For large lists, is speed more important than label accuracy/coverage?
• Are people generally using expensive intelligence platforms for this, or building their own pipelines?

I’m trying to figure out whether I’m solving a real problem or just making something technically convenient, so I’d genuinely appreciate feedback from people who have dealt with this kind of data.

On Hugging Face SalatielJordao/incels_is_1788055901

PS: Mods Reddit filters removed the last post, I suppose its because of the forum name associated on the title, let me know if its ok. Im the main author, you can AMA if you want.

I checked Hugging Face for a neuroscience/biology QA dataset that wasn’t either scraped Wikipedia with zero voice or dumbed down to the point of being wrong. Nothing fit, so I built one.

NeuroBio GenZ 1K: about 1000 questions like “why does doomscrolling give me dopamine” or “why do I forget why I walked into a room,” each answered in 2-5 sentences, casual tone, but using the actual terminology instead of vague placeholders. Nucleus accumbens instead of “reward center.” GABA instead of “chill chemical.” The system prompt explicitly banned the “dopamine is the pleasure chemical” style oversimplification that’s everywhere in this space.

Fully local generation. Ornith 1.5 35B-A3B, Q3_K_XL quant, through llama.cpp on a Ryzen 7 5700X, 32GB DDR4, 8GB GPU. No API calls, no cloud spend.

The part I actually want to share because it’s a real lesson: I ran the whole generation at 10-13 tok/s because I hadn’t figured out the right server flags yet. After the run finished I found the config that gets the exact same hardware to 30-35 tok/s: speculative decoding via --spec-type draft-mtp plus properly tuned --n-cpu-moe for MoE expert offload. Both configs are in the repo if you’re doing something similar and don’t want to waste the time I did.

Dataset: https://huggingface.co/datasets/luka0x12/NeuroBio-GenZ-1K
Source code + server scripts: https://github.com/luka0x12/NeuroBio-GenZ-1K

CC BY 4.0, use it for whatever.

Looking for someone with experience sourcing large-scale Amazon/eBay product datasets
Does anyone know how companies like Tiendamia, Ubuy, etc. obtain or build large product catalogs from Amazon, eBay, and other retailers?
We’re specifically looking for structured product data at scale, including things like:
Product title / description
UPC / EAN / GTIN / ASIN
Brand / model
Product dimensions
Item weight and, ideally, packaged/shipping weight
Images
Variants
Categories
Pricing / availability where possible
Historical or continuously updated data
The biggest issue we’re trying to solve is weight accuracy. Amazon data is generally usable, but eBay is much messier because individual sellers enter their own product information, and weights/dimensions can be missing or completely inaccurate.
We’re interested in knowing what legitimate sources exist for this kind of data — commercial datasets, data providers, APIs, retailer feeds, catalogs, or ways companies build their own datasets at scale.
If you’ve worked on e-commerce data ingestion, product catalogs, Amazon/eBay datasets, product matching, or shipping-weight estimation, I’d love to hear how you approached it.
Especially interested in sources that can provide millions of products rather than an API where we have to query products one-by-one.

I’ve created a dataset (over 7,700 files/17GB) on Hugging Face that can help with testing and tuning scanned document processing:

https://huggingface.co/datasets/morzel85/synthetic-medical-document-recognition-benchmark

It’s meant to be realistic but fully synthetic (based on Synthea/FHIR).

I used it to assess OCR + NER for de-identification, and because documents are clearly marked as synthetic these are safe for demos. The README file contains all details.

Maybe it will be useful for someone else. Released under cc-by-4.0 (free for any use, watch out: some datasets are produced with components that not allow commercial use).

Source (original): DART (Data Analysis, Retrieval and Transfer System), run by Korea’s Financial Supervisory Service — the mandatory disclosure filing system, same role as SEC EDGAR. Free OpenAPI, 20,000 calls/day on a free key, no commercial-use restriction.

• https://opendart.fss.or.kr/ (site is Korean; API docs have English param names)

What’s in it. Every KOSPI/KOSDAQ listed company’s financial statement line items from annual reports: balance sheet, income statement, cash flow, equity changes. Each row carries the current year amount plus the prior two years side by side, so you get a 3-year trend per line without extra calls. Consolidated (CFS) preferred, falls back to separate (OFS) for companies with no subsidiaries.

The part worth posting about: DART already tags every line with an IFRS taxonomy id — ifrs-full_Assets, ifrs-full_Revenue, ifrs-full_ProfitLoss, etc. — the same taxonomy US/EU filers use. That means no Korean-to-English account-name translation step, unlike most Korea data sources (my other actor, the real-estate one, needs a full translation layer). You get the Korean label (account_nm, e.g. 매출액) and the international taxonomy id side by side for free.

Gotchas (cost me actual debugging time):

1. Company lookup isn’t by ticker — you first pull corpCode.xml (a zip, ~forgot-to-check-Content-Type gotcha), which maps every DART-registered entity (listed and unlisted) to an internal corp_code. Unlisted entities have an empty stock_code field; filter those out or you’ll query companies with no ticker.
2. No single “give me financials” call reliably returns consolidated statements. You have to try fs_div=CFS first and fall back to OFS — a normal, expected branch, not an error path. A company with no subsidiaries simply has no CFS.
3. Status 013 (“조회된 데이타가 없습니다” / no data found) is not an error, it’s an empty result — some report/year combos genuinely don’t exist. Status 020 (quota exceeded) is a real error. Conflating the two silently turns a rate-limit hit into “this company has no 2019 filings.”
4. Amounts come as strings ("514531948000000"), not numbers — cast, don’t trust the JSON type.
5. rcept_no (the filing receipt number) encodes the filing date in its first 8 digits — that’s the only place you get an as-of date; there’s no separate date field on the row.

Real example — Samsung Electronics (005930), FY2022–2024, annual consolidated:

That FY2023 revenue dip and FY2024 recovery is the memory-chip downturn/rebound showing up directly in the raw filing data, three years in one API response, no crawling three separate annual reports.

Disclosure — I built a free wrapper. Raw DART responses are one financial-statement/report/year combo per call, Korean-first field names (thstrm_amount, sj_div), and require the corp_code lookup above. I published an Apify actor that does the corp_code mapping, CFS→OFS fallback, and reshapes rows into English field names (currentAmountKrw, statementType, accountId) — free, no API key of your own needed:

• https://apify.com/ootssu/korea-company-financials

The source of truth is DART, not me — if you’d rather hit opendart.fss.or.kr yourself, the gotcha list above should save you the debugging session it cost me.

Hello all, I have a paper due SHORTLY (my fault, i know) on socioeconomic factors on the Navajo tribe. I want to access data on the US census bureau for the year 2004. This is the site (need Arizona, Utah and New Mexico) but it looks like this: file:///var/folders/t7/thn9pmf93zx7t525wn14vs240000gn/T/TemporaryItems/NSIRD_screencaptureui_yMdzpl/Screenshot%202026-08-29%20at%2022.46.43.png

(WHICH IS INCOMPREHENSIBLE FOR ME) Does anyone know how read this?? I would like to know how to read it because I need to repeat this with other factors such as housing, education, etc…

For context this is data is on Income and poverty in Arizona (2004) and I would be interested in the Apache and Navajo county.

Thank you to whoever can help!