I’ve been trying to access the ICMR Young Diabetes Registry dataset for my research. I submitted the official data request months ago but haven’t received any response. I also reached out directly to one of the researchers involved, but unfortunately I still haven’t heard back.

Has anyone here successfully obtained this dataset? Is there another contact person, process, or institution I should approach? Any advice would be greatly appreciated.

Please let me know if you know of any other datasets that concerns type 1 and type 2 data and diabetes and pre-diabetes on indian population.

Been working on a football prediction project and couldn’t find a dataset that had

the actual context needed to model match outcomes — just raw results everywhere.

So I built one from scratch on top of the International Football Results dataset

by Mart Jürisoo (the well known one on Kaggle with 49,000+ matches going back to 1872).

What I added:

**Elo ratings** — built from scratch, updated after every single match across 150

years. Both teams’ ratings, their difference, and the expected win probability

going into each match.

**Rolling form** — win rate, goals scored, goals conceded, goal difference, clean

sheet rate, both-teams-scored rate, scoring rate, and win streak. Computed at

three lookback windows: last 5, last 10, and last 20 matches. For both teams.

**Head-to-head history** — based on the last 10 meetings between those two specific

teams. Some teams have persistent edges over specific opponents that their general

form doesn’t explain.

**Fatigue signals** — days since each team’s last match and the difference between

the two.

**Penalty reliance** — fraction of each team’s historical goals that came from

penalties, pulled from the goalscorer dataset.

**Shootout composure** — historical penalty shootout win rate for each team, from

the shootouts dataset.

**Tournament context** — World Cup, qualifier, friendly, neutral venue, competition

importance weight, confederation.

The thing I spent the most time on: every feature is computed in strict

chronological order using only data that existed before that match was played.

State updates happen after each row is recorded, never before. No lookahead,

no leakage anywhere in the 102 columns.

102 features total. 49,094 rows. result column (H/D/A) included as the label.

Drop date and result, plug into any classifier.

Dataset is fully documented with column descriptors for every feature.

Link: https://www.kaggle.com/datasets/kriishgulati/football-match-results-1872-2026-with-ml-features

Built on top of the original dataset by Mart Jürisoo — full credit and link

in the dataset description.

Hello, I recently built a website about CO2 emissions of private jets. I’m looking to expand on (pleasure) cruises but I’m unable to find a historical data set.

Can anyone help me get this data or know where to find it?

Here’s my project to give insights to what my goal is with the data.
[https://paperstraw.info/](https://paperstraw.info/)

Genuinely trying to understand this from people actually building.

If clean, licensed, fully rights-cleared video data existed at the volume and style you needed, would you use it instead of scraped data? And if not, what’s the actual blocker? Cost, availability, doesn’t matter to your legal team yet, something else?

Building in this space and would rather understand the real objection than guess at it.

Happy to go deeper in the comments

Hi,

I made a new sub that’s tangentially related to this one and want to advertise it but don’t want to break any rules. Can any mods tell me the rules?

Hello everyone, hope you are all doing well.I live in Brazil, where Polymarket is currently geoblocked (ironically, sports betting sites work completely fine here, go figure). I am looking to extract Polymarket data to incorporate into my predictive models. Prediction markets serve as an excellent proxy for public sentiment, such as forecasting the final outcome of a World Cup match.

I considered using a VPN, but I know Polymarket actively blocks them. Does anyone know of an alternative repository on GitHub, Hugging Face, Kaggle, or Google Cloud BigQuery that hosts historical Polymarket data (order books, transaction-level data, or market resolution history)?

Ideally, I am looking for structured formats like .csv, .parquet, or public SQL tables so I can bypass the local geo-restriction. Any leads or links to open-source data dumps would be highly appreciated.

I scraped the census data files and arranged them in a kaggle dataset. Also added a notebook for quick-start. There are attributes on demography and housing. Unfortunately the attributes are all in german, but I did not want to change the original data with half assed translations (LLMS will do a much better job in explaining what is what than I could anyways). I think this is a very neat geo dataset with interesting correlations.

I built NorthTick after getting tired of patchy Canadian market data. It is local Parquet files, not an API: - TSX/TSXV/CSE/NEO OHLCV - Daily data back to 1993 - Minute bars from 2020 - Ticker metadata included - Free sample available Site: https://northtick.ca Disclosure: I built it. 

I built API for SEC 13F data with 57 endpoints as a site project. The data collection and the collectors are running on AWS, so the resources can be expended. I am thinking to add more data and endpoints. It is for free if someone wants to try it out.

So I was trying to figure out what Android app to build next, and the first step was doing some market research. I wanted to see what apps were already out there, so I ended up creating this dataset.

It contains app data across the top 10 fastest-growing Android categories. If you’re planning to build an Android app or just want to analyze the market, feel free to use it.

It also includes my GitHub repo, so you can customize it and scrape whatever kind of app data you want.

I want to get this DR(eye)VE dataset from AImage Lab
https://aimagelab-legacy.ing.unimore.it/imagelab/page.asp?IdPage=8

But the form on this site doesnt seem to work. So I tried contacting them through the methods in their new website
https://aimagelab.unimore.it/contacts/

But no responses to emails and even calls are stuck in a automatic response loop in Italian.
Does anyone have this dataset, or a similar one or know how I could ontain this via AImage Lab?

Any support is welcome! Thank you.

hi! So I have this dataset where the objective is to predict a student’s health risk, but I’m a lil confused about how to handle the null values. These are the % of null values for the columns:

 id 0.000000 health_condition 0.000000 sleep_duration 11.012943 heart_rate 1.135073 bmi 2.013946 calorie_expenditure 7.658878 step_count 2.016554 exercise_duration 1.000017 water_intake 6.300211 diet_type 1.000017 stress_level 12.000064 sleep_quality 8.452690 physical_activity_level 5.306715 smoking_alcohol 4.141791 gender 3.097141 dtype: float64id 

What would you recommend I do for these values? If I were to drop the columns <5%, I would be losing nearly 100,000 values (out of 700,000) which I don’t think is all that good. I thought of using K-means to fill the null BMI values but I don’t know.

I would appreciate any advice! Thanks 🙂

Hello everyone, I’m working on my university thesis and I was wondering if any of you would be willing to share access to datasets pertaining to the movie industry, specifically datasets containing Yearly box office revenue or amount of tickets sold for specific movie genres, more precisely for the horror genre. I started some searches already but the promising leads I found were pretty expensive.

I’m trying to find a database of surnames for use in writing/testing code that converts an author name (e.g, “Stan Sieler”) into a sortable/alphabetizable name (e.g, “Sieler, Stan”).

Many surnames are compound (“de Camp”, “Cartwright-Chickering” (bonus for people who recognize that one!), some with and some without hypens, and some with more than two words.

The U.S. Census database isn’t useful to me … they compress all last names, removing spaces.

(I’m ignoring people like “Arthur Conan Doyle”, whose last name at birth was “Doyle”, but later adopted the practice of using “Conan Doyle” as his surname … confusing librarians around the world 🙂

Any pointers appreciated, thanks!

The PSID has tracked the same American families since 1968 across income,

health, housing, wealth, education, and depression. It’s one of the most

powerful public datasets in social science, but the raw files arrive with no

meaningful column names and require a codebook crosswalk just to understand

what you have.

PSID-SHELF (from U-Michigan) reorganized the data into 34 topic areas with

real variable names. There’s now a browser app built on top of it — search

across all 34 topics in plain English and see sample data immediately. No

download, no account, no setup. Link in comments.

There’s also a local track that produces 34 clean CSVs from your own SHELF

download, ready for pandas, R, or Excel.

Happy to answer questions.

I’m currently working on a cosmetics/skincare startup and one thing I’ve been struggling with is finding reliable market data. Whenever I need information like market size, growth rates, consumer trends, pricing, competitor analysis, retailer performance, ingredient trends, or industry forecasts, I end up finding reports that cost anywhere from hundreds to thousands of dollars.

For those of you who regularly work with market research or data:

Which platforms do you actually use?
Which ones are worth paying for?
Are there any hidden gems that professionals use but aren’t widely known?
How do startups without huge research budgets access high-quality data?
Do you combine multiple sources (government data, retail data, consumer surveys, Google Trends, etc.) instead of relying on one platform?
I’m particularly interested in the beauty, cosmetics, skincare, and consumer products industries, but I’m also curious about general-purpose research platforms.

I’d love to hear what professionals, analysts, consultants, or founders use in their day-to-day work.

Was poking at the SF market and put together a clean dataset of homes + condos currently listed: list price, price/sqft, sqft, beds/baths, year built, lot size, agent, and the Redfin link for each.
A few things that jumped out:
– Median list price is ~$1.27M, median $980/sqft
– Cheapest thing on the market: a $369k 523-sqft condo at 601 Van Ness
– Priciest: a $6.6M unit at 188 Minna — which works out to $3,256/sqft lol
– Year built ranges from 1884 to 2021, which is very SF

CSV/XLSX here if anyone wants to take a look at it: https://docs.google.com/spreadsheets/d/17BhnTFkWtN6cI9Yn9f0BgPcLF6sVEk9T/edit?usp=sharing&ouid=108885207033845537587&rtpof=true&sd=true

Made it with an open-source tool called Bigset where you basically describe the dataset in a sentence and it goes and pulls + verifies the data from the live web.

Happy to pull a different slice if people want -by neighborhood, condos only, under $1M, whatever.

Hello folks i need an advice from DBAs.
I’m building a gentle data extractor from dabases.

What’s the most important metric that can confirm that ongoing data extraction is not harmful for database?

Started out with a question, how do I spend $300 in free GCC credits, and how much could I do with it. I started with figuring out how to query HTTP Archives, pulling CRuX data to correlate sites, and learning a bit about BigQuery along the way. I went from ~12 million total sites and pared that down to 1.5 million that I could verify were live, had enough data to be able to classify/categorize, and then built a front end to access the highlights.

So far, I’ve been focused on identifying key business segments with missing opportunities, classic one click misses, some schema mapping for business type, and wondering why in the world any sane business owner would use Weebly.

What would YOU want to know?

Sharing a dataset resource that may be useful for researchers, data scientists, and investigators.

Image-Meta has indexed the embedded metadata (EXIF/IPTC/XMP) from ~720 million publicly accessible images using ExifTool. The data is queryable via a REST API & web rather than a bulk download.

**What’s in the dataset:**

– Camera make, model, and serial number

– Author, copyright, rights, title, description

– GPS coordinates (where present, subject to strict TOS/paid tier not publicly free available)

– Software chain

– Creation, modification, and index dates

– Filename and document ID

– Creation, Modify Date, Date Found

– Extra JSON supplimental metadata in full per image

**Potential research uses:**

– Camera device attribution studies

– Metadata privacy/leakage research

– Image provenance and disinformation analysis

– Geospatial studies using embedded GPS

– Timeline reconstruction of image publication

**Access:**

Web or Queryable via REST API with field-level boolean search, date ranges,

https://image-meta.com

API docs: https://image-meta.com/api-docs

I just released a free sample dataset of 50 traditional recipes from 25 countries.
Each recipe includes:
Ingredients
Step-by-step instructions
Prep time & cook time
Serving size
Format: JSON
The full dataset contains 1,925 recipes from 194 countries and is available on HuggingFace under the name:
“FoodieAtlas World Traditional Recipes Dataset”
Disclosure: I am the creator of this dataset.