Category: Datatards

Here you can observe the biggest nerds in the world in their natural habitat, longing for data sets. Not that it isn’t interesting, i’m interested. Maybe they know where the chix are. But what do they need it for? World domination?

Dataset With 10k-50k Products With Many Attributes

I am doing a master thesis on how large language models compare to other tools when extracting structured data from natural language. Essentially my goal is to translate something like this:

“I want Asus laptops with relatively good reviews, at least 16 GB RAM, ideally 16 inch screen. Sort all the results by price and reviews”

into something like this:

{

“brand”: “Asus”,

“category”: “Electronics”,

“subcategory”: “Laptops”,

“sort”: [“price”, “review”],

“filters”: [

{

“attribute”: “ram”,

“condition”: “greater_than_or_equal”,

“value”: “16 GB”,

“is_hard_condition”: true

},

{

“attribute”: “screen_size”,

“condition”: “equal”,

“value”: “16 inch”,

“is_hard_condition”: false

},

{

“attribute”: “review_rating”,

“condition”: “greater_than_or_equal”,

“value”: “4”,

“is_hard_condition”: true

}

]

}

using large language models, and analyze how they compare to more traditional tools.

What I need is a dataset that has many products, and each product has at least a category (though subcategories would be ideal), branch, and many attributes which are dynamic, depending on product. For example laptop would have CPU, RAM, screen size and so on, while sofas would have very different attributes. It can be even smaller in size (1k-10k). Is there a dataset for this?

submitted by /u/PurpleYellowLeaf
[link] [comments]

Where Can I Find Top Websites By Traffic, Per Year.

I’m developing a game where players explore the internet through different eras, and I need data on the most popular websites over time. Ideally, I’m looking for a list of the top 100 most visited websites for each year over the past 20 years or so. The data doesn’t need to be all that accurate because the actual rankings will not affect the game, I just need a list of popular websites. Thanks in advance!

submitted by /u/jimmakoulis
[link] [comments]

Where Or How Can I Find E-commerce Datasets

Where can I find dataset to do product analysis? Something that will allow me to time based pricing trends (like best time to buy maybe black Friday sales) or competition between retailers (a product sold on Amazon vs Best Buy or Walmart).

I have visited almost every data platform I know and I can’t find anything that’s good. I feel like web scraping might be the only option.. but I’m new to it and it would take a lot of time.

Any suggestion/idea/resources is appreciated!

submitted by /u/_anomaly_0
[link] [comments]

Downloaded Large Image Dataset That Is Not Organized And Simply #s As Names.

Hey I hope this is a good place to ask.

I downloaded a large image dataset from google/bing/Baidu, unfortunately all the filenames are generic and have no identifying Metadata.

Is there a program/software ideally free/open source if not cheap you recommend that can scan and reverse google image a directory of 100k+ photos download and fill in Metadata.

I especially would like to embed/rename photos to include the people in it, group the photos together for instance 10 photos belong to the same shoot/background with slightly different variations but they are all mixed in and impossible to separate/organize manually.

I appreciate any suggestions!

submitted by /u/ifnbutsarecandynnuts
[link] [comments]

I Need Advice For My Portfolio And Job Search

I am new to data analysis. I have a portfolio with a couple projects I did using excel, powerBI, and mysql. I also collected my own data on kaggle for the MCU revenues project.

I do not have a degree or any professional experience to put on my resume so it’s hard to get a second glance.

Do you know of any companies that might hire a person like me? Or maybe free ways to get experience on my resume? And maybe any tips to spruce up my projects? Or any other tools that would be good to learn?

I am trying freelance but having no luck and fiver charges you and so does upwork after you run out of credits.

submitted by /u/FunkYourself55
[link] [comments]

Looking For A Dataset Of All PhDs In A Country

Hello everyone! I’m currently looking for a dataset of all PhDs defended in a country (preferably in Europe but if you have other examples, I’d love to hear from it too) and going back to at least the 2010s. Ideally, I would need something similar to the French theses.fr open dataset (doc in French here), with a field for the research area of the thesis and the list of PhD advisors and members of the defense jury.

Does someone know a dataset answering these criteria? As far as I understand it, the German dataset does not contain the members of the jury and the British Library lost a lot of data in a hack last year and does not resolve EThOS links for now.

submitted by /u/Unfair_Resident_5951
[link] [comments]

Finding A Dataset Of DSA/CP Problems

Working on an NLP based ML model that extracts key technical terms from raw DSA/CP statements.

The goal is to preprocess problem descriptions, identify relevant entities, and summarise them concisely.

Looking for any open source datasets that fit these requirements

submitted by /u/droffense
[link] [comments]

Looking To Find Specific Datasets Within A State.

Hello, I’m pretty new to retrieving through the internet. I currently need to find addresses and brand names to all gas stations in the state of Arizona. There is about 2200 from what I’ve seen online. I have 0 coding skills so I decided to go the manual route and I’ve gotten 1700 so far through google maps. Is there an easier/faster way to do this. Any tips would be greatly appreciated!

submitted by /u/FirefighterHealthy55
[link] [comments]

Looking For A Dataset For Classifying Electronics Products

Hi everyone,

I’m currently working on a project that involves categorizing various electronic products (such as smartphones, cameras, laptops, tablets, drones, headphones, GPUs, consoles, etc.) using machine learning.

I’m specifically looking for datasets that include product descriptions and clearly defined categories or labels, ideally structured or semi-structured.

Could anyone suggest where I might find datasets like this?
Thanks in advance for your help!

submitted by /u/ExtraPops
[link] [comments]

Income Data In The USA – Specifically Vallejo (CA)

Hey guys, what’s up?

I’m a brazilian researcher finishing data analysis on my PHD in Geography. One of my case studies is the city of Vallejo (CA) and I need to find census data regarding income, whether from households, families, people, whatever. The smaller the geographic unit used, the better. Would anyone know where can I find these types of data? I already explored the USA Census website but I got a little bit confused.

If it interests anyone and to clarify, I’m currently studying the territorial impact that participatory budgeting has on midsized cities.

Thanks a lot!

submitted by /u/pinguimuim
[link] [comments]

Where Do I Get Coral Cover Datasets?

Hello! I’m currently working on a paper and needs detailed coral cover datasets of different coral reefs all over the word. (Specifically, weekly or monthly observations of these coral reefs). Does anyone know where to get them? I have emailed a few researchers and only a few provided the datasets. Some websites have datasets but usually it’s just the Great Barrier Reef. It would be a great help if anyone could help. Thank you! 🙂

(I’ve tried kaggle but the one i need isn’t there unfortunately :'(( )

submitted by /u/takoyaki_elle
[link] [comments]

Any Databases To Pull A Simple Random Sample Of US Addresses?

I apologize if this belongs on r/askstatistics (I posed here since I am inquiring about a dataset). I’m developing a mapping algorithm and require a random sample of US addresses to validate the tool with. I was wondering if anyone had any tips on free databases that would be a statistically sound source to select a simple random sample from? Do you think openaddresses.io would be adequate? Alternatively, I was thinking of randomly generating a latitude and longitude within the United States and then using a reverse geocoding algorithm to provide an address. Though I’m not sure the latter would be a statistically sound method?

submitted by /u/Khianea
[link] [comments]

Sources For Weapons Impact Data In War

Hi all,

Would anyone have insight into a dataset of recent war incidents (ideally the last 25 years, not historical) which tracks specific munitions use and impacts?

Platforms like ACLED, S&P Global, LiveUAMap have good records of specific incidents (a drone strike here, an tank shelling there) but there’s not a focus on the consequences.

My ideal dataset would have date, location, weapon type and some measurement of destruction. The idea is to abstract different ‘types’ of war – Sudan vs Ukraine vs Gaza – in order to examine what would happen if these ‘war’ types hit elsewhere.

Grateful for any insights!

submitted by /u/Trebia218
[link] [comments]

Datasets/where To Look For Wide Range Of Company Data

Hi All, I am a data scientist trying to run an analysis on companies to identify potential new clients for the current company I work for. Currently, we have one very large client (think millions of workers) that we do most of our reporting work on, then we have 3-5 smaller clients (think 10k workers or less). I can’t get too far into specifics, but we essentially are an add-on service to a company’s medical plan (free for the employees to use, but we bill the company). We do outreach to offer our services, but obviously the list of people we can contact is finite and will decrease quickly over time. Our main goal is to identify workplace troubles and situations where work environments affect a worker’s mental health, then provide them with resources to help with whatever they are struggling with. Our busines model is that we can prove that providing these services proactively saves companies millions of dollars in medical spend in the long run (spend a little now to keep employees mentally healthy vs wait for problems to compound into more serious problems resulting in more medical claims spend in the future). I have been looking for an impactful project to work on, and the one that I keep wanting to explore more is to build some sort of clustering algorithm to 1) identify companies similar to the ones we currently work with, and 2) identify other companies that we can provide the most impact for. I would greatly appreciate any recommendations on what resources I can use to compile the data I’m looking for, where to start, or any other ideas to help refine my approach.

Thanks so much!

submitted by /u/CollectionShoddy8445
[link] [comments]

Request For MRI Brain Tumor Images (Meningioma, Pituitary, Glioma)

Hi everyone,

I’m working on my undergraduate thesis in statistics and need MRI images of brain tumors (meningioma, pituitary, and glioma) to apply machine learning techniques. I’m looking for reliable datasets, preferably from institutional sources, hospitals, or public databases.

If anyone knows where I can find these images, I would really appreciate your help!

Thanks in advance to anyone who can assist! 🙌

submitted by /u/ThomKm
[link] [comments]

Need Help Creating A Research Question

Hi all!

I’m taking a statistics class and the assignment is to create a quantitative manuscript. The prof wants us to use a publicly available dataset and then create a research question, do the stats/analysis and write the manuscript (instructions: Choose a research question that aligns with the available data in the selected dataset and is relevant to your chosen context). I’m thinking of using this database:

Hospitalization and Childbirth, 1995–1996 to 2023-2024 — Supplementary Statistics

https://www.cihi.ca/en/access-data-and-reports/data-tables?keyword=birth&published_date=All&acronyms_databases=All&type_of_care=All&place_of_care=All&population_group=All&health_care_quality=All&health_conditions_outcomes=All&health_system_overview=All&sort_by=field_published_date_value&items_per_page=10&page=0

I’m interested in maternal health, but I’m really struggling with creating a research question. I just don’t understand how you can do it from a database – I’m a qualitative researcher so i’m use to always doing data collection. Any help would be so greatly appreciated

submitted by /u/CupcakeCapital9519
[link] [comments]

Help Me With My Data Collection On Vehicle Data Using Simulator.

I’m doing an ML project on a study of various accident scenarios in vehicles, hence I would need to collect datas such as speed and steering wheel angle in timeseries format, at first I used euro truck simulator to collect some data but now I have reached a point where I need to collect the data of two vehicles at a time. Can someone help me with this, Carla is a heavy file and cannot be supported.

submitted by /u/The_Tropicals
[link] [comments]