Category: Datatards

Here you can observe the biggest nerds in the world in their natural habitat, longing for data sets. Not that it isn’t interesting, i’m interested. Maybe they know where the chix are. But what do they need it for? World domination?

Suggestion For Real Life Dataset For University Project

Hi, I need a real life dataset which should have more than 5000 records and could be broken down to atleast 10 tables after BCNF/other normalisation methods. It can be of any domain.I checked various domains like e-commerce and medical fields on kaggle, data.gov, data.world but I am struggling to find a dataset which can be broken down into 10 tables.

Does anyone have any suggestions for a dataset or where I can find this type of dataset?
Thanks!

submitted by /u/cod5_1o
[link] [comments]

Advice For Textual Dataset For A NLP Project

Hi I am doing a NLP based project where I am grouping community of different apps, games to classify then as toxic, supportive or neutral. I want to then compare different communities.

For apps and games, I am using Play Store and App Store reviews. For reddit, I am using past data sets available for different subreddits.

I need suggestions for 2 data types of data. 1. In game chat for different Massive Multiplayer Online (MMO) games. 2. Community Social Media apps posts and comments. Apps other than reddit. I don’t want to do Twitter.

Any suggestions on how to get this data or other data sources that I can explore will be really helpful.

Thanks in advance.

submitted by /u/SilentScroller23
[link] [comments]

How Would You Guys Go About Cleaning Up PDF Data?

I’m trying to take the CDSs (common data sets) of a bunch of universities and compare them together, but I need to find some way to automate the process of extracting the data from them (probably into a SQL database). The issue is that although the questions on the forms are standardized, some universities convery it very differently. For example, look at C7 on the Stanford and Princeton common data sets.

So how should I go about doing this? I tried to leverage Claude’s sonnet model but it didn’t go too well, the context was too large for Claude and it was mixing up multiple fields.

And using something like tabula or pdfplumber doesn’t really help since the universities format it so differently.

Any advice would be appreciated, thank you!

submitted by /u/Roxy201
[link] [comments]

Political Party Co-Preference Dataset

I’m running simulations of ranked-choice and other voting methods and I want to find a survey-supported dataset of related preferences between US politial parties. e.g. people who prefer the green party have some proportional preference for the democratic party. I would also accept a survey-supported metric or principal component analysis on quantitative or qualitative e.g. a political spectrum which captures meaningful variations of preference in survey samples. I would very strongly prefer non-partizan research, however if that is simply not possible to find, it would be at least necessary to find studies from multiple partizan organizations to compare.

(I’m also looking to learn more about who is doing research in this area so I can follow and look for more datasets that come up)

​

submitted by /u/bduxbellorum
[link] [comments]

Seeking Health-Related Longitudinal Datasets

Hi all,

We’re looking for good sources of longitudinal/time-series datasets in the area of health. The datasets have to include repeated entries (e.g., one person through a long time period). The domains we are interested include:

– exercise decisions (e.g., which days people choose to exercise/run etc)

– gym and fitness class attendance

– male/female birth order (per family) or in a delivery room

– dieting & nutrition (e.g., the order that people consume healthy or unhealthy foods each day)

– pain intensity

– weight development and progression

​

We have searched quite a bit on common repositories like Kaggle, Data World, and UCI Machine Learning, but we have not had much luck in finding data that meets our requirements and is a decent time-series. Any specific suggestions (e.g., organisations or repositories that have publicly available health data ) would be very helpful.

​

Please note that we are excluding datasets that show trends that are monotonically increasing or decreasing. This generally removes broader health domains like disease spread (e.g., Covid case numbers), worldwide health development (e.g., global nutrition), life expectancy, and mortality rates.

​

Thank you!

​

submitted by /u/Remarkable_Review327
[link] [comments]

Dataset With 10-15 Tables For SQL Project

Hi,

I’m a Masters student and need to do a project on SQL.

As a part of project, I need to work on a dataset, perform BCNF or other normalization.

After performing a normalization (Preferable BCNF), I should come up with at least 10 tables.

It should contain minimum of 5000 rows. It’s better if its a realistic data or practice commercial datasets from sources like Kaggle etc.

If anyone knows datasets like that can you please share the details?

​

Thanks 🙂

submitted by /u/Eren_94
[link] [comments]

Seeking Audio Data For Multilingual Project – 1000 Hours Needed In Various Languages

I hope you guys are doing well. I’m in need of audio data in several languages. Specifically, I’m looking for 1000 hours of data in each of the following languages:

Australian English Czech Danish Finnish Hungarian Portuguese Romanian Norwegian Bulgarian Croatian Serbian Iranian Persian Swedish Indonesian Chinese (Taiwan) Chinese (Hong Kong) Tamil Japanese

The audio data needs to meet the following specifications: – Audio file format: 16bit, 16khz or 16khz + (or any), WAV, 2 channels or 1 channel – Duration: Minimum 5 minutes and maximum 7 minutes (if other ranges are available, please provide samples and pricing) – Transcription file format: JSON or any other suitable format

Additionally, if you have transcribed files of the same audio data, please provide samples of those as well.

We will be using the data to train an LLM model to recognize events in text, and we will also require validation along with it.

If you have any leads, suggestions, or if you can provide the data yourself, please comment below or send me a direct message. Your assistance would be greatly appreciated.

Thank you in advance for your help!

submitted by /u/Disastrous_Piano7831
[link] [comments]

Where/How To Download All Genome Sequence Out Of NCBI?

I am planning to compare genome sequences, but for that I need data. So I came across National Center for Biotechnology Information. Which is an awesome organization.

But we have an issue here, we need to download them one by one. Is there any way we can download the whole thing into my server at once. Like all the available sequences.

I looked into there FTP page as well. But it downloaded data in different formats, like, gbff, faa, gpff, fna. And I’m pretty sure, there is more data than these, as it was just 8ish M.

Ref:

https://www.ncbi.nlm.nih.gov/datasets/taxonomy/37653/ https://www.ncbi.nlm.nih.gov/genome/doc/ftpfaq/

Any kind help or suggestions are appreciated.

submitted by /u/maifee
[link] [comments]

Where To Find Big Datasets With “updated_at” Date Column?

Hi,

I want to create a sample SCD Type 2 table. To do so, I am looking for some big dataset (>5GB) which updates daily and has “updated_at” date attribute that represents a date when a row has changed.

Example:

Today dataset looks like this

id color updated_at 1 blue 01.01.2024 2 red 01.01.2024

Tomorrow dataset looks like this:

id color updated_at 1 yellow 10.03.2024 2 red 01.01.2024

Do you know where I could find such datasets?

submitted by /u/Betelgeitze
[link] [comments]

A Shared Scorecard To Evaluate Data Annotation Vendors

Evaluating and choosing an annotation partner is not an easy task. There are a lot of options, and it’s not straightforward to know who will be the best fit for a project.
We recently stumbled upon this paper by Andrew Greene titled – “Towards a shared rubric for Dataset Annotation”, that talks about a set of metrics which can be used to quantitatively evaluate data annotation vendors. So we decided to turn it into an online tool.
A big reason for building this tool is to also bring welfare of annotators to the attention of all stakeholders.
Until end users start asking for their data to be labeled in an ethical manner, labelers will always be underpaid and treated unfairly, because the competition boils down solely to price. Not only does this “race to the bottom” lead to lower quality annotations, it also means vendors have to “cut corners” to increase their margins.
Our hope is that by using this tool, ML teams will have a clear picture of what to look for when evaluating data annotation service providers, leading to better quality data as well as better treatment of the unsung heroes of AI – the data labelers.
Access the tool here https://mindkosh.com/annotation-services/annotation-service-provider-evaluation.html

submitted by /u/AdventurousSea4079
[link] [comments]

Looking For A Large Unlabelled Handwritten Text Dataset

I’m looking for a large number of handwritten text (in image format) and they don’t have to be labelled. Simply put, scanned images of handwritten pages, raw, untouched, but lots of them. I’m not even very particular on the language. I mean it would be nice if the images are separated based on their language but even a total mess would be acceptable.

The ones I’ve found so far are all labelled and as the result, there are not that many samples in them. I was hoping if the dataset is not labelled, it would be easier to find ones with a large number of samples.

These are the ones I’ve found:

CENSUS-HWR (1,812,014 samples)

IAM (16,752 samples)

submitted by /u/Ziadloo
[link] [comments]

XGLM-564M – Fine Tuning For Ayacucho Quechua

Hi everyone,

I’m trying to perform fine-tuning on an XGLM-564 model on the Ayacucho Quechua language. Up until now, I’ve found two datasets from Huggingface that could be used to do this.

wikipedia/wikipedia hackathon-pln-es/spanish-to-quechua

I’m facing some problems with the first one and I’m not able to download it because of a missing package called apache_beam. I tried installing it but without any success (I’m using the latest PopOS).

For the second dataset, I’m mainly worried about the quality since I don’t have any knowledge of that language and I’m doing this fine-tuning as part of my uni assignment.

Any help will be greatly appropriated.

Thank you.

​

submitted by /u/dduka99
[link] [comments]

Dataset Of Books, Novels, And Other Literary Sources That Have Been Adapted Into Movies/tv Shows

I’m conducting exploratory data analysis on streaming platforms like Netflix, Amazon Prime, and others to guide content acquisition strategies for a new streaming service. Specifically, I’m investigating the performance of movies and TV shows that are adapted from literary sources compared to original content. By ‘perform better,’ I mean whether these adaptations, on average, receive higher ratings on the streaming platforms themselves or on external rating sites such as IMDb.

A similar question was asked before but never received a response: https://www.reddit.com/r/datasets/comments/gscwtz/request_is_there_a_comprehensive_database_of/

I would appreciate any assistance on this!

submitted by /u/2bapesrealm
[link] [comments]

Request: USDA 12 Basic Soil Class Dataset For Mapping

Hello,

I am a student researching the precontact cultivation of tobacco by Indian tribes in western North America. I am trying to find a map of the 12 basic soil classes (clay, loam, silt loam, etc) but am having trouble. This would allow me to note where nicotiana species have proliferated despite regions being outside of their “natural” range. I am accounting for other geospacial factors as well, but this would be extremely helpful. Any assistance would be greatly appreciated 🙂

submitted by /u/infernoparadiso
[link] [comments]

[Mock] Ideas For A Dummy Inventory Dataset

I’m about to launch into building a dummy warehouse inventory dataset. I’m trying to come up with a playful type of company and product line upon which to base it. I’m after something whimsical, but meaty enough to build a demo around. I’m thinking at least 400-500 SKUs (products), with a compelling set of product categories (2-3 levels of hierarchy, a few dozen total categories). I’ve thought of things like:

a surf shop chain, with swimming and snorkeling equipment, T-shirts, beach toys and accessories. a “Flintstonesque” shop with all sorts of sticks and rocks something inspired by Wiley Coyote’s “ACME” (bird seed, exploding tennis balls, anvils…) maybe something inspired by Sponge Bob Square Pants (shell emporium….)

Any ideas?

(I realize that this isn’t quite the normal fare here. If it’s not close enough, could you suggest another subreddit?)

submitted by /u/waitak
[link] [comments]

Need Help In A Timeseries Satellite Dataset For A GAN Based Simulation

Hello all,

I am working on an academic project where I am using a GAN to train my synthetic satellite data of a city / vegetation land. I am then changing my labels (air quality, water supply, urbanization parameters etc)to predict what will the new image look like after the feature changes. I am currently working on synthetic satellite data so the results are more or less good. However I want to scale my project to a timeseries data of either a city or a vegetation land so that I can train my model on real time data. Can you point me to the right direction if any such dataset exists ?

submitted by /u/ultrainstinctmasters
[link] [comments]

Help On Finding A Text Summarization Dataset

I’m working on a research idea which can summarize a content for different audiences. For example particular company document summary for marketing, HR or developers which highlight the most relevant content for them. Right now I’m having a difficulty finding a text summarization dataset which has ground truth for different audiences as such. Can anyone point me to the right direction finding this dataset?

submitted by /u/AGENT_SAT
[link] [comments]