Category: Datatards

Here you can observe the biggest nerds in the world in their natural habitat, longing for data sets. Not that it isn’t interesting, i’m interested. Maybe they know where the chix are. But what do they need it for? World domination?

What Percentage Of Bisexuals Are Trump Supporters ?

Are bisexuals statistically the least likely demographic of the LGBTQ+ to support Donald Trump and Republicans?

In my personal non statistical experience the lesbians I know all support Trump. The only lesbians I know of that don’t support Trump are Ellen Degeneres and JK Rowling. Most gay men I mean support Trump. Most trans people I meet in real life support Trump. Most gay men and trans people I see against Trump are on Reddit or content creators sites like Youtube.

But almost every bisexual I meet in real life is staunchly anti trump but there is no bisexual community like there is for gays, lesbians, trans etc so there are no bisexual content creators.

Are there any research studies done on LGBTQ trump supporting rates that can give us real numbers?

submitted by /u/lostacoshermanos
[link] [comments]

Malicious And Safe URL Dataset For ML

This dataset contains a mix of malicious and safe URLs, verified using sources like PhishTank and VirusTotal, making it ideal for training Machine Learning models. If you don’t have access to their APIs or are seeking a reliable and relevant URL dataset for ML, this is for you. This dataset will be updated daily. Cheers!

submitted by /u/TendouNoSaibA
[link] [comments]

Any Data Sets On Workers Unions Over Time?

I’m looking for data on Worker’s Unions. Number of strikes, numbers of unions, numbers of union members, numbers of contracts signed, numbers of bridge agreement/interim extension.

I’d really love to see data on union busting as well and maybe contract improvements, but I imagine those things are difficult to quantify?

I also imagine there are posts concerning this already, but I’ve already searched for ‘union’, ‘labor union’, and ‘workers union’ and haven’t come up with anything, so if there’s verbiage that I’m missing out on, feel free to chastise me for not searching so long as you tell me the terms I should have been using.

Thanks!

submitted by /u/inkblot888
[link] [comments]

What Medical Dataset Is Public For ML Research

i was trying to apply machine learning algorithm, clustering, on medical dataset to experiment if useful info comes out, but can’t find good ones.

Those in UCI repository have few rows like 300~ patient records, while many real medical papers that used ML used dataset of thousands patient records.

what medical datasets are publicly avail for ML research like this?

ps. If using dataset of 300~ patient records will be justifiable, plz also advise

submitted by /u/qmffngkdnsem
[link] [comments]

Looking For A Dataset For All London Restaurants

So I’m currently looking for a list of all restaurants in London, ideally with their M addresses.

I’ve been able to scrape a huge restaurant promotion site in the UK and pull around 7000 restaurants with this info however I’m sure I’m missing a large number of restaurants as I’m unable to find my favourite restaurants in the list.

Would anyone be able to point me in the right direction as to where I may be able to find a list like this?

submitted by /u/giveguys
[link] [comments]

Anyone Knows What Technology / Solution Was Used To Generate The Microsoft Security Incident Prediction Dataset?

So i am working on building a ML model to automate the classification of SOC environment alerts to identify the true positive ones & the false positives. The model is ready, however to be able to further test on new data, i will be needing to generate alerts similar to those that were in the training data. So if anyone has any idea what SIEM solution or EDR was used to generate these alerts, please let me know.

Microsoft Security Incident Prediction Dataset : https://www.kaggle.com/datasets/Microsoft/microsoft-security-incident-prediction?resource=download

Also are there any solutions that generate alerts with these features (OrgId, IncidentId, DetectorId, AlertId, AlertTitle, Category, Day, Id, Hour & EntityType)??

submitted by /u/Syn1ho
[link] [comments]

Any Way To Get A Set Of Seedless And Seedful Tangerine Photos?

I’m a software engineer, not super proficient in ML yet, so forgive me if my question is unrealistic.

Anyway, I want to create an app that detects whether there are seeds in a tangerine from a photo. Seedless tangerines slightly differ from seedful ones, so I believe this is somehow possible to implement. Since there is no pre-trained model for this, I’m ready to create my own, but gathering thousands of photos is an impossible mission task for me. How are tasks like this usually tackled?

submitted by /u/RoastPopatoes
[link] [comments]

Looking For A Dataset That Is Complex Enough To Do Big Data Analysis Relative To Mental Health/depression

Hello, I am in a big data class. My group is interested in doing our final project based on mental health/depression. Although, ‘big data’ will not be feasible because we are running these on our local PCs, we still need to perform big data analysis with map/reduce programs. We have been using PySpark for all of our assignments and they have been very complex assignments. Such as a friend recommendation program where you rank 10 recommendations from a very large text file that was in the format of <unique_id><list of friends>. This assignment, we had to perform multiple for loops/if statements inside of our PySpark map/reduce program which made it quite complex.

Now, we have found this dataset https://www.kaggle.com/datasets/anthonytherrien/depression-dataset that we want to use, but we don’t believe we can “wow” the professor with complex enough functions to make conclusions. Is this maybe not a good type of dataset for big data applications? We originally thought to make a depression “score” based on the given features and justify those based on how frequent/similar each unique person is.

Any ideas or datasets that you know about that would be just complex enough would be a big help. Thanks!

submitted by /u/idkwhatsgoingon4582
[link] [comments]

How To Handle Missing Values In A Dataset?

I am working on a diabetes prediction model for my project and I need help on how should I handle missing values in the smoking history column in my structured tabular dataset.

My dataset has 100,000 rows, with around 35% of rows having “No Info” for smoking history. Since smoking history has a significant impact on diabetes, this column cannot be ignored.

Other entries in this column are: “Never”, “Current”, “Not current” and “Former”

Key concerns:

Encoding: If I am encoding this column, then how should “No Info” be treated in this case? One hot encoding will lead to unneccessary high dimensionality whereas there is no clear order that I can choose between the values if I go with ordinal encoding.

Data Loss: Would dropping these rows (35%) lead to bias, or is it a valid approach?

I would appreciate your personal insights on the best approach for this since I have already searched this thing enough on the internet.

submitted by /u/shaitaanbaluck
[link] [comments]

LinkedIn Simple Dataset For Homework (how To Get?)

Hi, my teacher gave us an assignment, we need to get – how many active users by country -gender and age distributions -average users daily time on the app -percentage of the global population that uses the app. All of that in an excel or CSV. Many of my classmates had to do it with instagram, tik ton, etc. In my case it was LinkedIn, the thing is I tried to find the dataset the, only thing I could found was a statista report that I couldn’t even download. I need to put it in PowerBi so I don’t need a massive amount of data. But from what I searched in this subreddit LinkedIn API is private or I need to pay for money I don’t have.

Am not really sure on what to do, that’s why I am asking in this subreddit, where should I searched, I don’t wanna take the easy route but I spent a lot of time searching and found nothing, if there wasn’t much then u rather speak to my teacher about it. Any help would be appreciated it

submitted by /u/Jproxy122
[link] [comments]

Question For Improving Custom Floating Trash Dataset For Object Detection Model

I have a dataset of 10k images for an object detection model designed to detect and predict floating trash. This model will be deployed in marine environments, such as lakes, oceans, etc. I am trying to upgrade my dataset by gathering images from different sources and datasets. I’m wondering if adding images of trash, like plastic and glass, from non-marine environments (such as land-based or non-floating images) will affect my model’s precision. Since the model will primarily be used on a boat in water, could this introduce any potential problems? Any suggestions or tips would be greatly appreciated.

submitted by /u/Fit-Information6080
[link] [comments]

Looking For Dataset Of The Racial Wage Gap By Country

As part of a research paper, I’m currently trying to find data on the racial wage gap by country. Preferably the data will be from the at least the mid 2010’s to at least 2022, but I’d love to see anything someone can find. I’ve been looking all over the internet for it and haven’t come up with anything. Thank you!

submitted by /u/avancini12
[link] [comments]

Dataset With 10k-50k Products With Many Attributes

I am doing a master thesis on how large language models compare to other tools when extracting structured data from natural language. Essentially my goal is to translate something like this:

“I want Asus laptops with relatively good reviews, at least 16 GB RAM, ideally 16 inch screen. Sort all the results by price and reviews”

into something like this:

{

“brand”: “Asus”,

“category”: “Electronics”,

“subcategory”: “Laptops”,

“sort”: [“price”, “review”],

“filters”: [

{

“attribute”: “ram”,

“condition”: “greater_than_or_equal”,

“value”: “16 GB”,

“is_hard_condition”: true

},

{

“attribute”: “screen_size”,

“condition”: “equal”,

“value”: “16 inch”,

“is_hard_condition”: false

},

{

“attribute”: “review_rating”,

“condition”: “greater_than_or_equal”,

“value”: “4”,

“is_hard_condition”: true

}

]

}

using large language models, and analyze how they compare to more traditional tools.

What I need is a dataset that has many products, and each product has at least a category (though subcategories would be ideal), branch, and many attributes which are dynamic, depending on product. For example laptop would have CPU, RAM, screen size and so on, while sofas would have very different attributes. It can be even smaller in size (1k-10k). Is there a dataset for this?

submitted by /u/PurpleYellowLeaf
[link] [comments]

Where Can I Find Top Websites By Traffic, Per Year.

I’m developing a game where players explore the internet through different eras, and I need data on the most popular websites over time. Ideally, I’m looking for a list of the top 100 most visited websites for each year over the past 20 years or so. The data doesn’t need to be all that accurate because the actual rankings will not affect the game, I just need a list of popular websites. Thanks in advance!

submitted by /u/jimmakoulis
[link] [comments]

Where Or How Can I Find E-commerce Datasets

Where can I find dataset to do product analysis? Something that will allow me to time based pricing trends (like best time to buy maybe black Friday sales) or competition between retailers (a product sold on Amazon vs Best Buy or Walmart).

I have visited almost every data platform I know and I can’t find anything that’s good. I feel like web scraping might be the only option.. but I’m new to it and it would take a lot of time.

Any suggestion/idea/resources is appreciated!

submitted by /u/_anomaly_0
[link] [comments]

Downloaded Large Image Dataset That Is Not Organized And Simply #s As Names.

Hey I hope this is a good place to ask.

I downloaded a large image dataset from google/bing/Baidu, unfortunately all the filenames are generic and have no identifying Metadata.

Is there a program/software ideally free/open source if not cheap you recommend that can scan and reverse google image a directory of 100k+ photos download and fill in Metadata.

I especially would like to embed/rename photos to include the people in it, group the photos together for instance 10 photos belong to the same shoot/background with slightly different variations but they are all mixed in and impossible to separate/organize manually.

I appreciate any suggestions!

submitted by /u/ifnbutsarecandynnuts
[link] [comments]

I Need Advice For My Portfolio And Job Search

I am new to data analysis. I have a portfolio with a couple projects I did using excel, powerBI, and mysql. I also collected my own data on kaggle for the MCU revenues project.

I do not have a degree or any professional experience to put on my resume so it’s hard to get a second glance.

Do you know of any companies that might hire a person like me? Or maybe free ways to get experience on my resume? And maybe any tips to spruce up my projects? Or any other tools that would be good to learn?

I am trying freelance but having no luck and fiver charges you and so does upwork after you run out of credits.

submitted by /u/FunkYourself55
[link] [comments]

Looking For A Dataset Of All PhDs In A Country

Hello everyone! I’m currently looking for a dataset of all PhDs defended in a country (preferably in Europe but if you have other examples, I’d love to hear from it too) and going back to at least the 2010s. Ideally, I would need something similar to the French theses.fr open dataset (doc in French here), with a field for the research area of the thesis and the list of PhD advisors and members of the defense jury.

Does someone know a dataset answering these criteria? As far as I understand it, the German dataset does not contain the members of the jury and the British Library lost a lot of data in a hack last year and does not resolve EThOS links for now.

submitted by /u/Unfair_Resident_5951
[link] [comments]

Finding A Dataset Of DSA/CP Problems

Working on an NLP based ML model that extracts key technical terms from raw DSA/CP statements.

The goal is to preprocess problem descriptions, identify relevant entities, and summarise them concisely.

Looking for any open source datasets that fit these requirements

submitted by /u/droffense
[link] [comments]