Hello, suppose I have built a “user review on products” dataset by scraping from a website.

Now I want to publish the dataset, 1. Do I need to get their consent for publishing it? 2. What if I cant reach out to them to get consent?

If yall could kindly give me solutions to this. Thanks.

Wanting to do a practice project for the agriculture or food industries.

Open to adjacents as well

Thanks!

Hi all, I just released a lot of SEC datasets that you can either access using DropBox or my python package datamule.

Datasets:

Every 10-K & 10-Q since 2001 (~200gb unzipped each, split into archives of ~1gb) Every FTD since 2004 Company Metadata (e.g. sic code, address) Company Former names

If you’re interested in SEC data, I recommend taking a look at the package as it has a lot of nice features & contains information on the data sources. (Also XBRL, etc…)

Links: https://github.com/john-friedman/%20datamule-python, https://www.dropbox.com/scl/fo/byxiish8jmdtj4zitxfjn/AAaiwwuyaYp_zRfFyqfBUS8?rlkey=g1zk5pg7iendbsa34ltnokuxl&st=t7cb6pp5&dl=0

Hi, I need a influencers dataset, raw data of Instagram influencers. Looks easy but I cannot find this API based, every web that has this data it converts it into a web-based search, but I dont need that, I just need the data for my startup. API based would be perfect but also .csv is fine. I need to update it every month.

I need to search by followes, category/ncihe and location (of the influencer or target audience)

Hope somebody can help me…

PD: Also appreciate if you know if I can reach this using some easy Instagram Scraper, not much idea about these.

Are there any leads where I can find it?
Thanks

Couldn’t find dataset on photocatalyst Material ID,Material Name,Synthesis Method,Type of Defect,Defect Engineering Method,Characterization Techniques,Band Gap Energy (eV),Photocatalytic Activity,Applications,References 1,Titanium Dioxide,Nanosheets,Sulfur Doping,Doping,XPS,2.8,85% degradation of dye in 2h,Water purification,“Smith et al., Journal of Catalysis, 2020” 2,Zinc Oxide,Nanoparticles,Oxygen Vacancies,H2 Treatment,PL,3.0,90% degradation of pollutant in 1h,Air purification,“Johnson et al., Applied Surface Science, 2019”

It should be similar to this format.If anyone could help to find the datasets on this …

Hi.

I’m looking for any datasets related to historical box scores, game logs, or season totals for players in the NFL. I was previously using SportsData.io, but I found major inaccuracies with their season-long statistics for players (including some players having impossible records like 0.3 sacks).

I understand a lot of good datasets in this field are locked behind commercial grade API access, but I am curious if anyone here knows of any directions I can explore. Thanks!

I have a project that needs log data from linux machines that contains data about when users change their screen brightness, change their volume level, opens apps and other settings and options like these, I searched online and found some datasets but they were more focused on systemd and kernel logs, so is there a dataset with this kind of data or a way to log such actions?

Guys, for a project I need floor plan with dimensions dataset. If you know any dataset, please attach it here. Thanks in advance!

Thanks

Hello everyone,

I’m currently working on a university project where I need to build a machine learning system from scratch to recognize handwritten digits. The dataset I’m using is derived from the UCI Optical Recognition of Handwritten Digits Data Set but is relatively small—about 2,800 samples with 64 features each, split into two sets.

Constraints:

I must implement the algorithm(s) myself without using existing machine learning libraries for core functionalities. The BASE goal is to surpass the baseline performance of a K-Nearest Neighbors classifier using Euclidean distance, as reported on the UCI website; my goal is to find the best algorithm out there that can deal with this kind of dataset, as I plan on using the results of this coursework for another University’s application. I cannot collect or use additional data beyond what is provided.

What I’m Looking For:

Algorithm Suggestions: Which algorithms perform well on small datasets and can be implemented from scratch? I’m considering SVMs, neural networks, ensemble methods, or advanced KNN techniques. Overfitting Prevention: Best practices for preventing overfitting when working with small datasets. Feature Engineering: Techniques for feature selection or dimensionality reduction that could enhance performance. Distance Metrics: Recommendations for alternative distance metrics or weighting schemes to improve KNN performance. Resources: Any tutorials, papers, or examples that could guide me in implementing these algorithms effectively.

I’m aiming for high performance and would appreciate any insights or advice!

Thank you!

Hello guys

I am looking for dataset which contains videos. And it is better if the dataset is a benchmark dataset (completely optional).

I thought camvid has videos in it, turns out no. They are frame from dashcam.

Hello, I’m a university student and I’m making a machine learning model that will predict how much the population in a city would grow according to its infrastructure.
I have been able to extract and create my own infrastructure dataset with the OSM python library, but I’m having troubles finding and/or creating the population dataset.

I’ve found so far a few datasets with city population, but unfortunatly they only contain data from one or two years, and I would like for it to contain data from at least 5 years.

If anyone knows one, I’d apreciate the help! 😀

I’m working on a project to roughly estimate the ghg impact of flights going in and out of particular u.s. airports. A dataset including the airport symbol and ind’l flights with sources/destinations and aircraft type and airline would be the perfect world. Does anyone know if there is something publicly available like this?

PROEJCT 2 REGRESSION PROJECT GUIDELINES One of the most versatile and powerful tools of econometric analysis is the multiple regression model. This project will give you practical experience in applying multiple regression analysis to a “real-world” problem. You will do the following: 1. Formulate a relationship between some variable of interest (call it Y) and a set of explanatory variables, X1, X2, X3, etc. 2. Gather observations on Y and X1, X2, X3, etc. 3. At least one of the variables should be dummy variable (0/1). 4. At least 30-50 observations (Companies, people, countries, etc., as the case may be), 5. At least 6 variables (pieces of information about the observations; e.g., stock price, revenues, profits, salaries, gender, etc.), 6. Dependent variables can’t be 0/1 variable. It has to be continuous variable. 7. Perform regression analysis on the relationship and possible alternative specifications. 8. Test a number of hypotheses about the relationship. 9. Hold out anywhere between 5 to 7 observations from the building model. 10. Summarize your results, qualifying them and drawing appropriate conclusions.

I. PROPOSAL The topic should have an economic or business emphasis; however, you should feel free to introduce any dimensions or variables that you feel are important in explaining your model. Choose a topic that interests you and about which you have some knowledge. Feel free to speak to any professor from another class (or even me) about a possible topic. The topic must be a clear, analytical topic. You must pose a hypothesis or relationship, gather evidence or data, and come to conclusions about the relationship you have specified. This is not simply a descriptive paper. The paper must be technically challenging; in other words, the conclusion cannot be drawn by a casual look at the data. Choose a topic for which you can find data.

II. FINAL PAPER – OUTLINE 1. Title: The title must be related to the topic of your paper. It is acceptable to phrase your title as a question. Do not call your paper “Multiple Regression …,” since that is a technique, not a topic or problem. 2. Introduction: The introduction provides a concise, descriptive statement introducing the background (nature), objective, and scope of the study. The reason for the study should be explained, such as testing a particular hypothesis. 3. Theoretical Model: State what the hypothesis you are testing. Describe your dependent and independent variables. Explain why you include them and what impact you think they will have on your dependent variable. 4. Empirical Results: From the regression results, present your findings and discuss them. Interpret the results of the regression analysis in a report of no more than one page (per model) using non-technical language. This interpretation should be meaningful to the person who has never had a statistics course. 6. Hold Out Sample: Remove the variables, if you think does not make sense – from p- value or sign perspective. Use the hold put sample to predict the value. Compare with the actual value. How close do you come to actual value? 5. Conclusion: Sum up your results. Mention the key points of your analysis. Are there any implications from your research? (no more than one page) 6. Page Limit: at least 4 but no more than 5 pages Case Evaluation Your case will be evaluated on the following criteria: • Quality of data • Quality of writing; how well do you communicate your approach to the problem and your analysis of results. How well do you express technical issues in ‘plain English?’ • Correctness of analysis and conclusions.

Either my google-fu is failing me or they really do keep this really close to the chest. I was hoping to settle a debate between my friends and I about certain preference settings men use.

Anyone know where or if I would be able to find this?

Looking to get a dataset which includes information such as group names, age, and location, and size of groups if increased/decreased

Hi everyone!

I am doing an academic project, and I am trying to find a dataset or a source to scrape where I could acquire the faces of parents and children of the parents.

This would need to be on a pretty large scale, preferably thousands or tens of thousands of faces (I would use AMT to sort through the images to take out incompatible ones).

Do any of you have an idea of where I might look to find this?

P.S for other projects I am also looking for facial datasets from various regions of countries in the world, especially in Europe and faces of individuals with differing Jewish admixture. This is a bit more complicated, and I would likely need to gather data by survey.

Just found an API for lots of datasets, and it seems you can access them for free!

https://public.opendatasoft.com/

Who knows more about Opendatasoft? What exactly do they do, do they just provide partner with providers to provide APIs for different things?

Also share if you know any other great source of datasets or APIs, preferably that can be accessed for free!

Free data would be amazing, but of course, I assume a credible source would cost. I found a couple of craigslist data – but I am not sure how trustworthy they can be (lots of price = 0 there and prices above trillions).

If I had to pay for the data, who would I contact? KBB?

I know there is pdf version of votes but I dont have time for cleaning it. Is there some dataset or better way how to have the content, ammandment, voters in favour – their name, voters against and absence?

Im using yolov8, given that a camera willl always take 1280×720 resolution, will gathering dataset with same resolution and same camera will result in better performance or not really?

Assuming the model will always be used on that camera

I’m trying to find data sets on gambling that breaks down the type of gambling source age, sex, amount and time spent, maybe region etc. can you point me to the right right direction? Thank you

I’m an intermediate programmer and so far all I’ve been doing for datasets is scraping the internet. But I’m about to start a more advanced project and would love to have a more efficient way to grab data. I’d love to know what yalls specific sources are and any pros and cons you’ve found with them.

I’m dunking for UK biscuit data sets, real-time if possible. Any help in finding them would be rich-tea appreciated. Thanks

Hi,

I was interested in data engineering so do you have any idea on high volume (maybe real-time (maybe daily granularity can also work)) datasets ?

Thanks

Hey, I’m working on an iris recognition project and started with iris segmentation. I used a dataset from Kaggle https://www.kaggle.com/datasets/naureenmohammad/mmu-iris-dataset, but the model’s accuracy was low. I’m using a U-Net for segmentation.

Anyone know of better datasets or ways to improve accuracy? Any suggestions would be great!

Thanks!

Hello, I am looking for datasets of mastitis characteristics within cattle that are free to access/download. I want to basically perform an early diagnosis, and take parameters such as the breed, udder images, milk yield, etc.