Hello, I have a data science project I’m interested in doing. I want to web-scrape housing data from the Zillow website within a 15-mile radius of a potential career location. I don’t have much experience in web scraping but, I know I need to use selenium (an automated browser) and python’s beautiful soup library to execute this part of my project. Does anyone have experience in web scraping Zillow’s website specifically? Any advice or Youtube videos to help me get started?

​

P.S. I was informed to check to see if Zillow has an API. I checked and it looks like the best I’ll be able to get from an API is using RapidAPI: 40 records of data per GET request with a one-month limit of 20 GET REquest (800 records).

To all my computer vision friends working on real-world applications with messy image data, I just open-sourced a Python library you may find useful!

CleanVision audits any image dataset to automatically detect common issues such as images that are blurry, under/over-exposed, oddly sized, or near duplicates of others. It’s just 3 lines of code to discover what issues lurk in your data before you dive into modeling, and CleanVision can be used for any image dataset — regardless of whether your task is image generation, classification, segmentation, object detection, etc.

from cleanvision.imagelab import Imagelab imagelab = Imagelab(data_path=”path_to_dataset”) imagelab.find_issues() imagelab.report()

As leaders like Andrew Ng and OpenAI have lately repeated: models can only be as good as the data they are trained on. Before diving into modeling, quickly run your images through CleanVision to make sure they are ok — it’s super easy!

Github: https://github.com/cleanlab/cleanvision

Disclaimer: I am affiliated with Cleanlab.

Is there a way to identify the earliest timestamp of news? For example, when Silicon Valley Bank got into trouble, many news websites reported it. I need to find the earliest report of it.

Hey r/datasets,

I originally posted this library earlier this week, but it got downvoted once within 10 minutes and was never heard from again. And I get it, this is a place for posting/requesting datasets.

So, here’s an actual dataset of CA housing data I generated using the RedfinScraper library. Scraping these 47,000 records took just over 3 minutes.

While this data may be useful today, the fact is, it will only be useful for about a week longer. The high-velocity nature of housing data means that datasets need to be updated frequently.

This issue was the driving force for sharing this library publically: to allow users to quickly scrape the latest housing data at their leisure.

I hope you find this library useful, and I am excited to see what you create with it.

I tried using Common-crawl, but it seems to be full of HTML only. It’s hard to extract the CSS.

Hi,

I’m looking for a geojson file which has the polygons for all UK Cities (or just the major ones) for use creating a choropleth/heat map. Any help would be greatfully appreciated!

Thanks in advance!

Hey guys,

I am with some difficult to clear my dataset , originally I had 80 features, after apply some ML rules, for example remove a feature that just have null values etc, I am now with 67 features.

I decided to apply correlation, and I have many features with a strong correlation +0.9 or – 0.9 , I saw that I can remove features with a strong correlation.

But I could not find if there is a rule, or which of this feature should I remove.

For example if I have feature A,B,C,D , and A x B and A x C has a strong correlation should I remove A ? or B and C ?

If someone could kind give some help or some documents about It I will be more than glad.

Thank you.

Hi team,

I am trying to train an ML model to predict social media posts’ engagement metrics based on the copy. To pull it off, I am looking for a training dataset with the following structure:

[Post text] – [Number of Like/Reactions] AND/OR [Number of Retweets] AND/OR [ Number of Comments]

Any leads (paid or free) will be appreciated. Thanks!

RedfinScraper is a scalable Python library that leverages Redfin’s unofficial Stringray API to quickly scrape thousands of housing records.

I built this library to automate the task of collecting housing data, and to do it at a break-neck speed.

Let me know what cool uses you find for the data!

so, im almost done with my data science course. final project, which is supposed to take a month, will be due may 4th.

i was wondering, where does an average joe like me, get his hands on some nuclear fusion datasets? i have no clue what id be doing with it.. but i think nuclear fusion is fascinating, and if i can do something with it, why not.

ive tried google, kaggle and huggingface, couldnt find much.

i know everything is in development right now. its cutting edge technology. pushing the boundary of our knowledge.. and now im wondering, would those datasets be considered top secret?

well, anyway. thanks for reading and any of the help you could provide

Any helps here and thank you 😊

Magic: the Gathering deck dataset

I scraped deck lists from a competitive deck sharing platform called MtgTop8 for a project I’m working on.

Decks are separated by format in the following:

– standard

– modern

– pioneer

– historic

– explorer

– pauper

– legacy

– vintage

​

They’re stored as Apache feather files which can be easily converted to either pickle or csv files.

Feel free to use them for whatever purpose.

Here’s the link