It allows extracting custom user-defined entities from free text. Very exciting!
It can extract exact values (e.g. names, prices, dates), as well as provide ChatGPT-like semantic answers (e.g. text summary).
I like the interactive demo on their website (https://www.textraction.ai/) – it allowed me to try my own texts and entities within minutes. It works great 🙂
The service is accessible also as an API for any purpose via the RapidAPI platform: https://rapidapi.com/textractionai/api/ai-textraction (sign up to RapidAPI and get your own token)

Data from HDMA available on Snowflake:

https://app.snowflake.com/marketplace/listing/GZTSZAS2KG8/cybersyn-inc-us-home-mortgage-disclosure-act-loan-apps?originTab=provider&providerName=Cybersyn%2C%20Inc

Hello Redditors!

I’m excited to share Datalab — a linter for datasets.

I recently published a blog introducing Datalab and an open-source Python implementation that is easy-to-use for all data types (image, text, tabular, audio, etc). For data scientists, I’ve made a quick Jupyter tutorial to run Datalab on your own data.

All of us that have dealt with real-world data know it’s full of various issues like label errors, outliers, (near) duplicates, drift, etc. One line of open-source code datalab.find_issues() automatically detects all of these issues.

In Software 2.0, data is the new code, models are the new compiler, and manually-defined data validation is the new unit test. Datalab combines any ML model with novel data quality algorithms to provide a linter for this Software 2.0 stack that automatically analyzes a dataset for “bugs”. Unlike data validation, which runs checks that you manually define via domain knowledge, Datalab adaptively checks for the issues that most commonly occur in real-world ML datasets without you having to specify their potential form. Whereas traditional dataset checks are based on simple statistics/histograms, Datalab’s checks consider all the pertinent information learned by your trained ML model.

Hope Datalab helps you automatically check your dataset for issues that may negatively impact subsequent modeling — it’s so easy to use you have no excuse not to 😛

Let me know your thoughts!

I am onto a AI project on forgery detection in documents/images being submitted in the banking/insurance industry. I am finding it a little difficult to pick or find the right dataset. It would be great if you share any. Thanks!!!

Having a time series since 2019 would be ideal.

Hey everyone. I was wondering if anyone has a dataset of fake reviews. Please let me know if you find any. I couldn’t get a good one online

My friend and I are working on a small project to make it possible to request any data with one url easily. It’s best used for reference data, test data, education/teaching contexts and training data.

Here is an example of how you can use it to get any data in pandas:

url = “https://slct.ai/us_states_and_populations.csv” df = pd.read_csv(url)

Would love to get the community’s point of view on how they would like to see this evolve and general feedback.

Hi!

I am looking for datasets in which EEG recordings are made while participants are watching videos. I am not interested in specific videos, however I need a dataset in which both EEG recordings and videos employed in the experimental setup are provided. I would like to run some analysis correlating the visual properties of each frame (e.g. brightness) with EEG signals.

I have fond the SEEDs dataset, however the original video are not provided. Does anyone know of any dataset that provides both EEG recordings and videos?

Hey Everyone,

This is a bit of an odd request, but I am looking for help from anyone who works with big data, now or in the past.

I am specifically trying to learn about buying and selling of user data – where is it bought, how much does it cost, who sells it, what is the process like, is it just a giant csv file? etc

Any help is very much appreciated!

I am looking for statistics about tourist arrivals in different countries. I found WorldData.info, World Bank, and Statista, but I am not sure if these sources are reliable and the numbers are accurate. It seems that the data on these websites is inconsistent because they confuse the terms “tourists” (= people traveling for leisure) and “visitors” (= also including people traveling for business). Can anyone help me find a reliable and extensive dataset for tourism arrivals only?

I wnat a dataset on which I can apply ARIMA, Lazy Predict, LSTM, etc and that dataset has somewhat of a real world sense.

Hello 🤗

I want to build a dataset of manipulated documents with the original document and the modified version because I work on a model to localize those forgeries 🧐 The available public datasets that exist are not sufficient but I believe it is possible to build one without resourting to synthetic datasets. In the french gazette website, organizations and funds are required to upload their financial reports every year and they are publicly available. If they make a mistake, the wrong document is left on the website for a while and a rectified document has to be uploaded. Now if the two versions match everywhere pixel to pixel except for a tiny portion, the it has only been modified digitaly and not rescanned. I have been able to find a few pair of documents like that be no nearly enough to train a model. Do you know any websites that work the same way? Where people upload pdfs and these pdfs are sometimes rectified and both versions are still online? Preferably free form pdf and not a specific form like the US gazette.

Thank you for your help!

I’m looking for a data set that links demographic background of a person (e.g. age, gender, education, etc.) to a list of personal interests like hobbies or buying habits (e.g. pets, sports, cars, etc.). The dataset could explain consumer behavior for e.g. marketing analysis or targeted advertising.

​

Is there such a dataset that is “inexpesive” (e.g. 1000 USD one time purchase) or ideally free?

​

The ones I found turned out to be very costly yearly subscriptions.

Thanks a lot for any recommendations and insights!

Hi all!

​

I am not able to find a dataset including eu’s import for critical materials and metals for an “appropriate” period of time, I would say 20-30 years. Do you have any idea?

​

Thanks a lot!!

Hello, I’m currently working on my dissertation and one of the variables I’m planning on using is inmate population. Does anyone have any links to where I can find them?

Thanks!

Hey,

I’ve been mapping out Potentially Underserved Communities in North Carolina over the past few weeks, and have a time-series animation from 2010-2023 at the Census Tract Level, but my professors are wanting me to go further back with the data. It seems like the first American Community Survey 5-Year Estimates came out in 2010, so I think I’ll have to use Decennial Census Data, but was having trouble locating anything prior to 2010 on there website. Any tips?

https://www.kaggle.com/datasets/vinayak121/a-reddit-collectiontitles-from-popular-subreddits

This is my new dataset you can use this dataset for training NLP models

Please Upvote 🙂

Hello /r/Datasets,

I present you a dataset including all the details of all the series (+155k) available on The Movie Database. The dataset is available on Kaggle.

Generation

This dataset was generated in ~10 hours by fetching each ID from The Movie Database API (+225k IDs).

You can generate the same dataset using my NodeJS application available in open-source on GitHub.

Missing data

Some data are missing on some series because The Movie Database API does not provide them. It happens on old TV series not very well known.

Including

id name original name overview tagline in production ? status original language origin country created by first air date last air date number of episodes number of seasons production companies poster path genres vote average vote count popularity

​

I hope I got this post right, I wasn’t sure how to go about it. I also hope this dataset can be useful to you!

Where would I find datasets in regard to this topic? I tried Kaggle and just searched it up. I still can’t find any even though I know it’s out there. I’m new to this lol

When we launched Mockingbird a few weeks ago, the idea was to make it super simple to generate mock data from a schema that you could stream to any destination. When we launched it, you could send mock data streams to Tinybird and Upstash Kafka.

Now, we’ve added support for Ably, AWS SNS, and Confluent.

You can check out the UI here: https://tbrd.co/mock-rd and it’s also available as a CLI with npm install @tinybirdco/mockingbird-cli

Hope this helps when you can’t find the dataset you need!

Hello, as the title states, I am looking for all men under 55 in the state of Va, USA who died on that date.

I’ve tried VA newspapers but most are not online.

Any help would be appreciated.

Hey! I have many Json files that equate to 175 million pieces of data, I’m unsure how to store them in a database, I’ll need to first create one big json file or loop through each or the files and move the data from them into a database.

I’ll need to do querys against the whole dataset multiples times a day so the quicker the better.

I’ve already experimented with mongodb but I just can’t see past the way querys are written.

Any ideas?

I’ve been working on a tokenizer that determines the best possible tokens to represent the test dataset in the least number of tokens for various different vocabulary sizes.

It works well but I’ve been testing with The Pile test data, but it’s mostly English so it’s a not good representation for multi-lingual. It also lacks a fair amount of code and tags.

I need around 1-2GB raw text uncleaned and uncensored, that represents a few different languages and a fair amount of code from different programming languages. Better to be raw, and include data both with HTML tags as it would be when scraped, and also without HTML tags (as it would prioritize the HTML tags too heavily if they were always present).

So just a good representation of general text.

I know I could build my own dataset from various different ones, but it seems to me that a dataset like this should already exist. Any leads would be helpful. Thank you.

Hi. I’m looking for a data set of employee titles at US companies. Not specific companies and not specific people just generalities. Currently have a record set of a million plus job titles of all varieties from cashier to CEO… However looking to pair these up with job level. Wondering if anything like this exists anywhere. I’ve tried the SAMs database but that’s even messier than the records that I have.

Thanks

We are building a hosted embedding marketplace for builders to augment their leaner open-source LLMs with relevant context. This lets you avoid all the infra for finding, cleaning, and indexing public and third-party datasets, while maintaining the accuracy that comes with larger LLMs.

Will be opening up early access soon, if you have any questions be sure to reach out and ask!

Learn more here

Need help

Hey guys, does someone maybe know how I can get a global ODA or Aid-Data dataset. I am curently working on my Bachelors thesis and therefore I would need these Datasets in a form that I could use with Stata.

I tried downloading the ODA-Data and import it to stata but for some reason I didn’t have any obervations. I would need Data which would contain information on the amount of aid delivered to countries in a given year for an analysis of the impact of humanitarian aid on conflict dynamics.

If someone has a tip or could help me, that would be realy nice.