Spent the last few weeks building scrapers for the major AI tools directories. If everyone’s gonna over-hype this slop, the data should be useful.

What I scraped:

• Futurepedia: 1,302 tools
• TAAFT (There’s An AI For That): 6,248 tools
• TopAI: 1,880 tools
• MCP Server Directory: 10,614 servers

20,044 entries total. Clean CSVs with categories, pricing, ratings, links, whatever each site had.

Disclosure: this is paid data.

Doing anything with AI tools data? Building something? Just want to poke around? DM me.

I am building a tool to help with crosswords which would require chambers (nearly 3 times the words of most dictionaries and necessary for such puzzles) and contains definitions (unlike SCOWL).

Anyone know where to find any format of it that is machine readable?

Hi everyone,

I’m working on a small project that tries to read grocery receipts and automatically categorize the items (milk → dairy, apples → produce, etc).

The surprisingly hard part is that every store prints receipts differently. Walmart, Tesco, Costco, Aldi, and others all have their own formats, abbreviations, tax layouts, loyalty sections, and discount lines.

To make the parser reliable, I need a few real examples of receipts from different stores.

If you happen to have a receipt from one of these stores, it would help a lot if you could share one.

Examples of stores I’m currently looking for include:

US: Walmart, Kroger, Costco, Whole Foods, Target, Publix, Trader Joe’s, Aldi

Canada: Loblaws / No Frills, Costco, Sobeys, Walmart

UK: Tesco, Sainsbury’s, Asda, Aldi, Lidl

Australia: Woolworths, Coles

Singapore: FairPrice / NTUC

Switzerland: Migros, Coop

Japan: Aeon / MaxValu, Ito-Yokado

South Korea: E-Mart, Homeplus

What works best:

• a quick photo of the receipt

• a scanned receipt

• a digital/email receipt

You can blur or crop anything personal like card numbers or addresses. The only parts I really need are:

• the store name/header

• item lines

• prices

• tax/discount sections

Even one receipt helps because each retailer has its own format.

If you’re willing to help, you can:

• post an image here

• DM me

• share an Imgur / Google Drive link

I’d really appreciate it. And once the parser is in good shape, I’m happy to share the dataset and parsing rules with the community as well.

Thanks for helping a nerdy little project learn how to read grocery receipts 🙂

I am looking for a moderate to large dataset containing retail customer order data, some sort of customer demographic data, product details and reviews if possible. I know there’s probably not some single dataset that contains all these at the same place so any suggestions on what datasets i can combine or what to look for is also welcome. I had already seen the posts in this sub regarding this and asked chatgpt for help but what it came up with was vague to say the least. I just want a some suggestions on how to proceed on the dataset aspect for my project on retail consumer behaviour analysis that i want to do where i want to analyse and find out how external factors such as trends, weather, media perceptions, etc., contribute to consumer behaviour and sales patterns.

Any suggestions are welcome. Again TIA.

I’m working on a dataset toolchain aimed at LLM fine-tuning datasets, because I noticed most dataset failures aren’t “model problems”—they’re data problems: duplicates, leakage, unclear labels, inconsistent formatting, or missing documentation.

What the tool enforces

• Schema validation: every record must match a strict schema (fields, allowed labels, structure)
• Split integrity: supports splitting by topic/template-family so train/test don’t leak via shared scaffolding
• Dedupe + repetition control: catches exact and near-duplicates; flags templated collapse
• QC reports: acceptance rate, failure breakdown, and example-level rejection reasons

What I’m trying to get right (and want feedback on)

• What metadata is a must-have for you? (license, lineage, schema, label definitions, known limitations)
• Do you prefer datasets shipped as clean-only, or raw + clean + reproducible pipeline?
• How do you want near-duplicate removal described so you trust it didn’t delete useful diversity?

If people are interested, I can share a dataset-card template + QC report structure that’s been working well (no links unless allowed).

We started out using a mix of lowcode scraping tools and browser extensions to supply data for our AI models. That worked well during our proof-of-concept, but now that we’re scaling up, the differences between sources and frequent schema changes are creating big problems down the line.

Our engineers are now spending more time fixing broken pipelines than working with the data itself. We’re considering custom web data extraction, but handling all the maintenance in-house looks overwhelming. Has anyone here fully handed this off to a managed partner like Forage AI or Brightdata?

I’d really like to know how you managed the switch and whether outsourcing your data operations actually freed up your engineers’ time.

I’ve been fingerprinting what’s been running on the internet since September, right down to the patch version too. Just chucked a slice of what I’ve found on GitHub.

The schema for the dataset is available in the README file. It’s all JSON files, so you’d be able to easily dig through it using just about any programming language on the planet.

If you find something real cool from this data let me know, I want to see what you can do.

Hi everyone,

I’m a computer science student currently working on a project called 𝐒𝐢𝐠𝐧𝐁𝐫𝐢𝐝𝐠𝐞, an AI-powered accessible learning platform designed to improve classroom communication for hearing-impaired students.

The main goal of the project is to build a 𝐥𝐢𝐠𝐡𝐭𝐰𝐞𝐢𝐠𝐡𝐭 𝐬𝐢𝐠𝐧 𝐥𝐚𝐧𝐠𝐮𝐚𝐠𝐞 𝐫𝐞𝐜𝐨𝐠𝐧𝐢𝐭𝐢𝐨𝐧 𝐬𝐲𝐬𝐭𝐞𝐦 𝐭𝐡𝐚𝐭 𝐜𝐚𝐧 𝐫𝐮𝐧 𝐨𝐧 𝐥𝐨𝐰-𝐜𝐨𝐬𝐭 𝐝𝐞𝐯𝐢𝐜𝐞𝐬 (𝐧𝐨𝐫𝐦𝐚𝐥 𝐥𝐚𝐩𝐭𝐨𝐩𝐬 𝐰𝐢𝐭𝐡𝐨𝐮𝐭 𝐆𝐏𝐔𝐬) so that it could realistically be deployed in schools.

Current approach:

– MediaPipe Holistic for hand + pose landmark extraction

– Landmark normalization

– Random Forest classifier for sign prediction

– FastAPI backend + React frontend

– Real-time webcam input

The system currently supports 𝐛𝐚𝐬𝐢𝐜 𝐰𝐨𝐫𝐝-𝐥𝐞𝐯𝐞𝐥 𝐬𝐢𝐠𝐧 𝐝𝐞𝐭𝐞𝐜𝐭𝐢𝐨𝐧 and includes a 𝐜𝐥𝐚𝐬𝐬𝐫𝐨𝐨𝐦 𝐦𝐨𝐝𝐞 𝐟𝐨𝐫 𝐛𝐢𝐝𝐢𝐫𝐞𝐜𝐭𝐢𝐨𝐧𝐚𝐥 𝐜𝐨𝐦𝐦𝐮𝐧𝐢𝐜𝐚𝐭𝐢𝐨𝐧

– Student signs → converted to text

– Teacher speech → converted to live captions

Right now the biggest limitation is 𝐝𝐚𝐭𝐚𝐬𝐞𝐭 𝐬𝐢𝐳𝐞. I only have a small set of labeled sign images/videos, which makes it difficult to expand vocabulary or experiment with temporal models.

I’m looking for advice on a few things:

1. 𝐃𝐚𝐭𝐚𝐬𝐞𝐭𝐬 𝐟𝐨𝐫 𝐈𝐧𝐝𝐢𝐚𝐧 𝐒𝐢𝐠𝐧 𝐋𝐚𝐧𝐠𝐮𝐚𝐠𝐞 (𝐈𝐒𝐋) or similar landmark-based sign datasets.
2. Best ways to 𝐜𝐨𝐥𝐥𝐞𝐜𝐭 𝐚 𝐬𝐦𝐚𝐥𝐥 𝐛𝐮𝐭 𝐮𝐬𝐞𝐟𝐮𝐥 𝐝𝐚𝐭𝐚𝐬𝐞𝐭 for word-level or classroom-related signs.
3. Suggestions for improving the model while keeping it 𝐥𝐢𝐠𝐡𝐭𝐰𝐞𝐢𝐠𝐡𝐭 𝐞𝐧𝐨𝐮𝐠𝐡 𝐭𝐨 𝐫𝐮𝐧 𝐨𝐧 𝐂𝐏𝐔 𝐝𝐞𝐯𝐢𝐜𝐞𝐬.
4. Any feedback on the system design or architecture.

Eventually I’d like to extend it toward 𝐬𝐞𝐪𝐮𝐞𝐧𝐭𝐢𝐚𝐥 𝐰𝐨𝐫𝐝 𝐝𝐞𝐭𝐞𝐜𝐭𝐢𝐨𝐧 𝐨𝐫 𝐬𝐢𝐦𝐩𝐥𝐞 𝐬𝐞𝐧𝐭𝐞𝐧𝐜𝐞-𝐥𝐞𝐯𝐞𝐥 𝐢𝐧𝐭𝐞𝐫𝐚𝐜𝐭𝐢𝐨𝐧, but still keep it deployable on low-resource hardware. Currently this is done by the react side like when users sign it stores the sequence of words.

If anyone has worked on sign language recognition, accessibility tools, or dataset collection, I’d really appreciate your suggestions.

Thanks

• I’m packaging a dataset for ML training and want to do this “properly.”
• What fields make you trust a dataset fast? (license, data lineage, schema, label definitions, splits, leakage checks, etc.)
• Any examples of dataset cards/docs you consider “gold standard”? (Keep it discussion + best practices; avoid sales. r/datasets discourages low-effort requests and prefers original sources.)

Hey everyone.
I recently needed to use Dr. Duke’s Phytochemical database for a project, but the raw CSV dumps from the USDA are an absolute nightmare to parse (missing fields, inconsistent naming, random caps lock everywhere).

I spent the last couple of days completely cleaning, normalizing, and mapping the dataset into a relational JSON structure so it’s actually usable for data science pipelines.

I put a sample of 400 fully mapped chemical/plant entities on GitHub if anyone else needs this for their research. Saved me a ton of headache.
[https://github.com/wirthal1990-tech/USDA-Phytochemical-Database-JSON]

For ~38 days, a cronjob generated daily forecasts:

•⁠ ⁠10-day horizons

•⁠ ⁠~30 predictions/day (different stocks across multiple sectors)

•⁠ ⁠Fixed prompt and parameters

Each run logs:

•⁠ ⁠Predicted price

•⁠ ⁠Natural-language rationale

•⁠ ⁠Sentiment

•⁠ ⁠Self-reported confidence

Because the runs were captured live, this dataset is time-locked and can’t be recreated retroactively.

### Platform

I built a simple MVP to explore the data interactively:

https://glassballai.com

https://glassballai.com/results

You can browse and crawl all recorded runs here

https://glassballai.com/dashboard

### Goal

This is not a trading system or financial advice.

The goal is to study how LLMs behave over time under uncertainty:

forecast stability, narrative drift, confidence calibration, and prompt-conditioned bias.

### Dataset

After ~1.5 months, I’m publishing the full dataset on Hugging Face.

It includes forecasts, rationales, sentiment, and confidence.

(Actual prices are rehydratable due to licensing.)

https://huggingface.co/datasets/louidev/glassballai

###Stats:

Stocks with most trend matches: ADBE (29/38), ISRG (28/39), LULU (28/39)

Stocks with most trend misses: AMGN (31/38), TXN (28/38), PEP (28/39)

Feedback and critique welcome.

For ~38 days, a cronjob generated daily forecasts:

•⁠ ⁠30 stocks across different sectors

•⁠ ⁠10-day horizons

•⁠ ⁠~30 predictions/day (different stocks across multiple sectors)

•⁠ ⁠Fixed prompt and parameters

Each run logs:

•⁠ ⁠Predicted price

•⁠ ⁠Natural-language rationale

•⁠ ⁠Sentiment

•⁠ ⁠Self-reported confidence

Because the runs were captured live, this dataset is time-locked and can’t be recreated retroactively.

### Platform

I built a simple MVP to explore the data interactively:

https://glassballai.com

https://glassballai.com/results

You can browse and crawl all recorded runs here

https://glassballai.com/dashboard

.

### Goal

This is not a trading system or financial advice.

The goal is to study how LLMs behave over time under uncertainty:

forecast stability, narrative drift, confidence calibration, and prompt-conditioned bias.

### Dataset

After ~1.5 months, I’m publishing the full dataset on Hugging Face.

It includes forecasts, rationales, sentiment, and confidence.

(Actual prices are rehydratable due to licensing.)

https://huggingface.co/datasets/louidev/glassballai

###Stats:

Stocks with most trend matches: ADBE (29/38), ISRG (28/39), LULU (28/39)

Stocks with most trend misses: AMGN (31/38), TXN (28/38), PEP (28/39)

Feedback and critique welcome.

Hi guys I am currently doing my Academic Research Paper, I would like to ask for help where I can get data sets for AI Generated Human Face (image or video is fine) which is Open Source, and Paid? Thank you guys, hope you guys have time to help me currently having a hard time to find datasets. I currently looked up in huggingface and Github.

If you’re building a web crawler and need a large seed list, this might help.

I extracted ~72M unique domains from the latest Common Crawl snapshot and published them here:

https://github.com/digitalcortex/72m-domains-dataset/

Use it to bootstrap your crawling queue instead of starting from scratch.

I got some old books, receipts, invoices, posters etc like the stuff you cant find on the internet in different languages and I planned to make those to a digital asset like cvs or json file maybe ecxel too but I have a doubt that is even make a dime without a company. In summary Can I make money (as a one dude) in online sites with enough of those old documents? If the answer is yes where? Thank you for your help in advance

Studies show that many traders lose their gains not because of poor strategies, but due to unnoticed behavioral patterns.

While most traders focus on macro and microeconomic indicators, technical analysis, and stock tips, losses often stem from panic selling, overtrading, and behavioral biases that go unnoticed during trading.

To address this, I built a solution that helps identify whether these behavioral traits are affecting your performance — and pinpoints the root cause

It’s completely free and requires no signup.

Hello guys do you know where I can get pictures atleast 300 pictures of pig with ASF I can picture it myself but pigs with asf are quickly disposed of so it’s hard for me to take a pictures. Thank you

Hi everyone,

I’m looking for ideas and direction from experienced folks for a uni project built on open data. The goal is to create a public-facing service that doesn’t really exist yet (or is clearly missing), and deliver a realistic prototype within a student timeline.

If you have experience in civic tech / open data projects and can help orient me, I’d really appreciate:

• ideas for high-impact problems worth tackling, • suggestions on datasets that are actually workable, • and how you would validate impact (basic metrics / evaluation). 

I’m open to many domains (mobility, environment, public spending, health, education, safety, etc.), as long as it’s powered by open data and results in a useful public service (search, comparison, alerts, maps, dashboards, scoring, etc.).

Thanks for any guidance!

How do we Modify such a large scale Criteria with each has a Json, a level of Refinement that there won’t be copyright related issues… It is definately AI but how do we do like 180k or more. Itenerating each..

Built a dataset of 10,000+ real screenshots and code of modern websites with details of styling, framework used, and box bounds for all viewports (Desktop, mobile, tablet).

I fine-tuned QWEN 2.5 VL-7B-Instruct with this dataset and ran it on DesignBench (An LLM Web UI benchmark), and the model showed improvements in the pixel similarity score of generated websites!

I have a temporal sequence dataset but it is scattered to many small groups of dataset. How to manage the dataset by keeping the temporal sequence?

Here is my case: Let’s say I have a total of 100 dataset frames scattered to 4 groups with the same size. Each group is a temporal sequence but in different time, not continues. 2 set of groups is used for train, 1 set for validation, and 1 set for test. Is it fine for my NN to learn from this dataset? What is the drawback from the 100 frames continues temporal frames with the usual 80% train, 10% 10% val-test split?

I’m planning to conduct research on banks in Asia, but I’m struggling to find reliable data sources beyond standard financial indicators (e.g., assets, liabilities, equity). Could anyone advise where I can obtain or purchase datasets for metrics such as FinTech adoption/digital maturity and ESG performance, especially for less-covered markets like Vietnam?

Spent the weekend refining AhamData’s analysis models with feedback from colleagues (special thanks to @Zayne). Even with test data, the new features are worth exploring:

– Data Quality Report (0–100 integrity scoring)

– Descriptive Statistics Engine (auto column classification)

– Transparent Methodology Panel (see PCA, Shapley, regression, clustering in real time)

– Sample Datasets Catalog (25 curated datasets across domains)

www.ahamdata.com

Still a bit slow with larger datasets — more backend work is needed to scale performance and avoid applying sampling logic when size exceeds model capabilities. Feedback & feature requests welcome!