I’ve been building an on-device assistant and, like anyone doing tool-calling SFT, kept shipping subtle junk into my dataset, hallucinated tool names, invented arguments, a tool-call with no result, the model over-calling a calculator for “2+2”, near-duplicate examples quietly collapsing my diversity. So I wrote a validator, and I figured it might save someone else the same headaches.<\/p>\n
It runs entirely in your browser<\/strong> \u2014 you drop in your .jsonl and your tool catalog, and nothing is uploaded anywhere. Your dataset never leaves your machine (that mattered to me, and I assume to some of you too).<\/p>\n What it checks, per example, with a typed verdict (keep \/ KTO-negative \/ discard):<\/p>\n – structure & roles, and every tool_call name\/args against your<\/strong> catalog (required present, no invented args, types)<\/p>\n – tool-flow (each call gets its result, no orphans, ends with a real answer) It spits out clean.jsonl \/ kto-negatives.jsonl \/ rejected.jsonl so you can just use the output. submitted by \/u\/Key-Outcome-2927 <\/a>
– over-calling, verbosity\/filler, prompt-injection defense
– lexical and<\/strong> semantic dedup (the embedding model runs locally too, via WASM)
– diversity (distinct n-grams) + a difficulty breakdown<\/p>\n
It’s free, no account, no catch. Link: nothumanallowed.com\/tools\/dataset-validator<\/strong><\/a>
It’s early and opinionated (the heuristics come from my own pipeline), so if it flags something it shouldn’t, or misses something it should, I’d genuinely like to hear it \u2014 happy to adjust. Hope it’s useful to someone.<\/p>\n<\/div>\n
[link]<\/a><\/span> [comments]<\/a><\/span><\/p>
