Skip to content

v0.1.0 — first public release

Choose a tag to compare

View all tags
@pixerwar pixerwar released this 01 Jul 10:54
v0.1.0
1b822b8

Dataset Doctor 0.1.0 — first public release

A health check for your fine-tuning datasets. Upload a file, get a quality/risk
report, and fix the problems before you train — all locally, your data never
leaves your machine.

🔗 Try the live demo — no install, no signup: https://dataset-doctor-production.up.railway.app
(the Upload screen ships three sample datasets so you can see a full report in one click)

What it does

  • 7 input formats — CSV, JSONL/JSON, XLSX, TXT, Markdown, PDF, DOCX. Combine
    several files into one dataset.
  • Quality & risk report — diversity, category balance, size adequacy, a
    training-time estimate, and a composite risk level with plain-language reasons.
  • Semantic map — a 2D projection of your samples (TF-IDF or offline model2vec
    embeddings, no torch) so you can see clusters and outliers.
  • Clean step, one click per fix:
    • Near-duplicate detection + quality lint
    • PII / secret scan (emails, phones, Luhn-checked cards, IPs, API keys) — redact
      in place or drop the samples, fully offline
    • Local class-imbalance fix — downsample the dominant category, no synthetic data
  • Export to ChatML / OpenAI / Alpaca / ShareGPT / prompt-completion, with an
    optional stratified train/val split.
  • Persistent & self-hostable — SQLite-backed, restart-safe, single-Dockerfile deploy.

Run it yourself

git clone https://github.com/pixerwar/datasetdoctor
cd datasetdoctor
python -m venv .venv && .venv/Scripts/python.exe -m pip install -r requirements.txt
.venv/Scripts/python.exe -m uvicorn dataset_insight.api.main:app --reload
# then, in frontend/: npm install && npm run dev

See the README for the full
guide and the CHANGELOG
for details.

The hosted demo is a shared, unauthenticated, ephemeral instance — don't upload
real sensitive data there. Self-host for the privacy guarantee.

Assets 2
Loading