Skip to content

v0.1.1 — terminal CLI

Latest

Choose a tag to compare

@pixerwar pixerwar released this 02 Jul 09:12

Dataset Doctor 0.1.1

This release makes Dataset Doctor usable without the web UI — straight from
the terminal and in CI.

Terminal CLI

# Quality/risk report (summary, or --json)
python -m dataset_insight analyze data.csv --category topic

# CI gate: exit 1 when the risk level is at or above the threshold
python -m dataset_insight analyze data.csv --fail-on medium_high

# Convert + export to a training format (optional train/val split)
python -m dataset_insight export data.csv --format chatml -o out.jsonl --split 0.1
  • analyze prints an ASCII summary (risk, diversity, balance, size, and cleaning
    • PII + imbalance signals) or full JSON with --json.
  • --fail-on <low|medium|medium_high|high> exits non-zero so you can gate a
    pipeline on dataset quality.
  • export reuses every export format with an optional stratified split.
  • Structured files auto-detect common instruction/output column names.
  • Runs fully offline; never starts the server or touches the database.

Also in this release

  • Demo abuse protection (opt-in, env-gated) — per-IP rate limit and a dataset
    cap with oldest-first eviction for the public demo.
  • Internal: format→parser maps shared between the API and CLI via a new
    formats.py (no behavior change).

Full report of everything the tool does: see the
README. Changelog:
CHANGELOG.md.

Full Changelog: v0.1.0...v0.1.1