Add Claude Code skills for the example pipelines#402
Merged
Conversation
Project skill under .claude/skills/setup-e2e-template covering setup, run, training and troubleshooting of the e2e_template pipeline on your own data, plus an optional tags-addon recipe for per-scenario metrics. README documents how it is auto-discovered and used.
Varfalamei
force-pushed
the
add-e2e-template-claude-skill
branch
from
58f01ba to
db97c6a
Compare
Adds project skills setup-embedder-fiftyone and setup-sam-cvat alongside the e2e_template skill, and generalizes the README section to list all three.
Varfalamei
changed the title
…ames, concise per-example pointers
…d, fix S3/demo knobs master's config separates input ($DATAPIPE_E2E_DIR/images) from the working dir ($DATAPIPE_E2E_DIR/datapipe), so the working-dir-overlap and dataset-grows notes no longer apply; rewrote the 'where your images go' knobs to the env vars master actually ships, corrected the seed-demo description, and dropped the node-specific offline/pre-AVX2 note (polars-lts-cpu is pinned for old CPUs and the seed already retries per image).
…cs-0 note and stale train-race advice DETECTION_MODEL_CONFIG ([16,16]) is the pre-annotation fallback model, not the training config (training is yolov8n imgsz 320), so metrics~0 on the seed set is a data-size issue, not that config. master also ships resume_config that auto-retries failed/raced training attempts, so the manual re-run note is stale.
…t a fixed problem
Varfalamei
requested review from
bobokvsky and
korotaS
and removed request for
korotaS
korotaS
reviewed
Jul 1, 2026
|
|
||
| This skill = run the DINO→FiftyOne embedder on YOUR images. The FiftyOne-zoo dataset (caltech101) is just a smoke-test; the real goal is your own images — set the knobs below first. | ||
|
|
||
| **Before starting, if not already provided, ask the user:** demo (zoo) or your own data? external Postgres up or provision it? GPU? — ask only what's unresolved, then do only what's needed. |
Contributor
There was a problem hiding this comment.
я бы здесь добавил что-то типа "задай максимум вопросов", потому что мне при прогоне не хватило как минимум нескольких штук:
- он сам вписал в .env путь к локальной постгре (которая слава богу была пустая), так что надо бы явно спрашивать "какую именно базу в постгре используем", а не только "используем ли локальную или поднимает докер". также можно в явном виде прописать отдельный шаг "подготовить .env", чтобы он его подготовил, потом ты вручную проверил, а потом уже запускать дальше
- он все шаги трубы типа
create-allиrunпрогонял без логов, типаdatapipe run 2>&1 | tail -60, а лучше пусть спросит, нужны ли логи или нет - можно еще спросить про uv, вдруг есть уже готовое окружение, чтобы не ставить новое. кстати тут он сам понял, что с текущим toml на мак не встанет, поэтому пофиксил его - это прикольно
плюс не хватило немного отчетности - чтобы он сначала, как все прочитал, сказал "план такой, выполняем?" и на каждом этапе говорил побольше, типа "я сделал вот это и вот это". сейчас он сам сразу стартанул и все сделал, но довольно мало говорил
но вообще это все мелочи, потому что пишешь чисто /setup-embedder-fiftyone и происходит магия)))
…irst with an .env checkpoint From real-run feedback: ask which Postgres/database to use (don't silently write a localhost default or target an existing DB), whether to reuse an existing venv, and whether to surface stage logs; propose a plan before acting, prepare .env and pause for the user to verify it, and report what changed after each stage instead of running the pipeline silently.
…ebug-sql into a file + grep Replaces the tail-60 guidance (which hid progress and still cost tokens): run stages with their normal INFO logs shown, and only when a failure's cause is unclear re-run with datapipe --debug (or --debug-sql) written to a file and grepped, instead of dumping the verbose debug firehose into context.
Documents how to recreate the tag demo end-to-end: seed a non-trivial cat/dog set, build a darkened tagged scenario with inherited GT, train a baseline with the scenario excluded, retrain with it included, compute tag_metrics, and compare on the val holdout. Notes evaluating on final-epoch weights.
Router that points to the right example + setup skill and lists the universal prerequisites (Postgres, GPU, annotation backend, uv env, HF auth). Carries the same ask-first / plan-first / logs / --debug guidance as the setup skills.
…emo recipe The 10-image smoke default is too small — the pipeline's val metric is noisy and its built-in metrics/checkpoint selection misbehave. Default --detection-limit to 120, at which a clean from-scratch run trains, computes metrics natively, and shows the tag arc (baseline low on the tagged scenario, higher after the tag is added to training). Pin the demo-scenario counts in tags-addon.md (120 base, ~40 night 30/10, gamma 0.10) and soften the checkpoint-selection note now that native metrics work.
… ready-made GT Skipping the human annotation step (injecting COCO/inherited ground truth) vs a real Label Studio pass is a demo-shaping choice, so surface it in the Ask-first block and flag it in the tag demo recipe rather than deciding silently.
Add the concrete recipe to surface tags as a graph node in datapipe-app: catalog entries (tag/image__tag/tag_metrics) + an aggregation step, and note it MUST be a DatatableBatchTransform (whole-table SQL group-by), not a chunked BatchTransform, since it aggregates many images into one (model,tag,subset) row. These pipeline edits stay docs-only / applied physically for the demo, not committed to the example.
…ation For unattended/demo runs without a human annotator: write image__ground_truth + image__subset directly, but following the pipeline conventions — image_name as list_s3_images emits it (basename relative to INPUT_IMAGES_DIR, no invented prefix), labels matching DETECTION_CLASSES casing (Cat/Dog not cat/dog), via DataStore so _meta stays consistent. Captures the two mismatches that silently break freeze/metrics.
…UI -> train x2 -> 2 models + metrics)
… pipeline schema AND public observability run history separately)
…ercase classes; drop tags-addon New self-contained example examples/detection_tags: YOLO detection with tags built into the pipeline (tag/image__tag/tag_metrics + a compute_tag_metrics BatchTransform), NO Label Studio and NO FiftyOne, ground truth injected (COCO labels, lowercase cat/dog), postgres+minio only, deps pin polars-lts-cpu + pi-heif so it runs from scratch. Two-step data load (base batch, tagged scenario batch) via scripts/load_batch.py. New setup-detection-tags skill + router row. e2e_template LABEL_CONFIG lowercased (cat/dog) to match COCO; the old tags-addon.md recipe is removed in favour of the real example.
…oad BatchTransform) Replace the standalone load_batch.py with a load stage in the pipeline: a load_request table (request_id/n/offset/tag/darken) feeds a load BatchTransform that downloads COCO cat/dog, uploads to MinIO and emits s3_images + ground truth (+ tag/image__tag). Loading a batch = add a request row (scripts/add_request.py) then datapipe step --labels=stage=load run — so ingestion is datapipe-native and shows in the graph. Two requests (base, tagged night) drive the two-model demo.
… skip COCO on slow networks)
…-lts-cpu (pin alone insufficient)
…0 for faster demo iteration
…otal), gamma 0.25, epochs 10; add pi-heif to e2e deps
compute_tag_metrics already summed per-image TP/FP/FN to derive precision/recall/f1 but did not return them; surface them as calc__TP/FP/FN and add the columns to the tag_metrics table. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…problem->fix flow) Reorder the demo so the low night/val tag metric is shown on the baseline model first (metric stages run on model A over the new night batch, no training) and framed as the problem, then retraining closes it. Note the inference->count-metrics->tag-metrics ordering and that night/val is small so the rise is directional. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fixed val (correct A/B metric comparison): - add_request.py --subset pins a batch to train/val; load_batch emits image__subset_hint; split honors it (random split only fills un-pinned images). Load base-val + night-val up front so val is frozen; add night-train only for the retrain. FiftyOne (ported from e2e_template/image_detection): - stage=fiftyone: download_images + publish GT and per-model predictions (predictions_model_a=baseline, predictions_model_b=retrained; slot by sorted model id); tag/subset as sample fields. mongo added to docker-compose; datapipe-ml[fiftyone]+fiftyone dep. Tag dimension: numeric tag_id (deterministic via config.TAG_IDS) + tag_name + tag_description; image__tag/tag_metrics key on tag_id, join tag for readable name. Bug fixes: - darken=NaN: a NULL numeric column arrives as NaN when several requests load in one batch; pd.isna guards so base batches are not darkened with gamma=NaN into garbage. - download_images: don't pass S3 storage options to fsspec.open of a public HTTP URL. Defaults/robustness: DB_SCHEMA defaults to public; standard ports. SKILL.md: two-part checkpoint flow (demo-only), fixed-val rationale, pre-flight port/schema checks, always-print-full-table rule, service-watch guidance, troubleshooting (polars-lts on pre-AVX2, count-metrics ordering, double-train, port/tunnel/proxy pitfalls); snapshot/rehearse as inline pg_dump/psql (no committed demo scripts). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move all COCO/demo-only code (constants, download+cache, gamma darkening) into a dedicated detection/coco_demo.py (CocoDemoSource + darken). steps.load_batch is now source-agnostic plumbing: it pulls (bytes, boxes, labels) from the source and lays them into the pipeline tables. For real data you replace one file (coco_demo.py); steps.py/app.py are untouched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…nto add-e2e-template-claude-skill # Conflicts: # .claude/skills/setup-detection-tags/SKILL.md
bobokvsky
approved these changes
Jul 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds Claude Code skills as project skills under
.claude/skills/for the example pipelines, plus a README section on how they are added and used:setup-e2e-template—examples/e2e_template(YOLO detection / keypoints + Label Studio → train → FiftyOne). Includes an optionaltags-addon.mdrecipe for per-scenario tag metrics (tag images → part flows into training → a separatetag_metricstable; old vs new model on the tagged subset).setup-embedder-fiftyone—examples/embedder_fiftyone(DINOv2/DINOv3 embeddings → FiftyOne UMAP + similarity search).setup-sam-cvat—examples/sam_cvat(SAM3 text-prompt boxes + masks → CVAT pre-annotations).Why
Lowers the barrier to running each example on real data first-try: external prerequisites, env knobs, and data-alignment gotchas are baked in. Being committed project skills, they are auto-discovered by Claude Code with zero install for anyone who clones the repo.
Notes
The
datapipe-examplesrouter skill (picks the right example + universal prerequisites) is not included here and can follow separately.