feat(pro): graphstore pro CLI subcommand + [pro] extra#182
Merged
KailasMahavarkar merged 2 commits intomainfrom May 2, 2026
Merged
feat(pro): graphstore pro CLI subcommand + [pro] extra#182KailasMahavarkar merged 2 commits intomainfrom
KailasMahavarkar merged 2 commits intomainfrom
Conversation
PR#3 of 5 toward profile="pro". Builds on #177 (gpu module) and #179 (pro.py spec/resolver). Wires the resolver up to a real CLI; setup + probe stubbed pending PR#3.5 probe runner. Pieces: 1. **`graphstore pro {check, setup, probe, status}` subcommand** in src/graphstore/cli.py with per-slot overrides (`--embedder`, `--reranker`, `--ingest-mode`, `--bonsai-quant`, `--bonsai-skill`, `--vision`, `--audio`, `--ner`), `--cache-dir`, and `--json`. - `check` / `status`: full implementation. Captures live host, loads calibration cache, runs `pro.resolve()`, prints either a pretty text breakdown (Host / Spec / Resolved / Shortfalls / Suggestions / Warnings / Calibration) or JSON. - `setup` / `probe`: stubs. Print a manual-fallback message pointing at `graphstore install-embedder` + the `CalibrationCache` Python API, with a clear "PR#3.5 will ship the probe runner" line so users know it's coming. Exit 2 so scripts can branch on it. - Exit codes: 0 fits, 1 doesn't fit (resolver shortfall), 2 not implemented or extras missing, 3 calibration missing. 2. **`[pro]` extra in pyproject.toml** wraps existing extras (`ingest`, `vision`, `audio`, `embedders-extra`, `gpu`) plus `tokenizers` + `onnxruntime` + `huggingface-hub` (deps shared across slot probes). No new pip dependencies introduced; this is purely a convenience aggregate so users can `pip install 'graphstore[pro]'` instead of stringing extras. 3. **`tests/test_cli_pro.py`** (10 tests): top-level + check `--help` lists every slot; empty-cache `pro check` returns exit 3 with `calibration missing` message and parseable JSON; slot overrides reflected back in JSON; host block surfaces live RAM / CPU / GPU; setup/probe stubs return exit 2 with PR#3.5 pointer in both text and JSON modes. Test plan: pytest tests/test_cli_pro.py -q -> 10 passed pytest --tb=short -q --ignore=tests/test_server.py --ignore=tests/test_e2e_real_embedder.py -> 1912 passed, 102 skipped (was 1900; +10 new + 2 from #179) Live smoke against an empty cache on this host: $ graphstore pro check --cache-dir /tmp/empty-cache Host CPU 8 physical / 16 logical cores RAM 31114 MB total, 19593 MB available GPU NVIDIA GeForce RTX 3060 (12288 MB total, 11411 MB free) Spec embedder jina-v5-small [...] Resolved fits NO Shortfalls - calibration missing for: embedder:jina-v5-small, [...] $ echo $? 3 Next: PR#3.5 (probe runner: per-component download + measure RAM/disk/ TPS into CalibrationEntry); PR#4 (GraphStore(profile="pro") + lazy BonsaiIngestor); PR#5 (docs).
…te test CI install matrix is `pip install -e .[dev,playground,ingest]` which gives onnxruntime + tokenizers but NOT llama-cpp-python (lives in [vision]). Default ProSpec uses ingest_mode="bonsai" → required_dists() includes llama-cpp-python → check_extras_installed() raises before the resolver sees the empty cache, so tests asserting exit 3 + the "spec" field landed exit 2 + an extras-not-installed payload instead. Fix: tests that target calibration behaviour now pass an explicit `--ingest-mode deterministic --embedder model2vec-256d --vision none --audio none` quartet whose required_dists are guaranteed satisfied by the [dev] extra alone. New TestProCheckExtrasGate locks in the extras-missing path explicitly: spec with embedder=fastembed-bge-small (needs [embedders-extra]) → exit 2 + structured error + missing_dists list. Test tolerates either path (extras-missing → 2, or calibration-missing → 3) so it works on CI matrices with either install profile. Local pytest tests/test_cli_pro.py -q -> 11 passed.
KailasMahavarkar
added a commit
that referenced
this pull request
May 2, 2026
…#183) PR#3.5 of 5 toward profile="pro". Builds on #182 (CLI). Replaces the `pro setup`/`pro probe` stubs with a real probe runner that downloads each selected component and measures live RAM/disk/TPS into the calibration cache the resolver already reads. Architecture in `src/graphstore/pro_probe.py`: - Probe base class declares download() + measure() + the shared run() orchestrator that handles error isolation + timing. - One Probe subclass per slot value: Model2VecProbe (embedder:model2vec-256d) JinaOnnxEmbedderProbe(slot, install_name, dims) × jina-v5-small / jina-v5-nano / embeddinggemma-300m FastembedProbe (embedder:fastembed-bge-small) JinaV3RerankerProbe(reranker:jina-v3) - GGUF via huggingface_hub BonsaiProbe(quant, skill) × tq1_0/tq2_0 × lite/full = 4 entries - measures CPU RSS+TPS, plus VRAM+TPS when host.gpu_ready WhisperProbe(model_size) × tiny/base/small VisionSidecarProbe(model) - placeholder; full RSS/VRAM via sidecar subprocess deferred to PR#3.6 TinyBERTNERProbe (ner:tinybert) - _REGISTRY maps every component_id ProSpec.component_ids() can emit to a factory; list_probable() lets tooling enumerate. - probe_components(component_ids, host, cache_dir, on_event) is the orchestrator: sequential, atomic cache write per success, per-probe error isolation, optional progress callback. - skip_probe=True (for `pro setup --skip-probe`, future flag) records a download_only=True entry without loading the model. Hard rules preserved: - Sequential to avoid one probe's RSS poisoning the next measurement. - Atomic cache writes between probes so partial progress survives a mid-suite crash. - Per-component error isolation: download or measure raising → ProbeFailure recorded, suite continues. - No hard-coded numbers. All RAM/TPS/disk values measured live on this host. Knob tier hints (n_ctx_min/default/max, n_batch_at_default, embed_batch_at_default, reranker_max_at_default) populated into CalibrationEntry.extra so the resolver upgrades knobs against actual measurement, never against an assumed threshold. CLI wiring (`src/graphstore/cli.py`): - cmd_pro now calls pro_probe.probe_components for setup/probe. - Streams probe_start / probe_done / probe_failed events to stdout in text mode, collects them for --json mode. - Same extras-installed gate as pro check (exit 2 with structured error when [pro] deps missing). - Exit 0 on all_ok, 1 on any failure. Test plan: pytest tests/test_pro_probe.py tests/test_cli_pro.py -q -> 24 passed pytest --tb=short -q --ignore=tests/test_server.py --ignore=tests/test_e2e_real_embedder.py -> 1926 passed, 102 skipped (was 1912; +14 new + others) Coverage: - registry: every ProSpec slot combo's component_ids() ⊆ _REGISTRY (regression guard - new slot values without probes silently break the resolver into "calibration missing" forever) - orchestrator error isolation: failed download/measure → ProbeResult failure, rest of suite continues, cache contains only successes - cache durability: partial-progress survival across re-invocation - on_event callback: probe_start / probe_done / probe_failed ordering - skip_probe flag: download() called, measure() not, cache marks download_only=True - measurement helpers: process_rss, vram_free graceful 0 fallback, tps median across iters Live smoke on this host (model2vec): $ python -c "from graphstore import pro_probe; ..." probe_start: embedder:model2vec-256d probe_done: embedder:model2vec-256d 1.16s OK: ram_idle=63MB ram_default=63MB tps=162486 Next: PR#4 GraphStore(profile="pro") integration + lazy BonsaiIngestor; PR#5 docs.
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR#3 of 5 toward
profile="pro". Builds on #177 (gpu module) + #179 (pro.py resolver). Wires the resolver to a real CLI; setup+probe stubbed pending PR#3.5 probe runner.What's in this PR
graphstore pro {check, setup, probe, status}subcommand with per-slot overrides +--cache-dir+--jsoncheck/statusfully implemented (host capture → resolve → pretty/JSON output)setup/probestubs that print PR#3.5 pointer + manual-fallback steps; exit code 2[pro]extra in pyproject wrapping existing extras (no new deps)Exit codes
[pro]extras missingTest plan
Live smoke (empty cache):
Next
GraphStore(profile="pro")integration + lazy BonsaiIngestorwebsite/docs/guides/pro-mode.md)