pgw#609: greenfield test system — hub-double harness, P1-P10 suite, legacy deletion (166→40 files)#359
Merged
Merged
Conversation
…idency reconcile, slot binding precedence, CAS multiwriter, discovery lock, cancel/ backpressure, boot smoke) Harness extracted per design: hub_double.py generalizes FakeScheduler from test_worker_grpc_e2e.py (the only double, at the true hub gRPC boundary); blob_host.py is a real blake3-verified HTTP blob server; toy_endpoints.py consolidates toy fixtures (echo/stream/GPU-slot-yield/Hub-bound/Slot-bound/ billable); subprocess_runner.py generalizes the gw#591 boot-smoke pattern. P3 absorbs PR #353 (pgw#606/th#938) and PR #307 (pgw#583) as real hub-double dispatches (no mocking of gen_worker internals, unlike the original unit tests) - revert-verified red against lifecycle.py's pre-fix state, restored clean. P1/P2/P4/P5/P6/P7 absorb assertions from test_worker_grpc_e2e.py, test_shared_cas_root_multiwriter.py, and test_boot_smoke_gw591.py. Found + fixed in the harness itself: WorkerHarness must set the TENSORHUB_CACHE_DIR env var (gen_worker.models.cache_paths reads the process-wide get_settings() singleton, not the per-worker Settings instance) - passing tensorhub_cache_dir= to load_settings() alone silently no-ops, and every hub-double test defaults to a fresh per-call temp dir so downloaded bytes never leak between tests. 40 passed, 1 skipped (pgw#605 heartbeat: proto fields not generated yet) in ~33s local. Existing test files not touched or deleted (Phase 3).
…load metrics), plus ruff cleanup + a hub_double(file_base_url=) knob P9 needs P8 (tests/convert/test_p8_convert_publish_contract.py, hermetic fake_hub): dtype-mismatch cast honesty (pgw#589/th#901, condensed from test_publish_as_is_dtype.py), classifier corpus table (pgw#593), and an xfail(strict=True) test-first pin for pgw#566 (open bug: normalize_adapter _state_dict only passes unet_config to converters with a NAMED parameter - diffusers' real SDXL lora_state_dict takes **kwargs only, so the SGM block remap silently never runs). pgw#569's W8A8 verifier ULP gate is a documented skip - no factored-out comparator to test hermetically without a real w8a8 artifact round trip. P9 (tests/test_p9_result_upload_metrics.py): inline vs blob_ref by size alone over a real hub-double + real local media-upload dedup sink (no S3 part scripting, matching test_media_upload_owner.py's pattern); JobMetrics typed usage survives both wire forms. Needed hub_double(file_base_url=) so the sink URL rides the FIRST HelloAck instead of a racy post-connect mutate. ruff clean (unused imports, one E731 lambda->named helper). mypy: src/gen_worker clean (untouched); new tests/harness + test files carry the same untyped- fixture-param convention as the rest of tests/ (mypy only gates src/gen_worker per CI, tests/ was never in scope). Full new suite (P1-P9 + P8's own file): 50 passed, 1 skipped (pgw#605, proto fields not generated), 1 xfailed (pgw#566, fix open) in ~31s local.
test_slot_boot_precedence_th938.py, test_model_slot_identity_gate.py -> superseded by test_p3_slot_binding_precedence.py (same assertions, over a real hub-double boundary instead of monkeypatched Executor.store internals). test_shared_cas_root_multiwriter.py -> superseded by test_p4_cas_multiwriter_integrity.py (same 3 tests reproduced verbatim + 1 new row). tests/convert/test_publish_as_is_dtype.py -> superseded by tests/convert/test_p8_convert_publish_contract.py (same 3 core tests against the same fake_hub). Deletions only, no other changes. Full deletion manifest (all 166 files audited, reasoning for what's NOT deleted) is in the th#960/pgw#609 tracker checkpoint - the bulk of the pre-existing suite tests production surface (CLI, LoRA runtime, quantization lanes, VRAM/RAM admission, media/upload, Slot/discovery SDK, tests/convert producer suite) that no P-test covers, so it stays pending explicit direction on the larger absorb-or-accept-loss effort that would be needed to delete it safely.
Manual dispatch + PR-into-master only (drafts/doc-only paths run nothing); drop the on-push trigger. Same gates (mypy gating, ruff non-gating per debt note, HTTP-timeout guard, llama-server, uv build); pytest targets the whole tests/ dir so P-suite additions land automatically.
…3/P5/ P6/P8/P9 + 2 new consolidated files, ahead of the bounded aggressive deletion sweep Bucket-level triage per the coordinator's ruling (Paul's directive: minimal greenfield set, ~92% removal). Absorbed only distinct bug-classes carrying a real pgw#/gw# incident number or a revert-catches-it story; untagged "tested thoroughly once" coverage goes without absorption (git history is the archive). P1 (+6): auth-rejection exit/retry-within-window, permanent-precondition fast-exit, hello-ack-deadline reconnect, not-leader redirect, hub-restart reconnect - absorbed from test_worker_grpc_e2e.py (#372) ahead of its deletion. P2 (+3): setup-failure FnUnavailable+recovery, host-RAM-failure wire ordering (th#807) from test_worker_grpc_e2e.py; corrupt-load quarantine+ retry-once (gw#408, J17 flood) from test_snapshot_corruption.py. P3 (+1): bare-ref prefetch binding resolution (#377) from test_executor_prefetch_binding.py. P5 (+1): heavy-dep stub contract (pgw#506) from test_discovery_heavy_deps.py. P6 (+1): SendQueue capacity-generation fencing from test_worker_grpc_e2e.py. P8 (+5): diffusers root-allinone skip (e2e J7 ENOSPC), standalone-component canonical-weight selection (gw#426), transformers sharded-index onnx/pickle exclusion, gguf explicit-quant refusal, non-safetensors-format refusal family - from test_classifier.py ahead of its deletion. P9 (+1): token-bound-owner routing (J19 run34) from test_media_upload_owner.py. New consolidated files (coordinator-authorized, one per lane with no P-test home): tests/test_quantization_lanes.py (W8A8/W4A4/FP8 contract + gw#409 promote-device-integrity, from test_w8a8.py/test_w4a4.py/ test_fp8_and_emergency_loading.py/test_promote_device_integrity.py) and tests/convert/test_convert_producer_contract.py (clone concurrency gw#442, disk preflight gw#462, download-skip bank th#592, publish resilience gw#462, from the tests/convert/ producer suite). Also reverted a premature local-only Phase 4 CI commit (c39efb6, from a prior session before an interruption) - never pushed, out of THIS issue's authorized scope ("do not touch CI yet"); preserved in history via revert, not discarded. Full new-suite gates: mypy clean, ruff clean, each new/changed file green in isolation. Deletion commit (the ~125 files these rows supersede) follows as a separate, deletions-only commit per the coordinator's mechanics.
…8 LoC) superseded or absorbed per the coordinator's ruling Every file here is either fully superseded by test_quantization_lanes.py / test_convert_producer_contract.py, absorbed as rows into P1/P2/P3/P5/P6/P8/ P9 (chaos b8274a1), or carries no incident pin (git history is the archive, per the coordinator's bound). Full manifest, bucket-by-bucket reasoning, and the dangling-import sweep are in the th#960/pgw#609 tracker Checkpoint 6, pushed before this commit. Deletions only, no other changes. Foreign-WIP clusters untouched: compile_cache/executor_adopt/fleet_cells (this issue's own claim-checkpoint note) and test_disk_telemetry_pgw610.py (landed on chaos by another lane during this session). Final state: 14 test files (3,293 LoC) + tests/harness/ (6 files) + conftest.py x2 + fake_hub.py, target was ~12-15 files per the coordinator.
PaulFidika
marked this pull request as ready for review
July 21, 2026 07:29
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Promotes pgw#609 (th#960 Phase 2, gen-worker half of the greenfield test system) from
chaostomastervia a release branch, perWORKSPACE-GIT-POLICY.md's promotion pattern.tests/harness/(hub_double gRPC server, blake3 blob host, toy endpoint packages, subprocess runner) + 9 P-tests (P1-P9; P10 stands as the pre-existingtest_llama_runtime.py, per design).test_quantization_lanes.py,convert/test_convert_producer_contract.py) — the coordinator's one-per-lane allowance for W8A8/W4A4/FP8 and clone/convert-producer coverage that had no P-test home.workflow_dispatch+pull_request-into-masteronly, drafts/doc-only-paths skip,pytest tests/targets the whole directory so future test files are picked up automatically.Test-file count on this branch: 152 → 40 tracked
tests/**/*.pyfiles (master's current baseline differs slightly from chaos's own 166→40 snapshot, sincemasterhad already absorbed some intervening work — pgw#610 disk telemetry, gw#608 residual fixes — that chaos's count predates).Excluded / deviations from chaos
5a628e8(revert of the Phase-4 CI draft) was never picked — cherry-picking3ea5a50(the CI rewire itself) directly makes that revert moot, per the coordinator's note.chaosentirely — not part of this issue's scope.test_executor_adopt.py,test_compile_cache.py,test_fleet_cells.py, and 13 siblings — the compile_cache/executor/fleet_cells lane) is untouched, exactly as pgw#609 left it.Gates (this branch, local)
uv run pytest tests/ -q: 426 passed / 5 skipped / 1 xfailed (~45-60s). One flaky failure mode observed in ~1 of 4 runs, isolated entirely totest_executor_adopt.py(the untouched foreign-WIP cluster) — a pre-existing cross-test torch-inductor-cache-dir latch issue, not introduced by this promotion; reproduces independently of any file this issue touched.uv run mypy src/gen_worker: clean (128 source files, 0 issues).uv run ruff check tests/ src/gen_worker: clean.Follow-ups (not this PR, tracked in pgw#609)
Closes/tracks pgw#609.
🤖 Generated with Claude Code