feat(smoke-test): promote validation harness into the repo (Phase 5)#54
Open
TechAlchemistX wants to merge 1 commit intomainfrom
Open
feat(smoke-test): promote validation harness into the repo (Phase 5)#54TechAlchemistX wants to merge 1 commit intomainfrom
TechAlchemistX wants to merge 1 commit intomainfrom
Conversation
Phase 5 — extract the live-backend smoke harness that has lived in
/tmp/secretenv-test/ since v0.2.0 and turn it into a checked-in,
parameterized, reproducible piece of the repo at scripts/smoke-test/.
What's new:
- lib/common.sh — shared bootstrap (env-driven cloud IDs, BIN auto-detect
via git rev-parse, fixture seeding with @@RUNTIME_DIR@@/@@GCP_PROJECT@@
/@@AZURE_VAULT@@ sentinel substitution, idempotent local-secrets
git-init for Section 17 history tests).
- fixtures/ — checked-in templates for local-secrets, local-registry,
config.toml, project-repo. Cloud identifiers fall back to safe
placeholders (placeholder-project / placeholder-vault) so --local-only
works without any cloud env.
- provision.sh / teardown.sh — env-parameterized. No more user-specific
GCP project / Azure vault hardcoding. Required env: SECRETENV_TEST_
GCP_PROJECT, SECRETENV_TEST_AZURE_VAULT.
- run-tests.sh — gains three filter flags:
--sections N,M,P-Q run a chosen subset (range syntax supported)
--local-only shortcut for sections that need no cloud CLI
--list-sections print the inventory and exit
Selective gating is implemented via a SECTION_ACTIVE flag that
short-circuits run_test/assert_contains/record. The full matrix
remains the default; only it gates a release tag.
- README.md — setup walkthrough, section inventory, troubleshooting.
Cleanup along the way:
- Replaced the user-specific cargo-test path
(/Users/mandeep/Workspace/secretenv/secretenv) with $REPO_ROOT,
derived from `git rev-parse --show-toplevel` inside common.sh.
- All hardcoded /tmp/secretenv-test/ paths replaced with ${RUNTIME_DIR}.
- Section heredoc TOML configs use the same variable.
Validation: full live matrix runs 336/336 PASS (same count as v0.4
Phases 1-3 closeout). --local-only runs 53/53 PASS with no cloud env.
--sections 1,12 returns 6/6 in <2s.
Aggregate-release posture: this lands on main but the v0.4.0 tag still
requires Phase 7 (closing reviewer audit + secretenv-testing publishing
decision (decided: defer) + release.yml batch update). Not adding the
release.yml smoke gate in this PR — keeping scope tight.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 5 of v0.4 — extract the live-backend smoke harness that's lived in
/tmp/secretenv-test/since v0.2.0 and check it in atscripts/smoke-test/. Now reproducible by any contributor with the right cloud auth.What's new
lib/common.sh— env-driven bootstrap (cloud IDs, BIN auto-detect viagit rev-parse, fixture seeding with sentinel substitution, idempotent local-secrets git-init for Section 17 history tests).fixtures/— checked-in templates forlocal-secrets,local-registry,config.toml,project-repo. Cloud identifiers fall back to safe placeholders so--local-onlyworks without any cloud env.provision.sh/teardown.sh— env-parameterized; no more user-specific GCP project / Azure vault hardcoding.run-tests.sh— three filter flags:--sections N,M,P-Q— run a chosen subset (range syntax supported)--local-only— shortcut for sections that need no cloud CLI--list-sections— print inventory and exitSelective gating is implemented via a
SECTION_ACTIVEflag that short-circuitsrun_test/assert_contains/record. The full matrix remains the default; only it gates a release tag.README.md— setup walkthrough, section inventory, troubleshooting.Cleanup
$REPO_ROOT(derived fromgit rev-parse --show-toplevel)./tmp/secretenv-test/literals replaced with${RUNTIME_DIR}(env-overridable).Validation
--local-only--sections 1,12Out of scope
release.ymlto call this harness as a pre-tag gate — separate PR (Phase 5 follow-up or part of Phase 7).release.ymlanddeny.tomlAGPL tightening — separate small PRs.Test plan
bash -n scripts/smoke-test/{lib/common.sh,provision.sh,teardown.sh,run-tests.sh}parses cleanscripts/smoke-test/run-tests.sh --list-sectionsprints the 20-section tablescripts/smoke-test/run-tests.sh --helpprints the usage bannerscripts/smoke-test/run-tests.sh --local-onlyruns 53/53 PASS with no cloud env🤖 Generated with Claude Code