Skip to content

Add CI smoke test for CLI launch + --dry-run flag + fix isOnPath isolation#200

Merged
Alan-Jowett merged 4 commits intomainfrom
copilot/add-ci-smoke-test-cli-launch
Apr 7, 2026
Merged

Add CI smoke test for CLI launch + --dry-run flag + fix isOnPath isolation#200
Alan-Jowett merged 4 commits intomainfrom
copilot/add-ci-smoke-test-cli-launch

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 7, 2026

v0.6.0 shipped a broken CLI (shell: true in spawn() split the bootstrap prompt into multiple args) with no CI coverage of the launch path. This adds that coverage and the infrastructure to keep it green.

Core fixes

  • isOnPath() rewrite — replaced execFileSync("which", ...) with direct fs.accessSync() iteration over PATH entries. Eliminates the dependency on which/where being on PATH (broke test isolation), uses F_OK on Windows where X_OK is meaningless.
  • Pre-existing test failures fixedTC-CLI-070/TC-CLI-072 were silently broken because runDetectCli() set PATH=mockDir only, leaving which unreachable. Now works correctly since isOnPath needs no shell utilities.

New: --dry-run flag

$ promptkit interactive --cli copilot --dry-run
[DRY RUN] Would spawn:
  cmd:  copilot
  args: ["--add-dir","/tmp/promptkit-xxx","-i","Read and execute /tmp/promptkit-xxx/bootstrap.md"]
  cwd:  /home/user/project

Prints the exact spawn invocation and exits 0 — no LLM CLI required. Used directly in CI smoke steps.

New tests

TC What it catches
TC-CLI-084 shell: true absent from non-comment lines in launch.js — direct regression guard for v0.6.0 bug
TC-CLI-085 --dry-run exits 0, args JSON parses to a single bootstrap.md element (verifies no shell-splitting), path is absolute

CI workflow

  • Added cli/** to path triggers on both push and pull_request.
  • Added permissions: contents: read at workflow level.
  • New cli-smoke-test job: npm ci && npm test + explicit --dry-run invocations for copilot and claude.

Copilot AI linked an issue Apr 7, 2026 that may be closed by this pull request
Copilot AI and others added 2 commits April 7, 2026 01:59
…isolation

Agent-Logs-Url: https://github.com/microsoft/PromptKit/sessions/4a201d67-41ca-45a4-8698-27f187f2ce41

Co-authored-by: Alan-Jowett <20480683+Alan-Jowett@users.noreply.github.com>
…heck, JSON args parsing in tests, workflow permissions

Agent-Logs-Url: https://github.com/microsoft/PromptKit/sessions/4a201d67-41ca-45a4-8698-27f187f2ce41

Co-authored-by: Alan-Jowett <20480683+Alan-Jowett@users.noreply.github.com>
Copilot AI changed the title [WIP] Add CI smoke test for CLI launch Add CI smoke test for CLI launch + --dry-run flag + fix isOnPath isolation Apr 7, 2026
Copilot AI requested a review from Alan-Jowett April 7, 2026 02:03
@Alan-Jowett Alan-Jowett marked this pull request as ready for review April 7, 2026 02:11
Copilot AI review requested due to automatic review settings April 7, 2026 02:11
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds CI coverage for the CLI launch path and introduces a --dry-run mode to validate spawn invocation formatting without requiring a real LLM CLI, while also improving PATH detection isolation by removing reliance on which/where.

Changes:

  • Reworked CLI PATH detection to scan PATH directories directly (no external which/where dependency).
  • Added --dry-run flag to print the exact spawn command/args and exit without launching.
  • Added regression tests + a GitHub Actions smoke-test job to keep the CLI launch path covered in CI.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 5 comments.

File Description
cli/lib/launch.js Implements PATH scanning for CLI detection and adds dryRun option to launchInteractive().
cli/bin/cli.js Exposes --dry-run option and passes it into launchInteractive().
cli/tests/launch.test.js Adds regression tests for shell: true absence and validates --dry-run output/args structure.
.github/workflows/validate-manifest.yml Triggers workflow on cli/** changes and adds a CLI smoke-test job (Node 20).

1. isOnPath: filter empty PATH segments to prevent false positives
   from checking CWD when PATH has empty entries (leading/trailing
   delimiters or unset PATH).

2. Dry-run cleanup: add force: true to rmSync for reliable cleanup
   on Windows where read-only files would cause failures.

3. TC-CLI-084: use whitespace-tolerant regex (/shell\s*:\s*true/)
   instead of exact string match to catch formatting variants.

4. TC-CLI-085: use mkdtempSync for unique temp dirs with cleanup
   in finally block. Fix bootstrap path extraction to strip known
   prefix instead of splitting on last space (handles paths with
   spaces).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Alan-Jowett Alan-Jowett merged commit 880e1de into main Apr 7, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add CI smoke test for CLI launch

3 participants