Skip to content

Pre‐Release Manual Testing

Jump to bottom
eunolabs edited this page Jun 14, 2026 · 3 revisions

Pre-Release Manual Testing

A pre-release checklist for validating an eunox-mcp release candidate on macOS, Windows, and Linux. Enforcement behavior is identical across platforms — what differs is the delivery surface (install channel, archive format, signature/checksum verification, first-launch OS guard, and file paths), so the shared steps are written once and per-OS variants are called out inline.

.goreleaser.yml builds every OS for both amd64 and arm64 — test each arch on matching hardware if you ship both.

What differs per platform

macOS Windows Linux
Install channel Homebrew cask (stable tags only) winget (stable tags only) .deb / .rpm / tarball
Archive .tar.gz .zip (eunox-mcp.exe) .tar.gz
Checksum tool shasum -a 256 Get-FileHash sha256sum -c (native)
First-launch guard Gatekeeper quarantine Mark-of-the-Web / SmartScreen none
Audit dir (~/.eunox) ~/.eunox %USERPROFILE%\.eunox ~/.eunox
Key-mode check stat -f '%Sp' → -rw------- icacls (ACL, not POSIX) stat -c '%a' → 600

The released binaries for macOS and Windows are unsigned (per .goreleaser.yml), so users hit Gatekeeper / SmartScreen on first launch; Linux has no equivalent. The first-launch step below documents the unblock for each.

0. Prep

# macOS (Homebrew):
brew install go jq cosign gh
# Windows (winget); commands below are PowerShell:
winget install --id sigstore.cosign; winget install --id GitHub.cli; winget install --id jqlang.jq
# Linux (apt/dnf); install cosign + gh per their upstream instructions:
sudo apt-get install -y jq      # or: sudo dnf install -y jq

go (>= 1.25) is only needed for the source build in Step 1; node is for the npx filesystem-server smoke test in Step 5.

1. Source sanity gate (catches gross breakage)

# macOS / Linux:
cd eunox
make check-fmt && make check-license && make check-notice
make test          # go test -race -count=1 ./...
make build         # -> bin/eunox-mcp
./bin/eunox-mcp version    # a source build prints "dev" -- expected here
# Windows (no make; use the raw go commands):
cd eunox
go test -race -count=1 ./...
go build -o bin\eunox-mcp.exe .\cmd\eunox-mcp
.\bin\eunox-mcp.exe version    # prints "dev" -- expected for a local build

The dev string is correct only for a local build. On the released binary the version must be the real tag.

2. Released-artifact verification (the core pre-release check)

Download the candidate's artifacts + signature chain (replace <tag>, e.g. v0.1.0-rc1). Swap the OS pattern as needed (darwin / windows / linux):

mkdir -p ~/eunox-rel && cd ~/eunox-rel        # PowerShell: mkdir ~\eunox-rel; cd ~\eunox-rel
gh release download <tag> --repo eunolabs/eunox \
  --pattern 'eunox-mcp_*_<os>_*' \
  --pattern 'checksums.txt' --pattern 'checksums.txt.pem' --pattern 'checksums.txt.sig' \
  --pattern '*<os>*.sbom.json'
# Linux native packages, additionally:
#   --pattern 'eunox-mcp*.deb' --pattern 'eunox-mcp*.rpm'

2a. Verify the Sigstore signature on checksums.txt (same on every OS; PowerShell uses backtick line-continuations instead of \):

cosign verify-blob \
  --certificate-identity-regexp "^https://github\.com/eunolabs/eunox/\.github/workflows/release\.yml@refs/tags/" \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  --certificate checksums.txt.pem \
  --signature  checksums.txt.sig \
  checksums.txt
# expect: "Verified OK"

2b. Verify the downloaded artifact against the proven checksum file — the one command that differs per OS (hashes in checksums.txt are lowercase):

# Linux (the form documented in SECURITY.md):
sha256sum -c checksums.txt --ignore-missing      # every downloaded file prints ": OK"
# macOS (no sha256sum; compare manually):
shasum -a 256 eunox-mcp_*_darwin_arm64.tar.gz
grep darwin_arm64 checksums.txt                  # the two hashes must match
# Windows (no sha256sum):
(Get-FileHash .\eunox-mcp_*_windows_amd64.zip -Algorithm SHA256).Hash.ToLower()
Select-String -Path checksums.txt -Pattern 'windows_amd64'   # the two hashes must match

2c. Unpack, clear the first-launch guard, and confirm the real version — the other per-OS step:

# macOS -- Gatekeeper quarantine. Reproduce a real download, then clear it:
tar xzf eunox-mcp_*_darwin_arm64.tar.gz
xattr -w com.apple.quarantine "0081;00000000;Safari;" ./eunox-mcp
./eunox-mcp version                  # EXPECT: blocked ("cannot be opened")
xattr -d com.apple.quarantine ./eunox-mcp
./eunox-mcp version                  # EXPECT: the real <tag>, not "dev"
# Windows -- Mark-of-the-Web / Defender SmartScreen:
Expand-Archive .\eunox-mcp_*_windows_amd64.zip -DestinationPath .\win
Get-Item .\win\eunox-mcp.exe -Stream Zone.Identifier   # shows MotW on downloaded zips
.\win\eunox-mcp.exe version          # EXPECT: SmartScreen may warn / block
Unblock-File .\win\eunox-mcp.exe
.\win\eunox-mcp.exe version          # EXPECT: the real <tag>, not "dev"
# Linux -- no quarantine; just confirm the executable bit survived and it runs:
tar xzf eunox-mcp_*_linux_amd64.tar.gz
./eunox-mcp version                  # EXPECT: the real <tag>, not "dev"

Record the exact wording of any Gatekeeper / SmartScreen dialog — it's what users hit first, and what release notes (and the Homebrew caveats) should pre-empt.

2d. SBOM present & parseable (grype sbom:<file> / trivy sbom <file> optional):

jq -e '.spdxVersion' eunox-mcp_*_<os>_*.sbom.json >/dev/null && echo "SBOM ok"
# PowerShell: Get-Content .\eunox-mcp_*_windows_*.sbom.json | ConvertFrom-Json | Select spdxVersion

3. Install via the platform channel

Casks and winget publish only on stable tags (skip_upload: auto skips rc/pre-release).

# macOS:
brew install eunolabs/tap/eunox-mcp
which eunox-mcp          # /opt/homebrew/bin (arm64) or /usr/local/bin (Intel)
eunox-mcp version        # real tag; note whether the cask clears the quarantine
# Windows (open a fresh shell afterward so PATH refreshes):
winget install eunolabs.eunox-mcp
eunox-mcp version        # real tag; note whether install clears SmartScreen
# Linux -- confirm binary at /usr/bin and docs at /usr/share/doc/eunox-mcp/:
sudo dpkg -i eunox-mcp_*_linux_amd64.deb      # or: sudo rpm -i eunox-mcp-*_linux_x86_64.rpm
which eunox-mcp; ls /usr/share/doc/eunox-mcp/  # LICENSE + README.md

4. CLI smoke -- every subcommand resolves

Subcommands: proxy validate init suggest kill audit-verify stats doctor version.

eunox-mcp --help
eunox-mcp doctor                       # redacted support bundle; nothing uploaded
eunox-mcp validate demo/manifest.yaml  # from a checkout; expect "OK"  (Windows: demo\manifest.yaml)

doctor prints os/arch: <goos>/<goarch> — confirm it matches the build you installed, and that it redacts secrets.

5. Zero-config wiretap against a real MCP server (stdio path)

Point an MCP host at the proxy. Host-config locations differ:

  • macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
  • Windows: %APPDATA%\Claude\claude_desktop_config.json (escape backslashes in JSON paths)
  • Linux: Claude Desktop isn't available; use a Linux MCP host (VS Code + Copilot agent mode, Cursor, Cline, Roo) — same command/args shape.
{
  "mcpServers": {
    "fs-wiretap": {
      "command": "eunox-mcp",
      "args": ["proxy", "--audit", "--",
               "npx", "-y", "@modelcontextprotocol/server-filesystem", "<path>"]
    }
  }
}

Drive some tool calls, then inspect the tape and the audit dir:

eunox-mcp stats                        # per-tool allow/deny histogram
# macOS / Linux:
ls -l ~/.eunox/
stat -f '%Sp' ~/.eunox/audit.key       # macOS  -> -rw-------
stat -c '%a'  ~/.eunox/audit.key       # Linux  -> 600
# Windows: ~ resolves to %USERPROFILE%
Get-ChildItem $env:USERPROFILE\.eunox
icacls $env:USERPROFILE\.eunox\audit.key   # 0600 is POSIX-only; confirm via ACL instead

Confirm: wiretap forwards everything, tools/call records carry full args, and .../list calls are not recorded. (No host handy? Drive it over HTTP via Step 7 or the demo scripts in Step 10 — no host required.)

6. Enforcement mode -- allow + deny + conditions

eunox-mcp validate demo/manifest.yaml
eunox-mcp suggest --output /tmp/draft.yaml   # turn the wiretap tape into a draft manifest

Verify the four documented outcomes: allowed read_file /reports/*; denied write_file (AUTHORIZATION_FAILED, not in manifest); denied off-path read (CONDITION_FAILED); denied non-SELECT query_db (CONDITION_FAILED). Denials must return a structured JSON-RPC error and the upstream is never called.

7. HTTP / gateway transport + health, metrics, kill switch

eunox-mcp proxy --config demo/gateway.yaml &     # transport: http  (Windows: run in its own terminal)
curl -s localhost:3000/healthz | jq             # status, sessions, auditDropped...
curl -s localhost:3000/metrics | grep eunox_    # Prometheus counters
eunox-mcp kill all --port 3000                  # loopback /control/kill

Windows: PowerShell aliases curl to Invoke-WebRequest. Use curl.exe, or Invoke-RestMethod http://localhost:3000/healthz and (Invoke-WebRequest http://localhost:3000/metrics).Content -split "\n" | Select-String eunox_`.

Confirm /healthz, /metrics, /control/kill are loopback-only. Confirm a route with neither policy: nor enforcement: audit makes the gateway fail closed at startup (route named in the error).

8. Audit-log integrity

eunox-mcp audit-verify    # expect: "Checked N record(s): N valid, 0 invalid, 0 skipped."

Tamper test: edit one byte of a value in the audit log (~/.eunox/audit.jsonl, or %USERPROFILE%\.eunox\audit.jsonl on Windows), re-run audit-verify, confirm it flags that record (and the rest of the chain) invalid; then restore/delete. Also confirm --require-audit makes the proxy exit non-zero at startup when --audit-log points at an unwritable path (no silent blind run).

9. Drift detection

With a manifest + live upstream, confirm startup drift warnings on stderr (fm1 glob over-permission, fm2 dead reference, ...). Confirm --strict-drift turns over-permission / dead-ref / serverVersion findings into a fatal startup abort, and that a descriptionHash mismatch aborts with or without the flag. eunox-mcp validate <m> --live --upstream-url <url> reports the same out of band.

10. Docker demo + image verification

The Docker demo is the fastest independent end-to-end cross-check (Unix shell + Linux containers — on Windows run it from WSL2):

make -C demo up
make -C demo allow      # ALLOWED  read_file /reports/q3.pdf
make -C demo deny       # DENIED   write_file (AUTHORIZATION_FAILED)
make -C demo deny-path  # DENIED   /etc/shadow (CONDITION_FAILED)
make -C demo deny-op    # DENIED   query_db DELETE (CONDITION_FAILED)
make -C demo audit
make -C demo gateway-up && make -C demo ci-test-gateway
make -C demo down && make -C demo gateway-down

On Linux, if the demo audit log throws a permission error, run sudo chmod 777 demo/audit && sudo chown -R 0:0 demo/audit, then make -C demo up (noted in demo/README.md).

Verify the released image signature (bound to the manifest digest):

cosign verify \
  --certificate-identity-regexp "^https://github\.com/eunolabs/eunox/\.github/workflows/release\.yml@refs/tags/" \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  ghcr.io/eunolabs/eunox-mcp:<tag>

11. Cleanup

# macOS / Linux:
rm -rf ~/.eunox ~/eunox-rel
brew uninstall eunox-mcp                       # macOS, if installed via cask
sudo dpkg -r eunox-mcp                         # Linux deb  (or: sudo rpm -e eunox-mcp)
# Windows:
Remove-Item -Recurse -Force $env:USERPROFILE\.eunox, ~\eunox-rel
winget uninstall eunolabs.eunox-mcp

Remove the test entry from your MCP host config.

Platform gotchas most likely to bite

  1. First-launch guard on the unsigned binary — macOS Gatekeeper (xattr -d com.apple.quarantine) and Windows Mark-of-the-Web / SmartScreen (Unblock-File). Linux has none.
  2. Checksum command differs — sha256sum -c (Linux) vs shasum -a 256 (macOS) vs Get-FileHash (Windows). SECURITY.md only documents the Linux form.
  3. Both amd64 and arm64 per OS — verify and run each on matching hardware.
  4. Version string is the real tag, never dev, on the released binary.
  5. Audit-key permissions — 0600 is POSIX (stat on macOS/Linux); on Windows it doesn't map to ACLs, so check icacls / Properties instead.
  6. ~ / audit dir — ~/.eunox on macOS/Linux, %USERPROFILE%\.eunox on Windows.
  7. Loopback-only endpoints — /healthz, /metrics, /control/kill must refuse a non-loopback bind on every OS.

Clone this wiki locally