Skip to content

Pre‐Release Manual Testing

Jump to bottom
eunolabs edited this page Jun 15, 2026 · 3 revisions

Pre-Release Manual Testing

A pre-release checklist for validating an eunox-mcp release candidate on macOS, Windows, and Linux. Enforcement behavior is identical across platforms — what differs is the delivery surface (install channel, archive format, signature/checksum verification, first-launch OS guard, and file paths), so the shared steps are written once and per-OS variants are called out inline.

.goreleaser.yml builds every OS for both amd64 and arm64 — test each arch on matching hardware if you ship both.

What differs per platform

macOS Windows Linux
Install channel Homebrew cask (stable tags only) winget (stable tags only) .deb / .rpm / tarball
Archive .tar.gz .zip (eunox-mcp.exe) .tar.gz
Checksum tool shasum -a 256 Get-FileHash sha256sum -c (native)
First-launch guard Gatekeeper quarantine Mark-of-the-Web / SmartScreen none
Audit dir (~/.eunox) ~/.eunox %USERPROFILE%\.eunox ~/.eunox
Key-mode check stat -f '%Sp' → -rw------- icacls (ACL, not POSIX) stat -c '%a' → 600

The released binaries for macOS and Windows are unsigned (per .goreleaser.yml), so users hit Gatekeeper / SmartScreen on first launch; Linux has no equivalent. The first-launch step below documents the unblock for each.

0. Prep

# macOS (Homebrew):
brew install go jq cosign gh
# Windows (winget); commands below are PowerShell:
winget install --id sigstore.cosign; winget install --id GitHub.cli; winget install --id jqlang.jq
# Linux (apt/dnf); install cosign + gh per their upstream instructions:
sudo apt-get install -y jq      # or: sudo dnf install -y jq

go (>= 1.25) is only needed for the source build in Step 1; node is for the npx filesystem-server smoke test in Step 5.

1. Source sanity gate (catches gross breakage)

# macOS / Linux:
cd eunox
make check-fmt && make check-license && make check-notice
make test          # go test -race -count=1 ./...
make build         # -> bin/eunox-mcp
./bin/eunox-mcp version    # a source build prints "dev" -- expected here
# Windows (no make; use the raw go commands):
cd eunox
go test -race -count=1 ./...
go build -o bin\eunox-mcp.exe .\cmd\eunox-mcp
.\bin\eunox-mcp.exe version    # prints "dev" -- expected for a local build

The dev string is correct only for a local build. On the released binary the version must be the real tag.

2. Released-artifact verification (the core pre-release check)

Download the candidate's artifacts + signature chain (replace <tag>, e.g. v0.1.0-rc1). Swap the OS pattern as needed (darwin / windows / linux):

mkdir -p ~/eunox-rel && cd ~/eunox-rel        # PowerShell: mkdir ~\eunox-rel; cd ~\eunox-rel
gh release download <tag> --repo eunolabs/eunox \
  --pattern 'eunox-mcp_*_<os>_*' \
  --pattern 'checksums.txt' --pattern 'checksums.txt.pem' --pattern 'checksums.txt.sig' \
  --pattern '*<os>*.sbom.json'
# Linux native packages, additionally:
#   --pattern 'eunox-mcp*.deb' --pattern 'eunox-mcp*.rpm'

2a. Verify the Sigstore signature on checksums.txt (same on every OS; PowerShell uses backtick line-continuations instead of \):

cosign verify-blob \
  --certificate-identity-regexp "^https://github\.com/eunolabs/eunox/\.github/workflows/release\.yml@refs/tags/" \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  --certificate checksums.txt.pem \
  --signature  checksums.txt.sig \
  checksums.txt
# expect: "Verified OK"

2b. Verify the downloaded artifact against the proven checksum file — the one command that differs per OS (hashes in checksums.txt are lowercase):

# Linux (the form documented in SECURITY.md):
sha256sum -c checksums.txt --ignore-missing      # every downloaded file prints ": OK"
# macOS (no sha256sum; compare manually):
shasum -a 256 eunox-mcp_*_darwin_arm64.tar.gz
grep darwin_arm64 checksums.txt                  # the two hashes must match
# Windows (no sha256sum):
(Get-FileHash .\eunox-mcp_*_windows_amd64.zip -Algorithm SHA256).Hash.ToLower()
Select-String -Path checksums.txt -Pattern 'windows_amd64'   # the two hashes must match

2c. Unpack, clear the first-launch guard, and confirm the real version — the other per-OS step:

# macOS -- Gatekeeper quarantine. Reproduce a real download, then clear it:
tar xzf eunox-mcp_*_darwin_arm64.tar.gz
xattr -w com.apple.quarantine "0081;00000000;Safari;" ./eunox-mcp
./eunox-mcp version                  # EXPECT: blocked ("cannot be opened")
xattr -d com.apple.quarantine ./eunox-mcp
./eunox-mcp version                  # EXPECT: the real <tag>, not "dev"
# Windows -- Mark-of-the-Web / Defender SmartScreen:
Expand-Archive .\eunox-mcp_*_windows_amd64.zip -DestinationPath .\win
Get-Item .\win\eunox-mcp.exe -Stream Zone.Identifier   # shows MotW on downloaded zips
.\win\eunox-mcp.exe version          # EXPECT: SmartScreen may warn / block
Unblock-File .\win\eunox-mcp.exe
.\win\eunox-mcp.exe version          # EXPECT: the real <tag>, not "dev"
# Linux -- no quarantine; just confirm the executable bit survived and it runs:
tar xzf eunox-mcp_*_linux_amd64.tar.gz
./eunox-mcp version                  # EXPECT: the real <tag>, not "dev"

Record the exact wording of any Gatekeeper / SmartScreen dialog — it's what users hit first, and what release notes (and the Homebrew caveats) should pre-empt.

2d. SBOM present & parseable (grype sbom:<file> / trivy sbom <file> optional):

jq -e '.spdxVersion' eunox-mcp_*_<os>_*.sbom.json >/dev/null && echo "SBOM ok"
# PowerShell: Get-Content .\eunox-mcp_*_windows_*.sbom.json | ConvertFrom-Json | Select spdxVersion

3. Install via the platform channel

Casks and winget publish only on stable tags (skip_upload: auto skips rc/pre-release).

# macOS:
brew install eunolabs/tap/eunox-mcp
which eunox-mcp          # /opt/homebrew/bin (arm64) or /usr/local/bin (Intel)
eunox-mcp version        # real tag; note whether the cask clears the quarantine
# Windows (open a fresh shell afterward so PATH refreshes):
winget install eunolabs.eunox-mcp
eunox-mcp version        # real tag; note whether install clears SmartScreen
# Linux -- confirm binary at /usr/bin and docs at /usr/share/doc/eunox-mcp/:
sudo dpkg -i eunox-mcp_*_linux_amd64.deb      # or: sudo rpm -i eunox-mcp-*_linux_x86_64.rpm
which eunox-mcp; ls /usr/share/doc/eunox-mcp/  # LICENSE + README.md

4. CLI smoke -- every subcommand resolves

Subcommands: proxy validate init suggest kill audit-verify stats doctor version.

eunox-mcp --help
eunox-mcp doctor                       # redacted support bundle; nothing uploaded
eunox-mcp validate demo/manifest.yaml  # from a checkout; expect "OK"  (Windows: demo\manifest.yaml)

doctor prints os/arch: <goos>/<goarch> — confirm it matches the build you installed, and that it redacts secrets.

5. Zero-config wiretap against a real MCP server (stdio path)

Point an MCP host at the proxy. Host-config locations differ:

  • macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
  • Windows: %APPDATA%\Claude\claude_desktop_config.json (escape backslashes in JSON paths)
  • Linux: Claude Desktop isn't available; use a Linux MCP host (VS Code + Copilot agent mode, Cursor, Cline, Roo) — same command/args shape.
{
  "mcpServers": {
    "fs-wiretap": {
      "command": "eunox-mcp",
      "args": ["proxy", "--audit", "--",
               "npx", "-y", "@modelcontextprotocol/server-filesystem", "<path>"]
    }
  }
}

macOS: point <path> at a dedicated scratch dir (e.g. mkdir ~/eunox-test), not your home folder or ~/Desktop / ~/Documents / ~/Downloads / ~/Pictures. Those are TCC-protected, so a recursive tool call (e.g. "find docx files in my home folder") makes the filesystem server touch them and macOS fires a cascade of privacy prompts attributed to eunox-mcp — it's the launched binary, and the server runs as its child. That's the OS consent layer, unrelated to eunox policy (eunox checks the MCP call's arguments, not the upstream's raw filesystem syscalls); a scratch dir avoids it.

Drive some tool calls, then inspect the tape and the audit dir:

eunox-mcp stats                        # per-tool allow/deny histogram
# macOS / Linux:
ls -l ~/.eunox/
stat -f '%Sp' ~/.eunox/audit.key       # macOS  -> -rw-------
stat -c '%a'  ~/.eunox/audit.key       # Linux  -> 600
# Windows: ~ resolves to %USERPROFILE%
Get-ChildItem $env:USERPROFILE\.eunox
icacls $env:USERPROFILE\.eunox\audit.key   # 0600 is POSIX-only; confirm via ACL instead

Confirm: wiretap forwards everything, tools/call records carry full args, and .../list calls are recorded as enumeration events (allow records named after the method, no argument map). (No host handy? Drive it over HTTP via Step 7 or the demo scripts in Step 10 — no host required.)

6. Enforcement mode -- allow + deny + conditions

Graduate the Step 5 wiretap into enforcement against the same upstream.

6a. Draft a manifest from the tape, then review and tighten it — the draft describes what the agent did, not vetted policy. Write it to a real reviewed path (not /tmp, which is ephemeral and doesn't exist on Windows):

eunox-mcp suggest --output ./eunox-manifest.yaml   # reads the wiretap audit tape
eunox-mcp validate ./eunox-manifest.yaml           # static check; expect "OK"
# Windows (PowerShell): same commands with .\eunox-manifest.yaml

6b. Write an enforcing config pointing at the same upstream you wiretapped:

# eunox.yaml
schemaVersion: "0.1"
transport: stdio
upstreams:
  - name: filesystem
    transport: stdio
    command: npx
    args: ["-y", "@modelcontextprotocol/server-filesystem", "<same path as Step 5>"]
    policy: ["./eunox-manifest.yaml"]

6c. Re-point the MCP host from wiretap to enforcement. In the same host config edited in Step 5, swap the wiretap args for the enforcing config (use an absolute path), then restart the host:

"args": ["proxy", "--config", "/abs/path/to/eunox.yaml"]   // was: ["proxy","--audit","--","npx",...]

6d. Exercise the policy from the host and confirm: a permitted call succeeds; a tool absent from the manifest is denied with AUTHORIZATION_FAILED; a call whose argument violates a condition (wrong path, disallowed SQL op) is denied with CONDITION_FAILED. Every denial returns a structured JSON-RPC error and the upstream is never called.

The canonical four outcomes — read_file /reports/* allowed, write_file denied, off-/reports/* read denied, non-SELECT query_db denied — are wired end-to-end against the mock server in the Docker demo (Step 10). Run that for a fixed, scriptable allow/deny matrix without standing up a host.

7. HTTP / gateway transport + health, metrics, kill switch

eunox-mcp proxy --config demo/gateway.yaml &     # transport: http  (Windows: run in its own terminal)
curl -s localhost:3000/healthz | jq             # status, sessions, auditDropped...
curl -s localhost:3000/metrics | grep eunox_    # Prometheus counters
eunox-mcp kill all --port 3000                  # loopback /control/kill

Windows: PowerShell aliases curl to Invoke-WebRequest. Use curl.exe, or Invoke-RestMethod http://localhost:3000/healthz and (Invoke-WebRequest http://localhost:3000/metrics).Content -split "\n" | Select-String eunox_`.

Confirm /healthz, /metrics, /control/kill are loopback-only. Confirm a route with neither policy: nor enforcement: audit makes the gateway fail closed at startup (route named in the error).

8. Audit-log integrity

eunox-mcp audit-verify    # expect: "Checked N record(s): N valid, 0 invalid, 0 skipped."

Tamper test: edit one byte of a value in the audit log (~/.eunox/audit.jsonl, or %USERPROFILE%\.eunox\audit.jsonl on Windows), re-run audit-verify, confirm it flags that record (and the rest of the chain) invalid; then restore/delete. Also confirm --require-audit makes the proxy exit non-zero at startup when --audit-log points at an unwritable path (no silent blind run).

9. Drift detection

With a manifest + live upstream, confirm startup drift warnings on stderr (fm1 glob over-permission, fm2 dead reference, ...). Confirm --strict-drift turns over-permission / dead-ref / serverVersion findings into a fatal startup abort, and that a descriptionHash mismatch aborts with or without the flag. eunox-mcp validate <m> --live --upstream-url <url> reports the same out of band.

10. Docker demo + image verification

The Docker demo is the fastest independent end-to-end cross-check (Unix shell + Linux containers — on Windows run it from WSL2):

make -C demo up
make -C demo allow      # ALLOWED  read_file /reports/q3.pdf
make -C demo deny       # DENIED   write_file (AUTHORIZATION_FAILED)
make -C demo deny-path  # DENIED   /etc/shadow (CONDITION_FAILED)
make -C demo deny-op    # DENIED   query_db DELETE (CONDITION_FAILED)
make -C demo audit
make -C demo gateway-up && make -C demo ci-test-gateway
make -C demo down && make -C demo gateway-down

On Linux, if the demo audit log throws a permission error, run sudo chmod 777 demo/audit && sudo chown -R 0:0 demo/audit, then make -C demo up (noted in demo/README.md).

Verify the released image signature (bound to the manifest digest):

cosign verify \
  --certificate-identity-regexp "^https://github\.com/eunolabs/eunox/\.github/workflows/release\.yml@refs/tags/" \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  ghcr.io/eunolabs/eunox-mcp:<tag>

11. Cleanup

# macOS / Linux:
rm -rf ~/.eunox ~/eunox-rel
brew uninstall eunox-mcp                       # macOS, if installed via cask
sudo dpkg -r eunox-mcp                         # Linux deb  (or: sudo rpm -e eunox-mcp)
# Windows:
Remove-Item -Recurse -Force $env:USERPROFILE\.eunox, ~\eunox-rel
winget uninstall eunolabs.eunox-mcp

Remove the test entry from your MCP host config.

Platform gotchas most likely to bite

  1. First-launch guard on the unsigned binary — macOS Gatekeeper (xattr -d com.apple.quarantine) and Windows Mark-of-the-Web / SmartScreen (Unblock-File). Linux has none.
  2. Checksum command differs — sha256sum -c (Linux) vs shasum -a 256 (macOS) vs Get-FileHash (Windows). SECURITY.md only documents the Linux form.
  3. Both amd64 and arm64 per OS — verify and run each on matching hardware.
  4. Version string is the real tag, never dev, on the released binary.
  5. Audit-key permissions — 0600 is POSIX (stat on macOS/Linux); on Windows it doesn't map to ACLs, so check icacls / Properties instead.
  6. ~ / audit dir — ~/.eunox on macOS/Linux, %USERPROFILE%\.eunox on Windows.
  7. Loopback-only endpoints — /healthz, /metrics, /control/kill must refuse a non-loopback bind on every OS.

Clone this wiki locally