Skip to content

Make a CLI/API-key project mismatch diagnosable instead of silent - #6

Draft
garethx wants to merge 1 commit into
tests/tools-coveragefrom
fix/cli-project-mismatch
Draft

Make a CLI/API-key project mismatch diagnosable instead of silent#6
garethx wants to merge 1 commit into
tests/tools-coveragefrom
fix/cli-project-mismatch

Conversation

@garethx

@garethx garethx commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Fixes #4. Stacked on #5 (base is tests/tools-coverage) because it extends the tunnel and doctor tests that PR adds — rebase onto main once #5 lands.

What was wrong

In cli mode "which project" has two independent answers, and nothing reconciled them. hermes hookdeck setup creates the connection in the API key's project; hookdeck listen looks for it in the CLI's. When they differ the gateway starts, logs that it is listening, and receives nothing — the tunnel restart-loops at warning level where it reads as routine churn.

Suggestions 1 and 2, implemented

doctor compares them.

✗ Project mismatch: the CLI forwards from tm_cli but the API key acts on tm_apikey.
  `hermes hookdeck setup` creates connections in the API key's project while
  `hookdeck listen` looks for them in the CLI's, so the gateway will report
  healthy and receive nothing.
  note: Point the CLI at the same project with `hookdeck login`, or set the API
  key to one belonging to tm_cli.

There is no endpoint that reports which project a key belongs to — I checked /teams/current, /teams and /projects against the live API and all three are unavailable. But every resource carries team_id, so the connections doctor already lists answer it for free, and only an empty result costs an extra request.

Two cases are deliberately not mismatches: a key that reaches no connections is reported as unverified, because a project can legitimately be empty and sending an operator to fix that would be worse than silence. A CLI with no session at all is its own failure, since hookdeck listen cannot start.

The supervisor escalates a standing failure. Three consecutive too-short runs now log once at error level with the CLI's own last lines and, where recognised, the cause:

[hookdeck] CLI tunnel has failed to stay up 3 times in a row (last run 1s). No
events are reaching the gateway. Likely cause: the CLI is logged into a different
Hookdeck project than the API key used by `hermes hookdeck setup`… Last output:
no connection found matching filter "livetest" for source "hermes-livetest"

Once per streak, not per restart. A healthy run re-arms it, so a second outage is not silent.

Suggestion 3: it does not work as described

hookdeck listen --hookdeck-config <path> is real, and this PR wires it up as a cli_config setting — so the CLI's project becomes a plugin setting rather than ambient state.

What does not work is having the gateway populate that config itself. I tested it: the CLI rejects a project API key as a session key.

$ hookdeck --hookdeck-config /tmp/gw/config.toml project list
Authentication failed: your API key is invalid or expired.

Only hookdeck ci can mint a session key — and that is the command that rewrites the shared config in the first place, so it cannot be the escape from it. ci does accept --hookdeck-config as a global flag, so if it honours it the loop closes; but given --local was found not to honour its own documentation, that needs verifying before relying on it, and verifying it means running a command that has already destroyed a session once in this project. I have left that alone rather than test it on someone's machine. Documented in Limitations as the constraint it is.

Verification

293 tests pass. New behaviour is mutation-checked: disabling the mismatch branch and removing the escalation each fail the suite. The tunnel tests replay the exact CLI output from the issue.

One incidental fix: the doctor tests read the developer's real ~/.config/hookdeck/config.toml, so they passed locally and would have failed in CI, where no session exists. The fixture now owns that path.

Fixes #4.

In cli mode "which project" has two independent answers: HOOKDECK_API_KEY,
which every `hermes hookdeck` command and the dashboard act on, and the CLI's
own config, which is what `hookdeck listen` forwards from. Nothing reconciled
them, and when they differed the gateway logged that it was listening while
receiving nothing — the tunnel restart-looping at warning level out of sight.

Three changes, matching the issue's first two suggestions:

* `doctor` now compares them and names both project ids. There is no endpoint
  that reports a key's project, but every resource carries `team_id`, so the
  connections doctor already lists answer it without an extra request. A key
  reaching no connections is reported as unverified rather than as a
  mismatch, since a project can legitimately be empty. A CLI with no session
  at all is its own failure — `hookdeck listen` cannot start.

* The tunnel supervisor escalates a standing failure. Three consecutive runs
  too short to be healthy now log once at error level with the CLI's own last
  lines and, for the failures that retrying cannot fix, the likely cause. The
  backoff is for network blips; `no connection found matching filter` is not
  one, and at warning level it read as routine churn. A healthy run re-arms
  it, so a second outage is not silent.

* `cli_config` passes `--hookdeck-config` to `hookdeck listen`, making the
  CLI's project a plugin setting rather than ambient state shared with
  everything else on the machine.

The issue's third suggestion — having the gateway populate that config itself
— does not work as described, and the docs say so: the CLI refuses a project
API key as a session key ("your API key is invalid or expired"), so only
`hookdeck ci` can mint one, and that is the command that rewrites the shared
config in the first place.

Also fixes a test that passed only on a machine with a CLI session: the doctor
fixture now owns the config path, where before it read the developer's real
one and would have failed in CI.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant