hermes debug: credential check should include credential_pool OAuth tokens, not just env vars

[teknium1]

Summary

hermes debug currently reports API-key status by checking env vars and ~/.hermes/.env entries only. For providers that authenticate via OAuth and store their tokens in auth.json under credential_pool.<provider>[*].access_token — notably Nous Portal, OpenAI Codex, Copilot — the report lists those providers as "not set" even when the user is actively logged in and the CLI is working fine.

This produces misleading debug output:

API Keys Status
  Set: gemini, elevenlabs
  Not set: openrouter, openai, anthropic, anthropic_token, nous, ...

…even when hermes model shows a healthy Nous Portal login and every inference call is succeeding. A reader of the debug report then wastes time chasing an "auth problem" that isn't real.

Reproduction

1. Fresh machine, no NOUS_API_KEY in env or ~/.hermes/.env.
2. hermes model → log in to Nous Portal via the OAuth flow. Token lands in auth.json under credential_pool.nous[0].access_token.
3. Run a few normal CLI turns — they work.
4. hermes debug (or /debug from inside the CLI).
5. Report lists nous in the "not set" API keys.

Why this matters

The debug report is the canonical artifact users share when something is broken. Right now it points "auth misconfigured" fingers at whichever provider is being used via OAuth — the exact providers we most want to expose smoothly (Nous Portal is our own). This has already caused at least one instance of the triage workflow spending cycles on "why isn't NOUS_API_KEY set?" before realizing it doesn't need to be.

Proposed fix

In the debug-dump API-keys-status section, before marking a provider as "not set," also check:

1. credential_pool.<provider>[*].access_token in auth.json (live OAuth tokens)
2. Any provider-specific non-env credential paths (Codex uses ~/.hermes/credentials/openai-codex.json or similar; Copilot has its own)

Then emit one of three states per provider:

• Set (env) — found in env var / .env
• Set (oauth) — found in credential_pool
• Not set — no credential anywhere

This matches the agent's actual runtime behavior (the provider resolver already checks credential_pool first for these providers) and makes the debug report match reality.

Bonus: surface the provider's credential-resolution status from hermes_cli/providers.py / hermes_cli/auth.py rather than re-walking the filesystem — those modules already know which providers are OAuth-backed and which are key-backed, so the debug dumper shouldn't have to re-implement that logic.

Files likely involved

• hermes_cli/debug.py (or wherever the debug dump API-keys section is assembled — owner can confirm)
• Cross-reference: hermes_cli/auth.py, hermes_cli/providers.py, agent/credential_pool.py

Labels

Suggest: enhancement, debug-tooling, user-experience

---

Filed off the back of a real debug-report diagnosis where provider=nous + model=deepseek-v4-pro showed both keys as "not set" in the report, even though Nous Portal was authenticated and Portal was routing DeepSeek requests normally.

[konsisumer]

It looks like this is being addressed in #20698. Linking for visibility; if I've misread the scope, let me know and I can take a fresh look.

[isheng-eqi]

This issue has been open for 2 months with two PRs addressing it from different angles:

• fix(cli): hermes debug now sees credential_pool OAuth tokens #20698 (konsisumer) patches hermes_cli/dump.py — the actual hermes debug command. Adds a pool_key field to the api_keys table so each provider can show set (oauth) inline, plus surfaces oauth-only providers (Codex, Qwen, Minimax, Gemini CLI) in a separate list. This directly matches the "three states per provider" proposal in the issue.

• fix(debug): hermes debug shows OAuth providers as 'not set' when authenticated (Fixes #20675) #60401 (mine) patches hermes_cli/config.py show_config() — the /config slash-command output. Adds a standalone "OAuth Tokens (credential pool)" section covering Nous Portal, Codex, Copilot, Anthropic OAuth, and xAI OAuth. Same root problem, different output path.

The two are complementary rather than competing — hermes debug and /config are different user-facing surfaces that should both reflect OAuth state. #20698 needs review as the primary fix for the issue as filed; #60401 extends the same awareness to the slash-command config view.

Happy to adjust either approach based on what direction you want for the debug/config output format.

[DeamonDev888]

PR #66972 provides the REST API foundation for this.

GET /api/credentials/pool/{provider} returns all pool entries (without api_key) including status, last_error_code, and request_count. This gives hermes status and the Desktop UI a clean API to query actual pool state instead of only checking env vars.

The split between hermes debug (env vars only) and pool state is a root cause of many "key not found" reports. #66972 makes pool state queryable from any surface.

[GottZ]

This was generated by AI during triage.

Summary

Three PRs reference #20675, but only #20698 changes the actual hermes debug/dump credential-status path; #60392 and #60401 instead add substantially the same OAuth section to show_config(), leaving the reported debug behavior unchanged.

Related pull requests

• fix(cli): hermes debug now sees credential_pool OAuth tokens #20698 [closed] best fix — (+302/-29) — partial fix and best implementation basis: it updates hermes_cli/dump.py, directly addressing the reported cause, but treats every persisted pool token as OAuth, misses runtime-loaded Qwen credentials, and lists unsupported google-gemini-cli. Although the COMMENTED keep_open review considered it salvageable, the author subsequently closed it after accepting those provider-awareness defects; it remains relevant as the closest reference implementation, not as mergeable code unchanged.
• feat(debug): check credential_pool OAuth tokens in API key status #60392 [closed] unrelated — (+606/-74) — unrelated/contaminated: its hermes debug: credential check should include credential_pool OAuth tokens, not just env vars #20675-specific hunk modifies hermes_cli/config.py rather than the run_dump() path used by hermes debug, while the remaining diff contains numerous unrelated fixes. It is closed but remains relevant because its config-display hunk is effectively the precursor/duplicate of fix(debug): hermes debug shows OAuth providers as 'not set' when authenticated (Fixes #20675) #60401.
• fix(debug): hermes debug shows OAuth providers as 'not set' when authenticated (Fixes #20675) #60401 unrelated — (+174/-0) — unrelated to the issue’s primary output path: it adds OAuth visibility to show_config() and tests that surface, but does not change hermes_cli/dump.py, so hermes debug remains incorrect; it also invokes potentially mutating load_pool() and omits Qwen and Minimax OAuth. Despite the COMMENTED keep_open review on fix(debug): hermes debug shows OAuth providers as 'not set' when authenticated (Fixes #20675) #60401, the diff supports closing or re-scoping it because that same review confirms it does not reach the issue’s requested debug path.

Duplicates

#60392 and #60401 contain essentially the same hermes_cli/config.py OAuth-status change; #60392 is additionally branch-contaminated. #20698 is not their duplicate because it targets the actual dump/debug path.

Suggested consolidation

Reopen and merge #20698 only after revising it around a non-mutating, provider-aware status resolver that classifies auth_type, covers runtime-loaded credentials such as Qwen, and removes unsupported providers; this explicitly addresses the COMMENTED keep_open review and the author’s closure rationale. Close #60392 as contaminated and duplicate of #60401’s config-only approach, and close or re-scope #60401 as a separate /config enhancement rather than a fix for #20675.

Complex graph

flowchart LR
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    I20675(["issue #20675 (open)"])
    P20698["PR #20698 (closed)"]
    subgraph Dup60392 ["PRs duplicating each other"]
        P60392["PR #60392 (closed)"]
        P60401["PR #60401 (open)"]
    end
    P20698 -->|best fix| I20675
    class I20675 open
    class I20675 target
    class P20698 closed
    class P60392 closed
    class P60401 open
    class P20698 best
    click I20675 "https://github.com/NousResearch/hermes-agent/issues/20675"
    click P20698 "https://github.com/NousResearch/hermes-agent/pull/20698"
    click P60392 "https://github.com/NousResearch/hermes-agent/pull/60392"
    click P60401 "https://github.com/NousResearch/hermes-agent/pull/60401"

Loading

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed or no verify verdict yet (state tag in the node label).

Cross-PR triage: Reviewed 3 pull requests and 1 issue in this complex. Each diff was read against this issue; Assessment working set: 65 kB of PR diffs, 7 kB of issue/PR text, 7 kB of discussion (10 comments), 5 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.