[Bug]: /model picker for Bedrock offers bare foundation-model IDs that 400 on on-demand accounts (and persists them)

[lianghong]

Bug Description

The in-session /model picker (and switch_model path) for the Bedrock provider offers bare foundation-model IDs alongside their inference-profile counterparts. On on-demand accounts the bare IDs are not invokable — selecting one persists it to config.yaml (model.default), and every subsequent call in every future session fails with:

HTTP 400: Invocation of model ID anthropic.claude-fable-5 with on-demand throughput
isn't supported. Retry your request with the ID or ARN of an inference profile that
contains this model.

The confusing part: the setup wizard already solves this. hermes_cli/model_setup_flows.py (~line 2224) deduplicates the discovery output, preferring us.*/global.* inference profiles and hiding bare IDs that are covered by a profile. But the /model picker path — list_authenticated_providers() → cached_provider_model_ids("bedrock") → bedrock_model_ids_or_none() (agent/bedrock_adapter.py:387) — returns the raw discovery list with no such dedup, so the picker shows both forms of the same model:

global.anthropic.claude-fable-5     ← works
...
anthropic.claude-fable-5            ← same model; 400s on invoke, but picker offers it

discover_bedrock_models() even tags which entries are profiles ("provider": "inference-profile"), so the information needed to dedup is already there — it's just dropped when the list is flattened to bare ID strings.

Not a duplicate of #10774: that one was closed as a missing bedrock:ListInferenceProfiles IAM permission (profiles absent from the list). Here discovery is fully working and profiles ARE listed — the bug is that the un-invokable bare IDs are listed too, as separate picker entries.

Steps to Reproduce

1. Configure model.provider: bedrock with a working AWS_BEARER_TOKEN_BEDROCK (on-demand account, no provisioned throughput), bedrock.discovery.enabled: true.
2. Start an interactive session, run /model, pick Bedrock, scroll past the global.* entries and select anthropic.claude-fable-5 (listed as a normal entry).
3. The switch persists to config (model.default: anthropic.claude-fable-5).
4. Next message → HTTP 400 above. Every new session also fails until the config is hand-edited back to the global.-prefixed ID.

Expected Behavior

The picker should apply the same profile-preferring dedup the setup wizard uses: when a bare foundation-model ID is covered by a discovered ACTIVE inference profile, list only the profile ID. (Alternatively/additionally: validate_requested_model's bedrock branch could warn when persisting a bare anthropic.* ID that has a profile sibling in the discovery output.)

Suggested Fix

Apply the dedup inside bedrock_model_ids_or_none() in agent/bedrock_adapter.py — that helper is the shared source for provider_model_ids and list_authenticated_providers sections 2 and 3, so fixing it there covers the CLI picker, gateway picker, and validation suggestions in one place:

_PROFILE_PREFIXES = ("us.", "global.", "eu.", "ap.", "jp.")
ids = [m["id"] for m in discovered]
profile_bases = {
    mid.split(".", 1)[1] for mid in ids if mid.startswith(_PROFILE_PREFIXES)
}
return [
    mid for mid in ids
    if mid.startswith(_PROFILE_PREFIXES) or mid not in profile_bases
]

Tested locally against live us-east-1 discovery: picker drops the covered bare IDs (anthropic.claude-*) while keeping profiles and uncovered bare models (e.g. amazon.titan-*); end-to-end /model selection of global.anthropic.claude-fable-5 then works. Happy to send this as a PR if the approach looks right.

Environment

• hermes-agent 0.18.0 (wheel install), commit 19d4174
• Python 3.14.6, Linux x86_64
• Provider: bedrock via AWS_BEARER_TOKEN_BEDROCK (bearer auth), region us-east-1
• anthropic SDK 0.116.0

[GottZ]

Three open PRs currently address or reference this issue:

• fix(bedrock): dedup profile-covered bare model IDs in /model picker (#58185) #58201 (+100/-1) — fixes the root cause: implements exactly the profile dedup proposed in the issue (_PROFILE_PREFIXES, profile_bases set) in bedrock_model_ids_or_none() in agent/bedrock_adapter.py — the spot named in the issue as the common source for picker, gateway, and validation. Five unit tests cover dropping the bare-ID sibling, keeping uncovered bare IDs, all regional prefixes, and order preservation; CI is green.
• fix: Bedrock model picker deduplicates inference profile IDs (#58185) #58202 (+9/-1) — fixes the root cause: adds the same dedup logic (identical prefix tuple and comprehensions) inline in bedrock_model_ids_or_none(); no tests, only 9 lines, but functionally complete for the described bug.
• fix(bedrock): skip profile-only foundation models in discovery (#58185) #65810 (+84/-0) — fixes the root cause: filters models in discover_bedrock_models() whose inferenceTypesSupported lacks ON_DEMAND — precisely the bare IDs that fail with HTTP 400 per the issue — removing them upstream from every picker list; two tests cover skipping INFERENCE_PROFILE-only models and permissive behavior when the field is absent. Uses the authoritative AWS field instead of a prefix heuristic, but has no CI run (ci: NONE).

Duplicates: #58201 and #58202 implement essentially the same change.

Suggested consolidation: merge #58201 — it implements the fix at the cause identified in the issue (bedrock_model_ids_or_none) as a named, documented helper, backed by five targeted unit tests and green CI. #58202 could be closed as a duplicate without tests. #65810 is an alternative, technically valid upstream approach via inferenceTypesSupported, but it lacks a CI run and has a broader blast radius (it filters discovery output for all consumers, including the setup wizard) — after merging #58201 it could either be closed as superseded or evaluated separately as a discovery hardening. Nothing remains open afterwards; the issue is fully addressed by #58201.

Cross-PR triage: each diff was read against this issue; verdicts reflect diff content, not PR titles. Part of an automated triage batch.

[GottZ]

This was generated by AI during triage.

Delta since our previous triage comment

Since the previous triage comment, @Sahil-SS9 closed #58202 in favor of #58201 and explicitly confirmed that it is the same fix, filed about 45 seconds later. No listed delta changes the assessment of #58201 or #65810.

Changed pull requests

• fix: Bedrock model picker deduplicates inference profile IDs (#58185) #58202 [closed] fixes — (+9/-1) — duplicate, now closed: despite the earlier keep_open review on fix: Bedrock model picker deduplicates inference profile IDs (#58185) #58202, its diff implements the same profile-sibling filter as fix(bedrock): dedup profile-covered bare model IDs in /model picker (#58185) #58201, while fix(bedrock): dedup profile-covered bare model IDs in /model picker (#58185) #58201 is the canonical earlier PR and includes regression tests; @Sahil-SS9 therefore closed fix: Bedrock model picker deduplicates inference profile IDs (#58185) #58202 in favor of fix(bedrock): dedup profile-covered bare model IDs in /model picker (#58185) #58201.

Suggested consolidation

The recommendation remains to consolidate on #58201, after addressing the contributor review’s missing apac. coverage; #58202 is now closed as its duplicate.

Complex graph

flowchart LR
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    I58185(["issue #58185 (open)"])
    P65810["PR #65810 (open)"]
    subgraph Dup58201 ["PRs duplicating each other"]
        P58201["PR #58201 (open)"]
        P58202["PR #58202 (closed)"]
    end
    P58201 -->|best fix| I58185
    P58202 -->|fixes| I58185
    P65810 -->|best fix| I58185
    class I58185 open
    class I58185 target
    class P58201 open
    class P58202 closed
    class P65810 open
    class P58201 best
    class P65810 best
    click I58185 "https://github.com/NousResearch/hermes-agent/issues/58185"
    click P58201 "https://github.com/NousResearch/hermes-agent/pull/58201"
    click P58202 "https://github.com/NousResearch/hermes-agent/pull/58202"
    click P65810 "https://github.com/NousResearch/hermes-agent/pull/65810"

Loading

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed or no verify verdict yet (state tag in the node label).

Cross-PR triage: Reviewed 3 pull requests and 1 issue in this complex. Each diff was read against this issue; Assessment working set: 11 kB of PR diffs, 12 kB of issue/PR text, 7 kB of discussion (10 comments), 8 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.