Skip to content

Get model and multiplier show up for Copilot CLI controller API route#313079

Open
anthonykim1 wants to merge 9 commits intomainfrom
anthonykim1/accurateCliModelToken
Open

Get model and multiplier show up for Copilot CLI controller API route#313079
anthonykim1 wants to merge 9 commits intomainfrom
anthonykim1/accurateCliModelToken

Conversation

@anthonykim1
Copy link
Copy Markdown
Contributor

@anthonykim1 anthonykim1 commented Apr 28, 2026

Before PR:
We went from being able to show model + token info after: #311769 to the model not showing up until user reloads.

Also not getting anything shown for auto mode before or after both #311769 and this PR branch.
Maybe related to: #303960 (comment)

Edit: Seems like related to controller API path. On latest insiders if you disable controller api setting the models show up without having to refresh.

Summary -> Live model footer comes from request handler's returned ChatResult.details.

  • Non-controller route had logic around request flow to build/ return details but the controller route was effectively completing the CLI request and returning {}. So live UI had no details to render.
  • Reason why it worked on reload: History rebuild could later recover details from persisted/session data, which is why it looked "fixed after reload" but did not show up on live.
  • Controller route now reads model after handleRequest via getCopilotCLIModelDetails.
  • Gating this behind setting: "github.copilot.chat.cli.modelDetails.enabled"

Thanks @justschen for helping with auto mode.
Side note: For some reason auto seems to always show sonnet 4.6

Co-authored-by: Copilot <copilot@github.com>
@anthonykim1 anthonykim1 self-assigned this Apr 28, 2026
Copilot AI review requested due to automatic review settings April 28, 2026 19:10
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to restore/improve Copilot CLI chat’s ability to display “last used model” (and related detail string) immediately on the first response, including across the untitled→real session swap and history rebuild paths.

Changes:

  • Track and surface a “last response model id” derived from assistant.usage to drive the model details badge.
  • Improve model-id matching (e.g., dotted vs dashed Claude ids) via aliasing utilities and use those aliases when rebuilding history and resolving preferences.
  • Defer the untitled-session swap so the request handler can return ChatResult.details before the swap happens; update/add tests accordingly.
Show a summary per file
File Description
extensions/copilot/src/extension/chatSessions/vscode-node/copilotCLIChatSessionsContribution.ts Uses last-response/selected/request model ids to compute details; schedules untitled swap asynchronously.
extensions/copilot/src/extension/chatSessions/vscode-node/test/copilotCLIChatSessionParticipant.spec.ts Adds assertions for “live” model details and waits for async swap behavior.
extensions/copilot/src/extension/chatSessions/copilotcli/node/copilotcliSession.ts Captures last response model id from assistant.usage and exposes it.
extensions/copilot/src/extension/chatSessions/copilotcli/node/copilotcliSessionService.ts Supplies a model-details-by-id map (with aliases) into history rebuild.
extensions/copilot/src/extension/chatSessions/copilotcli/common/copilotCLITools.ts Rebuilds turns with per-turn model ids and uses aliased model details when emitting response result.details.
extensions/copilot/src/extension/chatSessions/copilotcli/common/copilotCLIModelIds.ts New shared alias/equality helpers for Copilot CLI model ids.
extensions/copilot/src/extension/chatSessions/copilotcli/node/copilotCli.ts Adds matchesCopilotCLIModel and uses it for model resolution/defaulting.
extensions/copilot/src/extension/chatSessions/copilotcli/common/test/copilotCLITools.spec.ts Extends history rebuild tests to cover aliasing and per-turn model changes.
extensions/copilot/src/extension/chatSessions/copilotcli/node/test/copilotcliSession.spec.ts Ensures assistant.usage model populates getLastResponseModelId().
extensions/copilot/src/extension/chatSessions/copilotcli/node/test/copilotCliModels.spec.ts Tests dotted↔dashed Claude model id preference resolution.

Copilot's findings

  • Files reviewed: 10/10 changed files
  • Comments generated: 2

Comment thread extensions/copilot/src/extension/chatSessions/copilotcli/node/copilotCli.ts Outdated
Comment thread src/vs/workbench/api/browser/mainThreadChatSessions.ts Outdated
anthonykim1 and others added 5 commits April 28, 2026 16:15
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
@anthonykim1 anthonykim1 added this to the 1.119.0 milestone Apr 29, 2026
@anthonykim1 anthonykim1 changed the title Try to get model and token info to show up for cli in chat Get model and multiplier show up for Copilot CLI controller API route Apr 29, 2026
Co-authored-by: Copilot <copilot@github.com>
@anthonykim1 anthonykim1 reopened this Apr 29, 2026
@anthonykim1 anthonykim1 marked this pull request as ready for review April 29, 2026 01:41
* correct model details (e.g. for `auto`, where the resolved model is not otherwise
* recoverable from the persisted SDK event log).
*/
responseModelId?: string;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can get this from the events. hence do not need to store this here.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from the SDK, the usage actually doesn't survive on reloads for some reason.

that session event for assistant.usage is ephemerel which means when we build the session from events only, we don't get back the model id, but we get back auto or undefined, which means it never gets rendered.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, than you for clarification
Long live our metadata store solution

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

haha yeah it's a little unfortunate - i don't think there is anything we can do with SDK to turn that to emphemeral: false instead? otherwise this is the best we can do.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants