`gh copilot` should shim the copilot CLI

[williammartin]

Describe the feature or problem you’d like to solve

With the deprecated gh copilot extension and LLMs being trained so heavily on gh, the Copilot CLI agent (copilot) often seems to think that it would call gh copilot to invoke itself. See this conversation snippet for example:

 > If you wanted to execute yourself in a bash shell, what command would you call? No looking it up, just what your current context suggest

 ● Based on my version information and the fact that I'm the "GitHub Copilot CLI", the command would likely be:

     gh copilot

   This follows the pattern of gh being the GitHub CLI tool, with copilot as a subcommand/extension for the Copilot CLI functionality.

0.0.375-0
Commit: 52f36a4

Additionally, there are a lot of users out there for whom calling gh copilot now invokes the non-working extension. It would be nice if gh copilot could shim over the copilot CLI.

Proposed solution

A bit of handwaving here but since invoking copilot CLI would involve downloading a binary, we may want a confirmation step.

It doesn't need to be an extension, and it would be a subpar experience if it were an extension as extensions currently are today. However, in the past we've talked about having better support for "blessed" extensions where running the necessary command installs them without needing gh ext install, and this might dovetail nicely with that work.

Additional context

One pitfall in my memory is that cobra does an unstable sort on command names and when selecting the appropriate command, it linearly searches that sorted list. As such, we can't rely on ordering of command addition alone to prioritise a native command, if we went down that route.

[github-actions]

Thank you for your issue! We have categorized it as a feature request, and it has been added to our backlog. In doing so, we are not committing to implementing this feature at this time, but, we will consider it for future releases based on community feedback and our own product roadmap.

Unless you see the help wanted Contributions welcome label, we are not currently looking for external contributions for this feature.

If you come across this issue and would like to see it implemented, please add a thumbs up! This will help us prioritize the feature. Please only comment if you have additional information or viewpoints to contribute.

[BagToad]

@williammartin I don't think I'm following some things here - can you expand on what "shim" means to you? Why wouldn't we just create a full copilot core command that does what we want? Maybe that behavior would be invoking copilot CLI or it would be printing a message or something else, but it's not clear to me what behavior we are talking about. You mention downloading a binary, but I'm hoping to get clarity that you're talking about invoking the binary or not or something else that I misunderstood.

It may help to just describe what you envision should happen when the user runs gh copilot, perhaps in these situations:

• When the deprecated extension is installed
• When the deprecated extension is not installed

---

With the deprecated gh copilot extension and LLMs being trained so heavily on gh, the Copilot CLI agent (copilot) often seems to think that it would call gh copilot to invoke itself. See this conversation snippet for example:

With regard to the core problem described above:

• Is there a way Copilot can be "taught" how to invoke itself properly rather than changing gh? I'm trying to understand the value of changing gh's core.
  • Are we defaulting to wanting to change gh because it will help users with the deprecated gh-copilot extension installed?
• Why is copilot trying to invoke itself? Is that a typical/common use-case?

[anaskmuhammed97-byte]

###ndkddk

[williammartin]

All great questions.

can you expand on what "shim" means to you?

I use the word "shim" to indicate that gh would be a thin layer over interacting with the Copilot CLI in this case.

Why wouldn't we just create a full copilot core command that does what we want?

That's what I'm suggesting.

Maybe that behavior would be invoking copilot CLI

Yep, that's pretty much what I mean.

It may help to just describe what you envision should happen when the user runs gh copilot, perhaps in these situations:
When the deprecated extension is installed
When the deprecated extension is not installed

I don't think I would do anything different in either case except perhaps display a warning. Otherwise, something like:

Given I have Copilot CLI already on my path
When I run gh copilot -- [flags] (not sure about --)
Then I am using the Copilot CLI as if I had called it directly

Given I do not have Copilot CLI already on my path
When I run gh copilot
Then I am informed that this will download the Copilot CLI and use the gh token to authenticate (or something to that affect)

Broadly, something like https://github.com/github/gh-copilot2

Is there a way Copilot can be "taught" how to invoke itself properly rather than changing gh?

Yes this is possible at the cost of additional tokens. Not a huge deal. Mostly following the idea of https://en.wikipedia.org/wiki/Desire_path here.

Why is copilot trying to invoke itself? Is that a typical/common use-case?

Not extremely common. Can invoke itself for a variety of reasons e.g. testing itself.

I'm trying to understand the value of changing gh's core.

1. To expose another surface area for installing and interacting with Copilot CLI
2. To provide a better "upgrade" experience for anyone still trying to invoke the extension
3. To help the LLM achieve what it wants when it expects that Copilot CLI is invoked via gh copilot

@devm33 any more reasons?

[devm33]

Yep I think you covered it @williammartin!

[babakks]

Here are my two cents.

The main scenario that we want to fix in this issue is where Copilot trying to invoke itself through gh. In this scenario, copilot binary is already installed. So, I think the least minimum solution would be a hidden copilot command in gh that invokes the Copilot binary. I'm saying hidden, since:

• This minimal approach is not a nice/ergonomic UX for users and we may want to change it in the future.
• We don't want to advertise this as a legit way for users/LLMs to invoke Copilot CLI.
• A leaf command at the top level is a bit scary as we won't be able to safely touch/extend it in the future.

Another reason that I'm not really comfortable with installing Copilot CLI is that if we're going to install it, it should be stored somewhere discoverable via PATH dirs, and this can be challenging. For example, if for any reason (e.g. upgrading Copilot CLI), the user attempts to install Copilot CLI using Homebrew or npm, then it's not really clear what would happen.

On the other hand, if we want to install Copilot CLI in an isolated location (like gh-extensions), it'll be on gh to keep it updated, otherwise it'll go outdated at some point, and it'll be hard to debug for users.

[williammartin]

The main scenario that we want to fix in this issue is where Copilot trying to invoke itself through gh

I'm not sure I would say that is the main scenario, just the one I provided the most color to as I hastily wrote up the issue. See: last points on previous message. With that in mind, I'm not sure the hidden command and follow on makes sense.

I do think all the other concerns about lifecycle management are valid. FWIW copilot auto upgrades itself unless you configure it not to.