Speed up listing extensions by lazy-loading extension information when needed

[mislav]

Fetching extension information such as its GitHub URL, current version, and pinned property isn't strictly necessary for just listing extensions, so this restructures extensions code to lazy-load this information on ext.URL(), ext.CurrentVersion(), ext.LatestVersion(), and ext.IsPinned(). This generally is a shift in responsibility for these items from the extension manager to the extension itself.

This will speed up gh invocation and shell completion, since extensions are automatically listed when constructing the gh command tree. Followup to #7457

[samcoe]

@mislav Thanks for getting started on this work and caring deeply about the performance of gh. My understanding of the performance issue is that when listing extensions we shell out to git to retrieve information for non-binary extensions, these git invocations are slow when compared to reading the manifest files for binary extensions. Is that accurate?

Assuming it is the git invocations that we are really trying to reduce here I have two alternative approaches that I think we could consider here.

1. When mounting extensions on gh initialization we only rely on the extension name. Right now we use the name and the URL if it is available. Removing the URL would remove the git invocation. We would need to add in a new function to the extension manager to list just the extension names but that would be trivial. This is the simplest idea I could come up with, but slightly sacrifices the UX.

2. We start writing manifest files for non-binary extensions. When installing/updating an extension we will create manifest files just like we do for binary extensions. Then when listing extensions we can read the manifest files instead of invoking git. I like this approach as it actually brings the two types of extensions more inline with each other. The code for writing manifest files is already in place so I don't see this is a huge lift. The only thing I would suggest is that when listing extensions we try to write a manifest file if one does not exist so that users won't need to do an install/update of an extension to see get the performance improvement.

Thoughts?

[mislav]

1. Removing the URL would remove the git invocation. We would need to add in a new function to the extension manager to list just the extension names but that would be trivial.

I'd prefer this approach. However, I don't see why we should add yet another list functionality when we already have two (list w/ version metadata vs. list without version metadata). I'd rather reduce our current list function to only list extensions by name, and then when the caller requests more information per extension, those are looked up on request.

2. We start writing manifest files for non-binary extensions.

I appreciate this idea and I don't think it would be hard to do, but honestly I don't see why we should start writing manifest for git extensions when we can already look up information about them locally. We should just avoid looking up this information altogether unless the information is necessary.

The way I envision the lazy-loading approach, we wouldn't even load the binary extension manifest in plain list mode.

[samcoe]

I'd prefer this approach. However, I don't see why we should add yet another list functionality when we already have two (list w/ version metadata vs. list without version metadata). I'd rather reduce our current list function to only list extensions by name, and then when the caller requests more information per extension, those are looked up on request.

That's a fair point, the listing functionality with the two functions is already a bit confusing. One option, that may be less work than making this lazy-loaded would be to have the list function take in a list of fields that the caller is requesting being loaded, similar to a GraphQL request. Just an idea.

I appreciate this idea and I don't think it would be hard to do, but honestly I don't see why we should start writing manifest for git extensions when we can already look up information about them locally. We should just avoid looking up this information altogether unless the information is necessary.

Also a good point. We had previously discussed allowing manifest files for extensions to open up some new features for them so I thought this approach would get us on the right track to move that direction. The lazy-loading fix you are creating here does not impede that potential future direction though, so no concerns from me there.

[mislav]

@samcoe Could you see to finishing the lazy-loading aspect? Basically, the parts that I've commented out in this branch should run on request. Or feel free to go with your own approach if that's simpler. Thanks!

[samcoe]

This is a slight change in behavior, where we will now output a more user friendly error message but with less insight into what might have failed while retrieving the latest version.

[vilmibm]

Tested it locally and it worked well. I think this is a good step forward in general.