Support Platform Specific Extensions

[windyGex]

I can't find answer from stack overflow, so I open a issue.

[auchenberg]

In your extension activation callback, you are free to check and download extensions as soon as the extension get's activated. This is how Live Share is downloading their dependencies after installation

export async function activate(context: vscode.ExtensionContext) {
  /// 1 Check if deps are installed
 await downloadDependencies()
 // Update status bar or output channel to give an indicator of status
}

[windyGex]

If install dependencies when extension activate, I think it' too later and it maybe cost a long time.

[sandy081]

Idea is to provide a post install hook similar to uninstall hook. This will be executed after an extension is installed or updated.

"scripts": {
    "vscode:prepublish": "tsc -p ./",
	"vscode:install": "node ./out/src/extensionInstall"
    "vscode:uninstall": "node ./out/src/extensionUninstall"
},

An extension can install necessary libraries and stuff using this post install script. An unique location per extension is provided through an env variable for the extension to maintain any state. The same location will also be passed with post uninstall script.

[DonJayamanne]

Idea is to provide a post install hook similar to uninstall hook. This will be executed after an extension is installed or updated.

I would prefer a better name. Today vscode:prepublish applies to developer of the extension, i.e. to extension authors. Having vscode: install would indicate it's related to extension authors again and not in end user machines.

[rchande]

@sandy081 Thanks for pointing this feature out to us on the C# extension. Our dependency downloading story is actually a little complicated. Each version of the extension ships with built in URLs to specific versions of binary dependencies, and, if the user has no options set, we'll download them on first run. However, we allow users to control what version of our dependencies they download. You can see this in action by playing with the option "omnisharp.path". If you:

• Set it to latest: When the extension starts, it will check online to determine the latest version of OmniSharp, download it if we haven't downloaded it before, and start that version. Each new version gets downloaded to its own folder.
• Set it to a specific version number: When starting OmniSharp, the extension will attempt to start it from a directory based on the version you specified. If it's not available, it will try to download it using a well known version -> url mapping.

Basically, we need to be able to access our "unique location per extension" after install-time to read and write values. Would the new API support that?

[gregg-miskelly]

@sandy081 In addition to what Ravi said, I would first off like to say a thanks. I have been wanting this feature for so long. So I am very excited to see it happening!

Two other notes --

1. I am not sure what kind of UI interactions you are expecting to allow between the post install script and VS Code, but we would need to somehow inform the user of progress since downloads can be somewhat large, at least for users with very slow internet connections.
2. I would expect the most common action that an extension to want to do would be to install dependencies. However, with proxy servers, downloading files is not that easy of a task to get right. It would be great if there was some sort of 'DownloadFile' API or 'vscode-DownloadFile' npm package that was provided to extensions that would handle this task in the same way that VS Code does. In case it matters, as Ravi said, I think we would want some of our downloads to continue to go to the per-extension-version directory (because they change with essentially every version), while some of our downloads are stable enough that we might want to share them. So we would need to be able to specify the destination directory in such a way that either place could be targeted.

Here is the C# extension "download dependencies" code as a reference:

• Code to gather proxy settings: https://github.com/OmniSharp/omnisharp-vscode/blob/master/src/NetworkSettings.ts
• Code to download files: https://github.com/OmniSharp/omnisharp-vscode/blob/master/src/packageManager/FileDownloader.ts#L44

[sandy081]

@DonJayamanne

I would prefer a better name. Today vscode:prepublish applies to developer of the extension, i.e. to extension authors. Having vscode: install would indicate it's related to extension authors again and not in end user machines.

vscode:install is related to extension author but it impacts end user machines at the end. For eg: downloading and installing libraries.

Going with vscode:install because we already have a post uninstall hook called vscode:uninstall.

[sandy081]

@rchande @gregg-miskelly

Thanks for your feedback. Post install script will be given access to a unique location per extension (not per version). This location is provided to the script through process environment variable VSCODE_EXTENSION_STORAGE_LOCATION. You can update your post install script to download the dependencies matching the current version of extension being installed. You can use the above location to download and check the versions that are already downloaded.

I do not think this script will be given access to configuration API. So if you have user configurable versions, then you can download the specific version during run time by checking in the above location.

but we would need to somehow inform the user of progress since downloads can be somewhat large, at least for users with very slow internet connections.

Post install step runs as part of extension installation, so the extension installation progress in the extension view shows it. Logs show what step (eg: post install) is being running now.

It would be great if there was some sort of 'DownloadFile' API or 'vscode-DownloadFile' npm package that was provided to extensions

I am completely aware of this and we are working to come up with a solution for this. Hence not including this here.

Let me know if this helps.

[gregg-miskelly]

Post install script will be given access to a unique location per extension (not per version)

So you are saying that extensions will only have access to the per-extension location? If so, I would suggest changing the design to allow extensions to install to either place. Alternatively, if the 'DownloadFiles' API was really more a dependency management API where VS Code took care of removing outdated dependencies along with downloading the new ones, that could definitely work.

Post install step runs as part of extension installation, so the extension installation progress in the extension view shows it. Logs show what step (eg: post install) is being running now.

Are you saying it will only show that it is on the 'post install' step and give no more granular progress than that? Or are you saying it will show that it is on the 'post install' step, and there will be some ability to see with more detail (ex: click on that to bring up an output window pane). If the former, I don't think that will be good enough unless the forthcoming 'DownloadFile' API solves this problem.

[gregg-miskelly]

BTW: I would like to throw out one other idea here. Instead of designing a bit of code that needs to run, really the missing feature here is that extensions can't tell VS Code about all of their platform-specific dependencies. So another way to solve this would be to make post-install hooks a feature that is normally implemented with a tiny bit of javascript that just returns (ex: by writing JSON to stdout) the set of packages that the extension needs. This way extensions don't need to all maintain a bunch of code to do this, VS Code can hopefully just reuse most of the existing extension download and extension file management code, and VS Code doesn't need to allow any of its normal APIs to work in these post-install hooks.

Example of how an extension might declare its dependencies:

{
  "dependencies": [
    {
      // The description would be used in the progress indicator
      "description": ".NET Core Debugger (macOS / x64)",
      // The id+version would be used to select the install directory and determine if the package was already downloaded
      "id": "vsdbg-ui",
      "version": "1.17.1",
      // URL which points to a .zip file (or other archieve formats if you want) to download from
      "url": "https://download.visualstudio.microsoft.com/download/pr/90e2038c-960d-4018-924e-30f520f887ab/2d53db027d1e67b899a7f083800b2bd4/coreclr-debug-osx-x64.zip",
      // The optional fallback URL is used if downloading from the primary URL fails
      "fallbackUrl": "https://vsdebugger.blob.core.windows.net/coreclr-debug-1-17-1/coreclr-debug-osx-x64.zip",
      // The optional hash is used to verify the file was correctly downloaded
      "sha512hash": "1234125215215..."
    },
    {
      "description": "OmniSharp for OSX",
      "id": "OmniSharp",
      "version": "1.32.6",
      "url": "https://download.visualstudio.microsoft.com/download/pr/a8501982-f057-4675-bf2f-57e4aa9f0990/f175b994e2c0a97ea7b874dafec7f303/omnisharp-osx-1.32.6.zip",
      "fallbackUrl": "https://omnisharpdownload.blob.core.windows.net/ext/omnisharp-osx-1.32.6.zip"
    }
  ]
}

[srivatsn]

+1 to @gregg-miskelly's idea above of making it declarative. An additional benefit of this could be that these dependencies could also be cached in the temp folder. That way, if the extension has a newer version but the dependencies don't change (like for LiveShare where we download .NET Core), then the update could be much faster as we'd just reuse the cached download on that machine.

[sandy081]

I do not agree letting the platform (VS Code) to do the dependency or resource management for the extensions is the right approach. It is hard for the platform to understand and provide an API that will fulfil all kinds of resource management requirements. Hence such an extension point will be incomplete always. So, I think extensions are the right ones to do their dependencies management and we provide a location where you can cache them and maintain the state.

I see that the dependencies approach is trying to solve following problems

1. Showing Progress

I agree as an an end user it is important to know

• An extension is being installed
• A post installation process is going on
• Installation is finished or failed

If user wants to know more about what is happening during post installation, user can see it in the logs. It's same with extension installation that the shared process log captures all details.

One idea is that post install scripts logs their activity and VS Code show a status bar entry showing Installing Extension C#.... When user clicks on this entry, the installation log will be shown.

2. Proxy issues

I think this is a general issue and exposing the dependencies API will not help all cases. As said, we are working on it to provide a general solution that helps in all cases.

[avanderhoorn]

@sandy081 I think your statement above is true if you look at the 100% case, but there is a lot of value to be gained in solving the 80-95% case. I think you might find that the declarative API described by @gregg-miskelly might meet the majority of cases. Offline of this thread we have multiple discussions with @kieferrm and @egamma about this and I believe that the general consensus was that there would be value in providing something like this out of the box. I do agree that there are some underlying issues here (i.e. showing progress in a common way, proxy support, etc) but thats part of the reason that this issue exists. Lastly, to help get the declarative API to the point where it supports more cases, I don't see why there couldn't be strategic callbacks that people can tap to help direct things.

[sandy081]

@avanderhoorn If I am not wrong the reason for downloadable resources extension point proposal was to overcome the proxy issue otherwise the post install script extension point is the better way for the extensions to download / manage resources. Also, this resources extension point is not a complete and generic solution as this will not help extensions if they want to download during run time.

Again, we are doing some investigation and research on how to provide a generic solution for proxy issues. If that yields positive results the resources extension point might be redundant.

[sandy081]

Tagging for api-proposal because this needs to expose globalStorage api in vscode.ExtensionContext.

[rchande]

@sandy081 What's the rationale for providing a post-install script file as opposed to a javascript callback (as @gregg-miskelly suggested)? If extensions provide a post-install callback that calls a standard VS Code downloading API, you can make progress reporting very consistent.

[sandy081]

@rchande post-install script is going to be a node script that gets executed after installing the extension. VS Code API is available only for running extensions and hence the newly installed extension does not has access to it as it is not running. A new API has to be defined if post-install script wants to show some UI elements which we want to avoid.

Summarising the discussion:

Proposal: Post Install Script

• An extension can participate in its post installing step by providing the node script entry point using vscode:install key in scripts section of package.json
• Extension can log (console) the activities (like downloading or installing) it is doing during post install step
• VS Code shows a status entry for post install step and when user clicks on it, logs are shown. I think Live Share extension is doing a similar approach by showing the status while downloading resources.

Proxy Issues

• Under investigation for a general solution

Proposal: Resources Contribution Point

Given that Proxy issue is being investigated and if we arrived to a generic solution, Post Install Script option is a better solution.

Please let me know if this makes sense or are there any changes suggested?

[prashantvc]

@testforstephen

I'm trying to figure out how the universal and platform versions can coexist, so i used vsce@1.99 tool to publish multiple versions for a test extension

No, the universal and platform versions cannot coexist for the same version of an extension

Issue 1

Is this for same version of an extension? Can you give an example, please?

Issue 2

Yeah this looks like a bug. Could you file it on vsce repo?

[isidorn]

@tboby @testforstephen @Eskibear thanks for giving this a try and for providing feedback 👏
I believe Prashant answered the majority of the questions.

We do not recommend to have platform specific vsix and universal vsix for the same version. Instead we recommend that for a version an extension would publish a platform specific package for each platform (even if vsix is shared across different platforms). If this is too much manual work let us know (we are already looking into improve the flow in vsce for this).

Only @Eskibear issue was not covered by Prashat: @sandy081 is working on a fix and will probably push a fix for this soon.

[gregg-miskelly]

Thanks so much for implementing this. I am very excited!

Two questions:

1. Is there any limit on the size of a .vsix that we need to be aware of?
2. If we use platform-specific .vsix's for our official releases, but still use platform-agnostic .vsix's for beta releases (which would have a higher version number) is that going to be a problem?

[felipecrs]

If this is too much manual work let us know (we are already looking into improve the flow in vsce for this).

I think it's the case. For example, I use Semantic Release to manage the versioning of the ShellCheck extension through semantic-release-vsce. I cannot even think of a way of handling two versions at once for the platform specific and not platform specific extension with Semantic Release.

Not to say that having two different versions for the same codebase is a bit misleading. Ideally the "universal/non-platform-specific" should have the same version, so that the Git tag that we create for each release would reflect the true for both cases.

Also, there is the CHANGELOG maintenance. Having to update two entries, as the versions will differ is also cumbersome.

[sandy081]

Thanks for feedback. Fixed following:

• VS Code will now install the most suitable vsix
• Fixed the version duplication

[isidorn]

@gregg-miskelly

1. Current limit per platform package is 200mb. This is not set in stone, we can increase if we hear from you it is needed.
2. This is not going to be a problem. We always take the highest version which is compatible. However please be aware that we are looking into improving the extension insider flow and we have a proposal here. This would work very well with Platform Specific Extension structure

@felipecrs this is why we recommend automating this flow. Though we will also try to improve the manual flow as well. I will keep you updated. Thanks for feedback.

[jakebailey]

I would also think it would be a good idea to offer up a "generic" platform that's a fallback if you haven't published the right one. I don't think it's good for extensions to have to keep track of every platform VS Code could support (present and future) and ensure they publish versions for each of them. There's always the case that a new platform (say, darwin arm64) appears, then some older extension can no longer be used because it won't be installed unless you have the platform available.

But, I know this (and the insiders versioning) is sort of limited by what the marketplace can do.

[isidorn]

@jakebailey this is a good suggestion, however we are a bit limited due to marketplace support.
We should keep this in mind and potentially improve later, but this is not in the cards for the near future.

[gregg-miskelly]

1. Current limit per platform package is 200mb. This is not set in stone, we can increase if we hear from you it is needed.

Thanks. For the C# extension, have one platform (darwin-arm64) where we are fairly close to the limit -- 198425276 bytes (189.2 MB) in my quick test. I can do some experiments to see if we can reduce this a bit. But we have have an ask to raise this slightly.

[isidorn]

@gregg-miskelly ok, so it seems like we are good for now but I have notified the Marketplace that we would like to increase the size limit. Once they do that I will let you know.

[isidorn]

As I mentioned above
We now support the full flow for platform specific extensions 🚀
This document has all the details:
https://code.visualstudio.com/api/working-with-extensions/publishing-extension#platformspecific-extensions

Thus I am closing this issue. If there are questions feel free to ask, we will continue monitoring this issue even though it is closed. Thank you very much 👏

[felipecrs]

Just a question, because I didn't understand, was the thing about having to release different versions for non-platform specific and platform specific extensions solved?

I.e. can I release a amd64 and a non-platform specific .vsix targetting the same version?

[isidorn]

@felipecrs yes you can, but you need to publish your amd64 vsix for amd64, and your general one for each other platform. This is an issue to help make this flow smoother microsoft/vscode-vsce#625

[felipecrs]

@isidorn thank you!

[valentjn]

@isidorn When Marketplace increases the size limit, is this announced in this issue or in another? Or is this documented somewhere? I can't find the info about the 200MB anywhere else than here (the previous limit of 100MB is also hard to find, it's buried in some old release notes of VS Code).

Whether it's 200MB or say 250MB or 300MB makes a big difference for my extension with respect to migrating to platform-specific extensions, but I don't know a way of finding out the current limit other than trying to publish and see what happens. Obviously, at that point, a lot of effort has been invested, in vain if the limit is lower than expected.

[prashantvc]

@valentjn The size limit is not documented anywhere. Right now, we have 200MB limit, however we allow larger extension on need bases.

[felipecrs]

PS: I opened an issue to VS Marketplace to support generic platform, so that extension authors would not need to manage their own list of all platforms.

microsoft/vsmarketplace#219