VSCode various random issues when loading extensions from gitpod.yml

[szab100]

Bug description

I have the following list of extensions to be loaded for a spring-boot project that is using Lombok, so needs pre-processing before Java compilation.

vscode:
  extensions:
    - gabrielbb.vscode-lombok
    - vscjava.vscode-java-pack
    - vscjava.vscode-java-debug
    - vscjava.vscode-java-dependency
    - vscjava.vscode-java-test
    - vscjava.vscode-maven
    - pivotal.vscode-boot-dev-pack
    - rangav.vscode-thunder-client
    - ms-azuretools.vscode-docker

The experience seems quite random, depending on whether the Lombok extension was loaded before / after the vscode-java-pack extension, it sometimes prompts the user twice with the following popups:

• Java Extension Contributions changed, reloading Gitpod Code is required for the changes to take effect. Source: Language Support for Java(TM) by Red Hat (Extension) - User needs to click "Reload"
• Java Language Server configuration changed, please restart Gitpod Code. - User needs to click "Restart Now

However, sometimes these popups do not appear, but the loaded java projects have many errors due to the missing Lombok pre-processing step. Since the popup was not shown (same if the users misses them), the only way to resolve is by refreshing the entire page.

It looks to me that the problem itself may not be with this particular extension (that hooks up to other extensions' events, config etc), but how Gitpod loads and initializes the VSCode extensions in general (I found similar random timing / order / side-effect issues with other extensions too).

If that is the case, I believe there could be further improvements made in order to provide a stable, dependable experience, like allowing users to specify:

• ordering for the listed extensions (if the YAML sequence order is not preserved)
• a certain delay or dependencies on other extension(s) - so that extensions can be loaded in parallel by default, but those having dependencies must be loaded strictly after the deps are fully loaded
• actions to be performed between extensions [like restarting a component OR run a shell cmd]

Please let me know what you think is the best solution for these problems.

Steps to reproduce

described in description

Workspace affected

No response

Expected behavior

No response

Example repository

No response

Anything else?

No response

[jeanp413]

Hi @szab100, extensions activate depending on the events listed in the activationEvents entry in package.json, there's no way to specify the ordering, unless some extension requires some exposed API from another extension and request for it.
In the case of the Lombok extension I took a look at the source code and what it does it only installs the jar file (it also updates java.jdt.ls.vmargs setting but that only affects the java language server spawned by the java extension which is for the language smartness and the reason of the popups you see), I think just adding the lombok library as a dependency if you are using maven or gradle should be enough, if not it would be really helpful if you have a public repo where I can reproduce this

[akosyakov]

@szab100 I think you don't need the lombok extension which is not maintained for one year and has severe issues which can break Java integration like microsoft/vscode-lombok#49

This extension also does only 2 things:

• installs lombok
• and configures java.jdt.ls.vmargs to point to it that Java language server can pick it up

Instead in a Docker image for your project you can install lombok and then configure java.jdt.ls.vmargs VS Code settings on project level to point to it like -javaagent:/gitpod/workspace/deps/lombok.jar.

It will ensure that jar file is already there when you start a workspace and not downloaded in the meantime, so Java lang server won't crash and will pick up lombok immediately without restart/page reload.

[szab100]

@akosyakov yes, thank you, your suggested workaround seems to be working fine (fetching lombok.jar and add the agent to vmargs).

On the other hand, sometimes I see other issues that are seemingly happening because of how VSCode loads extensions. I understand there is no out-of-the-box way with VSCode to specify installation order or making sure an extension only gets installed if a set of extensions are already installed / loaded up.

Such an example is this (among other similar conflicts between extensions):

with the extension list defined in this order:
- vscjava.vscode-java-pack
- vscjava.vscode-java-debug
- vscjava.vscode-java-dependency
- vscjava.vscode-java-test
- vscjava.vscode-maven

What I see in the code is that currently Gitpod is simply iterating over the list of user-specified list of extensions (in order) and builds the vscode server command by adding each with "--install-extension <path_or_url>" that it executes once and lets VSCode to figure the order / dependency graph etc.

The approach I suggested here is that when the user specifies a certain order, or loading group, eg something like:

vscode:
  extensions:
    -
      id: vscjava.vscode-java-pack
      group: 1
      timeout: 3000
    - 
      id: rangav.vscode-thunder-client
      group: 1
    - 
      id: pivotal.vscode-boot-dev-pack
      group: 2
      group-timeout: 5000

And the codehelper component could first run:

• gitpod-code --install-extension vscjava.vscode-java-pack --install-extension rangav.vscode-thunder-client

..then it could keep checking the installed extensions using gitpod-code --list-extensions --show-versions, and only build & run the command with '--install-extension' for subsequent load-groups' extensions (in this example, the Pivotal Spring Boot extension) when all extensions from the previous group(s) are already fully installed or prompt the user with an error message if installing some extension(s) failed within the specified timeout.

Therefore after it verifies the 1st group's extensions are all installed (and the java-pack installed within 3000ms), it will then execute:

• gitpod-code --install-extension pivotal.vscode-boot-dev-pack

So in other words, implement this logic on Gitpod side by calling Code's existing dummy (list-installed + install) API, in case you guys are also experiencing timing issues due to how extension-loading is done today.

[akosyakov]

I don't think it is going to help, errors come from loading/activation which happen on each page reload, not only on first install. Besides it sounds like working around issues with extensions. I think we should rather investigate and reach out to authors about concrete issues. Both java debug and test extensions are well-maintained. @jeanp413 could you check when it happens and file issues for corresponding extensions? 🙏

[jeanp413]

After investigation it's a vscode bug, created an upstream issue microsoft/vscode#149309

[akosyakov]

@jeanp413 Would you be able to have a look into fix for this please? 🙏

[akosyakov]

I wonder whether we can propose some workaround for now. Maybe using Gitpod tasks and calling code cli from it? Or it can cause race conditions as well?

Another idea what if we create an pack extension which bundles vscjava.vscode-java-pack and pivotal.vscode-boot-dev-pack? Would it resolve the issue then? if it is so... we could derive one syntetic pack and put all deps in it and then only call code to install it 😬 or VS Code should be changed to apply the same logic to install all listed extensions as one extension pack

[akosyakov]

@jeanp413 Just want to ask a stupid question before we go on with creating something like spring java extension pack. Will explicitly adding java redhat extension as first in .gitpod.yml resolve the issue instead?

[jeanp413]

Not really as all extensions are installed in parallel, if it finish before because it takes less time to download or some other factor then yes.

[jeanp413]

@szab100 we were discussing a workaround that would consist on making a extension pack with the extensions that has dependencies in this case vscjava.vscode-java-pack and pivotal.vscode-boot-dev-pack as both depend on redhat.java, this way all extension in the extension pack will be awaited together to finish installation.
For publishing this new extension to open-vsx you would need to create an Eclipse account and follow this guide https://github.com/eclipse/openvsx/wiki/Publishing-Extensions
I will create a first version of the extension pack as a template for you to test it but in the end you'll have to manage it yourself so you are not blocked by us if you need to add/remove a new extension in the extension pack.
Let me know if you like this idea.

[jeanp413]

@szab100 I created jeanp413.test-java-extension-pack that contains vscjava.vscode-java-pack and pivotal.vscode-spring-boot.
Can you add it to your gitpod.yml (remove vscjava.vscode-java-pack and pivotal.vscode-spring-boot) and test that you don't have to reload after java test extension is installed?

[akosyakov]

@szab100 Was the pack helpful? We would like to understand whether you have anything blocking.

[szab100]

Hey @akosyakov @jeanp413, sorry for the delay. I have not tried the pack yet, it should be good as a workaround, but I am rather looking to get the underlying vscode issue properly fixed for the long run.

In general, this problem can be solved by running a topological sort of the extensions (based on their deps) initially and installing them in the resulting partitions (extensions on same levels can be installed in parallel), but this should give a slightly less optimal solution than the current greedy approach extended with the correct 'await' fix on the dependent deps' installation task(s) as suggested by @jeanp413 on the vscode issue, which should only hold up the installation of extensions with deps currently being installed, rather than blocking all extensions from the remaining partition(s) before the current partition is fully completed.

@jeanp413 - as I understand, your fix does not work yet as expected. This might be a silly question, but have you tried placing the blocking 'await' statement(s) before the main installation loop? As it seems to me that the install loop is where it fails to install the dependent extension if any of its dependencies' tasks are not being waited on to complete before attempting to install. If this is the case, maybe we do not even need to collect these tasks into 'externalRequestedInstallExtensionTasks', but simply put await this.installingExtensions.get(key)?.waitUntilTaskIsFinished(); after this line? I am not quite an expert of JS Promises so this may be completely bad, sorry.

Either way, I tried to reproduce this by building vscode from source, enabling gallery services & running ./scripts/code-server.sh --launch, then ./scripts/code-server.sh --install-extension pivotal.vscode-spring-boot --install-extension vscjava.vscode-java-pack from the terminal, but each time I tried (installing, then uninstalling them), both extensions were installed successfully. Maybe this is not happening on all platforms (tested on os-x / m1 pro) OR it is happening only when the IDE loads up a project when the 1st extension is getting invoked before the 2nd extension (and their common dependency) is fully installed?

[jeanp413]

Hey @szab100, as you pointed out, proper fix would be doing a topological sort, but that would need a medium size refactor of the logic so I was kinda hesitant doing a PR implementing it and also they triaged it as backlog issue.

From the test I did with vscjava.vscode-java-pack and pivotal.vscode-spring-boot the changes I did fixes the issue, but it didn't feel like clean fix 😅. I'll go ahead and create a PR with my proposed changes either way.

It would be help if you give 👍 on the upstream issue and comment so it gets more attention.

PS: to reproduce the issue the extensions need to finish installation after the workbench is ready, in production this is the usual case as it's minified into a single file and downloaded even before the server starts, when you try to reproduce from sources you launch the server first and then the browser has to download all source files one by one, as this takes some time, the extensions already finished installing, so you need to do some additional changes (check remoteAgentEnvironmentImpl.ts file) in source code to get the same behavior as in production

[akosyakov]

Hey @akosyakov @jeanp413, sorry for the delay. I have not tried the pack yet, it should be good as a workaround, but I am rather looking to get the underlying vscode issue properly fixed for the long run.

Yes, then someone needs to contribute it to upstream. It is non trivial bug which most likely happens in very rare conditions, i.e. here is at play also how these extensions depend on each other, for other dependent extensions it is not necessary an issue at all. Otherwise we would see already many complaints in upstream.

I would be glad if someone to contribute to upstream, but to unblock you: could you please check the extension pack from @jeanp413 in your setup? if it works for Java we can create spring java pack, maintain it and put in docs for time being. Till VS Code team addresses the issue.

Btw you don't need to list all extensions just:

vscode:
  extensions:
    - jeanp413.test-java-extension-pack
    - rangav.vscode-thunder-client
    - ms-azuretools.vscode-docker

[jeanp413]

This got fixed upstream microsoft/vscode#159822, and is already available in vscode web insiders, will be available in next stable release 1.72.0