Remove plugins from the Devfile schema

[l0rd]

Is your enhancement related to a problem? Please describe.

Devfile plugins were designed to specify a dev environment feature (java support for example) that would be independent of the particular IDE/editor used in the workspace.

After a couple of years we have observed that most of those plugins are Che Theia plugins and work only if Theia is the editor.

A first time user will probably assume the following devfile, where the Java plugin is specified but the editor is not, will work no matter what's the IDE/editor. In other words that Che will manage to include the right plugin for the editor that's choosed by the user:

apiVersion: 1.0.0
metadata:
  generateName: petclinic
components:
  - id: redhat/java11/latest
    type: chePlugin
    alias: java

But the truth is that will only work if the editor is Che-Theia. That's confusing for users that read the devfile and other projects that want to adopt the devfile spec and would need to support the devfile above somehow.

The devfile API is not adapted to support the specification of Che-Theia plugins yet (plugins of one devfile component).

There are a few plugins that still make sense as Che plugins as: editors, exec, telemetry, async volumes support. But those plugins are usually not included in devfiles, they are implicitly added by Che at workspace startup time.

Draft PR

Related issues

• Che Editors and Che Theia plugins support in Devfile v2
• Provide plugins recommendations
• Plugins and editors inline definitions in devfiles
• Use DevWorkspaceTemplates to specify a Che editor or other sidecar based plugins. The DevWorkspaceTemplate can be for just one workspace of a user or for every workspace of a user.

Describe alternatives you've considered

Using CustomComponents as mentioned in this example.

Additional context

This work should be included in the context of che workspaces that use the DevWorkspace operator, so that's not for v1 devfiles/plugins.

[tsmaeder]

@l0rd since plugins to che-theia will still be needed, what replacement are you proposing?

[benoitf]

@tsmaeder first bullet of the description: " Use extensions.json and che-theia-plugins.yaml to specify recommended vscode extensions and che-theia plugins"

[ericwill]

A devfile plugin is supposed to be a universal (cross-editors) specification of a development environment feature. But today most of the plugins in our registry are Che Theia plugins and won't work without Theia.

Is there somewhere that outlines the devfile 2.0 effort? Which other editors are looking at adopting the format? I found https://redhat-developer.github.io/devfile/ but it's v1, and looks to be Che specific.

[benoitf]

https://docs.devfile.io/devfile/2.0.0/user-guide/index.html

[tsmaeder]

https://docs.devfile.io/devfile/2.0.0/user-guide/adding-plugin-component-to-a-devfile.html talks about devfile plugins, but then uses the java che-theia plugin as an example and even has theia preference values. In the call yesterday I got the impression that editor-specific plugins and content had no more place in the devfile? At least not as part of the specification.

[l0rd]

docs.devfile.io/devfile/2.0.0/user-guide/adding-plugin-component-to-a-devfile.html talks about devfile plugins, but then uses the java che-theia plugin as an example and even has theia preference values. In the call yesterday I got the impression that editor-specific plugins and content had no more place in the devfile? At least not as part of the specification.

devfile 2.0.0 spec includes a "plugins" component type, this issue is about removing it (see 3rd checkbox in the description "Remove the plugin type component from the devfile spec").

[l0rd]

Which other editors are looking at adopting the format?

@ericwill currently the tools that are adopting the format are odo and OpenShift Dev Console. Those are not editors. With a Devfiles a developer can specify a development environment that includes (or not) an editor. With the v2 of the devfile and odo adoption, it's obvious that devfile is not supposed to specify an editor configuration. It can specify it but the editor is just one component amongst others.

The change that we are proposing is based on the observation that editors already have their conventions to specify plugins/commands. And it has more value for users that we support those conventions rather than implement our own universal editor plugin spec.

[tsmaeder]

editors already have their conventions to specify plugins/commands

As I mentioned in the call yesterday, I think we still need some engine to bind the editor specific plugins and configuration into the workspace, for example, starting the appropriate sidecar containers, etc. Like it or not, having an IDE (aka "editor") in the browser is the main value proposition of Eclipse Che. Until we have a story there, I'm don't think we should move forward with devfile 2.0 in Che.

[mickaelistria]

The devfile could list IDE extensions for the possible editors projects want to target: cheTheia would reference che plugins (when container/sidecar is necessary), theia would just use vsix URLs, eclipse would reference marketplace entries, intellij would reference some IJ marketplace entries...
With that, people using this devfile with "Import project from devfile" inside other IDEs could be prompted of some plugins to install just like they'd be prompted to clone Git repositories or to fetch the container that maps the production environment; and che would still be able to directly prepare the IDEs according to devfile content.

[ericwill]

Which other editors are looking at adopting the format?

@ericwill currently the tools that are adopting the format are odo and OpenShift Dev Console. Those are not editors. With a Devfiles a developer can specify a development environment that includes (or not) an editor. With the v2 of the devfile and odo adoption, it's obvious that devfile is not supposed to specify an editor configuration. It can specify it but the editor is just one component amongst others.

That's fine, but it seems illogical to me that we are removing features from Che's workflow just because other tooling using the devfile format doesn't support them. Especially when such changes aren't user friendly.

It seems there is already an established plugin format for devfile 2.0. Why can't we use that, and tools like odo that don't use plugins can just ignore those components/sections? I imagine it's easier for tools like odo to disregard components of a certain type, than it is for Che to come up with this complex plugin recommendations/override system.

I also like what @mickaelistria suggested, where the type field can reflect the IDE being used. It allows for the creation of one central devfile, which is shareable across multiple IDE's. Each individual IDE only looks for the components that match its type.

[l0rd]

@mickaelistria exact! But we currently do not have that in the spec: plugin components do not have a subtype. So we had 2 alternatives: modify the devfile spec and introduce plugin subtypes OR put that on hold and remove IDE specific configuration out of the devfile spec. Reasons why I think the latter is better:

• we should be careful adding new features to the API: it may be impossible to remove them in the future (and if instead we don't add them now we can still do that later)
• we should support file formats in .vscode, .idea folders anyway and the data in those folders overlaps with the data in the devfile plugins
• devfile has been adopted by other tools that do not have an IDE components and thus the plugin IDE abstraction doesn't work well for them.

[mickaelistria]

I agree not specifying is indeed the best way to not doing something wrong. But on the other hand, not specifying also reduces the likelihood for other IDEs to look at that format. I think it's not so much a technical discussion here but really more a discussion about what are the main goals of the devfile and the plans and priority to achieve that goals.

[l0rd]

Yes it's a matter of priorities. I would like to put that on hold, the adoption of the devfile by other IDEs is not a priority whereas a better support of vscode and idea configuration have higher priority.

[tsmaeder]

So we had 2 alternatives: modify the devfile spec and introduce plugin subtypes OR put that on hold and remove IDE specific configuration out of the devfile spec.

Alternative number 3 would be to have a standard way for tools to extend the devfile. VS Code extensions, for example can extend the VS Code preferences schema with their own preference schemas.

[azatsarynnyy]

On one hand, I agree that it looks weird removing plugins from the Devfile specification. To me, it looks more like a step back for Che, just to make Devfile adoption a bit easier for other tools, like odo.

On the other hand, it sounds much better comparing to synchronizing a Devfile content with all the possible IDE-specific files, like extensions.json, tasks.json, launch.json, etc. See #12709 that we've been discussing at the beginning.
But such a step is required if we have all this stuff in a Devfile.

If we choose from the two options:
1/ implement the IDE-specific plug-ins for handling the Che-specific cases, like restarting a workspace for adding the recommended plugins and so on
2/ implement the IDE-specific synchronizations for Devfile <-> extensions/tasks/launch.json

from the editor's perspective, I'd say that the 1'st looks much better, less confusing and error-prone than the 2'nd one.

[l0rd]

A recurring question is about editors: how are we going to specify the editor that should be included in the workspace?

Most of the devfiles v1 out there don’t have an editor explicitly specified in it. When Che transform it in a workspace it appends an editor to the original devfile definition. That won’t change in devfile v2: when the devfile gets transformed into a DevWorkspace, the editor will be appended.

What about devfiles with an editor explicitly specified in it? In devfile v1 we could add it as a component of type plugin. In devfile v2 it is still possible using attribute che-editor.yaml:

schemaVersion: 2.0.0
attributes:
  che-editor.yaml:
    <yaml content>

That allows to include references to editors defined in a che-plugin registry as well as inline definitions. Syntax details are in #18668

[amisevsk]

What about devfiles with an editor explicitly specified in it? In devfile v1 we could add it as a component of type plugin. In devfile v2 it is still possible using attribute che-editor.yaml:

schemaVersion: 2.0.0
attributes:
  che-editor.yaml:
    <yaml content>

I can understand this, but I'm not sure I'm on-board here. On one hand, we have

• Editors defined as DevWorkspaceTemplates, and can be referred to via Plugin components or inlined directly into a devfile (since they're ultimately just components, etc.)
  • This mechanism likely has to exist anyways for DevWorkspaces (if not devfiles themselves)
• Editors defined in attributes, where basically the same spec lives but in an unvalidated and harder-to-process way

  • For referring to a plugin registry, we basically have the spec used for plugin components, but unvalidated

  • For inlining, calling it <yaml content> is downplaying how infeasable the task actually is. Inlining Theia would mean inlining the following yaml

    theia.yaml

      - id: eclipse/che-theia/next
        title: Eclipse Theia development version.
        displayName: theia-ide
        description: Eclipse Theia, get the latest release each day.
        icon: https://raw.githubusercontent.com/theia-ide/theia/master/logo/theia-logo-no-text-black.svg?sanitize=true
        repository: https://github.com/eclipse/che-theia
        firstPublicationDate: "2019-03-07"
        endpoints:
          - name: "theia"
            public: true
            targetPort: 3100
            attributes:
              protocol: http
              type: ide
              secure: true
              cookiesAuthEnabled: true
              discoverable: false
          - name: "webviews"
            public: true
            targetPort: 3100
            attributes:
              protocol: http
              type: webview
              secure: true
              cookiesAuthEnabled: true
              discoverable: false
              unique: true
          - name: "mini-browser"
            public: true
            targetPort: 3100
            attributes:
              protocol: http
              type: mini-browser
              secure: true
              cookiesAuthEnabled: true
              discoverable: false
              unique: true
          - name: "theia-dev"
            public: true
            targetPort: 3130
            attributes:
              protocol: http
              type: ide-dev
              discoverable: false
          - name: "theia-redirect-1"
            public: true
            targetPort: 13131
            attributes:
              protocol: http
              discoverable: false
          - name: "theia-redirect-2"
            public: true
            targetPort: 13132
            attributes:
              protocol: http
              discoverable: false
          - name: "theia-redirect-3"
            public: true
            targetPort: 13133
            attributes:
              protocol: http
              discoverable: false
        containers:
          - name: theia-ide
            image: "quay.io/eclipse/che-theia:next"
            env:
              - name: THEIA_PLUGINS
                value: local-dir:///plugins
              - name: HOSTED_PLUGIN_HOSTNAME
                value: 0.0.0.0
              - name: HOSTED_PLUGIN_PORT
                value: "3130"
              - name: THEIA_HOST
                value: 127.0.0.1
            volumeMounts:
              - name: plugins
                path: "/plugins"
              - name: theia-local
                path: "/home/theia/.theia"
            mountSources: true
            ports:
              - exposedPort: 3100
              - exposedPort: 3130
              - exposedPort: 13131
              - exposedPort: 13132
              - exposedPort: 13133
            memoryLimit: "512M"
            cpuLimit: 1500m
            cpuRequest: 100m
        initContainers:
          - name: remote-runtime-injector
            image: "quay.io/eclipse/che-theia-endpoint-runtime-binary:next"
            volumeMounts:
              - name: remote-endpoint
                path: "/remote-endpoint"
                ephemeral: true
            env:
              - name: PLUGIN_REMOTE_ENDPOINT_EXECUTABLE
                value: /remote-endpoint/plugin-remote-endpoint
              - name: REMOTE_ENDPOINT_VOLUME_NAME
                value: remote-endpoint

  except without any help from your editor.

[l0rd]

Closing this issue since it's already tracked in the devfile/api repository devfile/api#407.