Add keepDir support for Devfiles

[mkuznyetsov]

Che 6 factories used to support "keepDir" param to specify a subfolder of a project repository.
Devfile model currently does not support this.

To support it on Che side, it requires to change the model, schema, DB. For Che 7 Theia based workspaces, would require to add support for Theia as well.

[sleshchenko]

It may be related somehow to #13688

[sunix]

What was the goal of keepDir in Che6 ? is this still accurate with che-theia ?
As far as I remember, keepdir was used because we had that 'project' model where 1 git project had to be 1 che project. I am not sure this is still relevant in che-theia: would be useful for the end user in real projects. ... but for samples and demos?
What is the value for the users ? in which use cases ?

[skabashnyuk]

@sunix I believe that idea is quite simple and not related to che-theia or GWT idea.
The goal is to store multiple subprojects in the same git repository but have an ability to checkout only part of it. It was very frequently asked by companies providing some courses and demos. Do you think this use case is invalid and we should abandon it?

[sunix]

In my opinion this was needed only because we couldn't set a 'type' for a subfolder of a che6 project.
But if you could find back real companies projects or courses or demos, we could figure out why this is important. Maybe this could be solved somehow else but what is the problem it is trying to solve ? you are just describing what it is doing but a feature without a need is useless. :)

[skabashnyuk]

@slemeur @l0rd can you confirm that this is a useless feature. If yes I'll close this issue.

[slemeur]

This was related to the sparse checkout in the factories. Yes it is important.
Look at this sample repo: https://github.com/vert-x3/vertx-examples you have a lot of samples and a developer might not want to get all of that at once.

[sunix]

ok. let's do it but i would rename it sparse-checkout rather then keepDir

old article (but still actual i guess) of how to achieve that with git: https://briancoyner.github.io/articles/2013-06-05-git-sparse-checkout/

[sunix]

@slemeur rethinking about this. For samples like the vertx ones, i think it would be better to clone all the repo and find a way to emphasize one of the samples by expanding it and opening files. Or just open a dedicated workspace root folder on the right sample.
Sparse checkout can be quite confusing for the user who won't understand where things are coming from.

[metlos]

When I was playing with the sample devfiles, I encountered this same problem with the go samples. I thought about having generic build/test/run tasks that would get the sample to work with using task inputs (https://code.visualstudio.com/docs/editor/variables-reference#_input-variables) but alas, that doesn't seem to be supported by Theia yet (eclipse-theia/theia#5836).

[sunix]

@skabashnyuk this is an ide2 task right ?

[sunix]

another option to concider is to use workspace root

[skabashnyuk]

@skabashnyuk this is an ide2 task right ?

It can be.

[maxandersen]

just want to state that keepdir/sparsecheckout is a useful but separate feature from being able to just import/focus on a subset of bigger checked out project; which is why I opened #14550

[sunix]

@maxandersen yes this was what I intended to say there

This issue is one proposal to solve a problem in a flow where the user would like to work on a specific subfolder of a bigger git project. I am not convince that sparse checkout is the way to go.

[maxandersen]

I am not convince that sparse checkout is the way to go.

correct - but sparse checkout is not a bad feature to have though. Having way to focus/limit import from a larger repo while still having it checked out would be of higher priority would I say as it applies to more usecases imo.

[mmorhun]

In the Che 6 specification we have keepDir in project config in the factory:

"projects": [
  {
    "source": {
        "type": "git",
        "location": "https://github.com/eclipse/che",
        "parameters": {
          "keepDir": "core",
          "branch": "master"
        }
    }
  }
]

Format of project definition in devfile is slightly different. So the suggestion is to put the keepDir parameter into source section:

projects:
  - name: che
    source:
      type: git
      location: 'https://github.com/eclipse/che'
      branch: 'master'
      keepDir: 'core'

@l0rd @skabashnyuk @evidolob @sleshchenko @mshaposhnik @mkuznyetsov is it ok for you?

[maxandersen]

what does keepDir actually do ? remove everything that does not match that path ?

[sleshchenko]

@maxandersen previously it used sparse checkout https://briancoyner.github.io/2013/06/05/git-sparse-checkout.html. Technically nothing is removed.

[sunix]

could we rename keepDir to something else ? like sparseCheckout so it makes sense for people who used this field what it is doing without having to go through the doc ? I am not sure Che6 naming compatibility is required here as the devfile is a complete different format.

[mmorhun]

We can give any name to the parameter, but the question is in meaningful name. Sparse checkout is a feature and having a directory for its value is weird as for me. I would expect a boolean value for parameter with such name...
If we want self explaining parameter, as you said:

so it makes sense for people who used this field what it is doing without having to go through the doc

IMO, keep directory works better.
Also it will be much easier for users to migrate from factory format to devfile if parameter name is the same.

[sunix]

@mmorhun
https://briancoyner.github.io/2013/06/05/git-sparse-checkout.html

Step 4: Tell Git which directories you want
echo contrib/completion/ >> .git/info/sparse-checkout

it is just not to confuse users with another name.

what does keepDir actually do ? remove everything that does not match that path ?

This is the kind of question people will have if we name it keepDir.

[sunix]

maybe renamed it sparseCheckoutDir what do you think ? @slemeur ? @maxandersen ?

[mmorhun]

sparseCheckoutDir is ok for me

[mmorhun]

Done