Go sample permission denied issues with dependencies

[sparkoo]

Description

There are issues with current Go devfile sample https://github.com/eclipse/che-devfile-registry/tree/master/devfiles/go. This is the sample project: https://github.com/golang/example. We have tasks just for its outyet part, which works ok. However, there are many other Go files with main functions. Problematic ones are those with 3rd party dependencies, like this one https://github.com/golang/example/blob/master/gotypes/skeleton/main.go. When theia tries to build the file, it fails with permission denied to /go directory, when it try to download dependency there.

Reproduction Steps

• create workspace from devfile https://github.com/eclipse/che-devfile-registry/blob/master/devfiles/go/devfile.yaml
• open gotypes/skeleton/main.go
• do any changes to file to kick in the compiler and save the file

There is another issue with clone path #13796 which affects also this devfile. This patch has to be applied to the devfile. Without that, project is cloned to ... golang/example/example. Once the issue is fixed, this patch must not be needed.

diff --git a/devfiles/go/devfile.yaml b/devfiles/go/devfile.yaml
index 8c09ff9..508a8c7 100644
--- a/devfiles/go/devfile.yaml
+++ b/devfiles/go/devfile.yaml
@@ -8,7 +8,7 @@ projects:
   source: 
     type: git
     location: https://github.com/golang/example.git
-  clonePath: src/github.com/golang/example/
+  clonePath: src/github.com/golang/
 components:
 -
   type: chePlugin

OS and version:
7.0.0-rc-4.0-SNAPSHOT

Diagnostics:
running go: Current GOPATH task say /go:/projects is the current GOPATH.
I believe this is the issue. GOPATH set in the plugin here https://github.com/eclipse/che-plugin-registry/blob/master/v3/plugins/ms-vscode/go/0.11.0/meta.yaml#L20 have /go as first and this is where it tries to download the dependency, but has no permissions there.

[sparkoo]

When we fix this one, the realworld sample will probably also work and we could reconsider to include also that one into 7.0.0.

[rhopp]

After discussing with @sparkoo I believe this should be dealt with in Che 7 Endgame.
Mainly because of two reasons:

1. Adding ANY external dependency will result in user not being able to develop in this workspace.
2. The repository cloned by default in this go devfile contains multiple examples, of which some of them contains external dependencies. (if user would try them, he will get these errors)

[tsmaeder]

So the issue is that "/go" is first in the gopath in the devfile and go language tools are downloading dependencies there?

[sparkoo]

@tsmaeder that's my theory. However, I don't know how to patch and deploy Go plugin locally so I haven't test it in Che.

[l0rd]

Can this be worked out by @amisevsk and @ibuziuk in the context of #13454?

[tsmaeder]

@l0rd I asked @tolusha to have a look and analyze what a fix should look like. Usually, the folder /go should not be writeable, right? That would require sudo (which we don't want).
Question is: since we "own" the container, should we make /go writeable or do we need another workaround?

[l0rd]

I think we have a few options (in order of preference):

1. Use ${HOME}/go instead of /go
2. Mount folder /go as a PV
3. chmod /go at build time as we do for /etc/passwd

[amisevsk]

I don't know that this issue has a convenient solution; Go has issues with developing outside of ${GOPATH} (e.g. when I'm working on che-plugin-broker, the code is checked out to ${GOPATH}/src/github.com/eclipse/che-plugin-broker).

Go 1.11 adds "preliminary" support [1] for modules and 1.12 improves this [2] in preparation for it being a default in 1.13 later this year. This would resolve our issue, but

1. Our current devfile runs on go version 1.10.7
2. Projects need to convert to the new format (create a go.mod file). This can in some cases be done automatically but still needs to be done.

With our current version of Go, the solution would look something like

1. Set ${GOPATH}=/projects
2. Check out workspace code to a $GOPATH compatible directory (e.g. /projects/src/github.com/eclipse/che-plugin-broker)
3. Work in this directory.

[1] - Docs on modules
[2] - Relevant Go 1.12 release notes

[tolusha]

Wrong permissions for /go/pkg folder.
Easily can be fixed by patching the corresponding docker file for go plugin [1]

/go $ ls -l
total 0
drwxrwxrwx    2 0        root           642 Jul 15 06:55 bin
drwxr-xr-x    3 0        root            25 Jul 15 06:55 pkg
drwxrwxrwx    5 0        root            65 Jul 15 06:55 src

[1] https://github.com/eclipse/che-theia/blob/master/dockerfiles/remote-plugin-go-1.10.7/Dockerfile#L45

[tolusha]

I've just check the PR [1] and it works fine.
Here is the devfile [2] which uses go plugin [3] with updated docker image

[1] eclipse-che/che-theia#348
[2] https://github.com/tolusha/devfiles-registry/blob/master/devfiles/go/devfile.yaml#L15
[3] https://github.com/tolusha/plugins-registry/blob/master/plugins/tolusha/go/0.11.0/meta.yaml#L15

[tsmaeder]

@tolusha any particular reason you're not proposing a PR with 2) and 3)?

[tolusha]

@tsmaeder I don't understand what you mean.

[tsmaeder]

Who is going to fix the devfile and plugin to use the new image?

[tolusha]

Nobody, the fix will be in the next build of go sidecar container.

[tsmaeder]

@sparkoo So can we consider this one fixed now?

[sparkoo]

@tsmaeder I wasn't able to verify the fix yet. All che-theia containers I've tried still have the bug. Current default is eclipse/che-theia/7.0.0-next https://github.com/eclipse/che/blob/master/assembly/assembly-wsmaster-war/src/main/webapp/WEB-INF/classes/che/che.properties#L657, which is still failing for me.

[sparkoo]

@tolusha I've just tried with eclipse/che-theia/7.0.0-next and the issue is still there. Could you please take a look?

[tolusha]

@sparkoo
I confused you. It is needed to check with the latest image for sidecar container which is eclipse/che-remote-plugin-go-1.10.7:7.0.0-next [1]

[1] https://github.com/eclipse/che-plugin-registry/blob/master/v3/plugins/ms-vscode/go/latest/meta.yaml#L15

I've just checked and permissions are set correctly. Could you confirm pls?

[sparkoo]

@tolusha ok, how could I test it?

[tolusha]

1. Stop go workspace
2. minishift ssh
3. docker pull docker.io/eclipse/che-remote-plugin-go-1.10.7:7.0.0-next
4. Start go workspace

[tolusha]

I see, now it is problem with /go/pkg/linux_amd64 directory.

/go/pkg $ ls -l
total 0
drwxr-xr-x    3 0        root            24 Jul 22  2019 linux_amd64

[tolusha]

I've introduced PRs to fix the problem.

[sparkoo]

yes, works now. thanks.

[amisevsk]

@tolusha Has this fix made it into the plugin-registry/sidecar image? Using https://che-plugin-registry.prod-preview.openshift.io/ I'm not seeing the fix for our golang devfile.

[tolusha]

@amisevsk correct.
The fix was for sidecar image.