Move Connect build to a new Docker container

[jakule]

This PR moves the Connect build to a new docker image. This will allow us to use the different base image for Connect and our CI, as at that point, the buildbox will be used only by our CI.

Dev build: https://drone.platform.teleport.sh/gravitational/teleport/24668/5/4

After this PR is merged, I'll create another PR upgrading the buildbox to use ubuntu:latest. I want to keep them separate for a cleaner history.

Contributes to #26856

[codingllama]

LGTM.

Nice, this should be a lot faster to build and easier to maintain.

[ravicious]

I tried to run this locally. make -C build.assets teleterm fails when doing yarn install because some Electron script tries to save something to a global node_modules folder (instead of the default local location) and the global folder is not writable.

Error log

[4/4] Building fresh packages...
warning Cannot find a suitable global folder. Tried these: "/usr/local, /.yarn"
warning Cannot find a suitable global folder. Tried these: "/usr/local, /.yarn"
warning Cannot find a suitable global folder. Tried these: "/usr/local, /.yarn"
warning Cannot find a suitable global folder. Tried these: "/usr/local, /.yarn"
warning Cannot find a suitable global folder. Tried these: "/usr/local, /.yarn"
warning Cannot find a suitable global folder. Tried these: "/usr/local, /.yarn"
warning Cannot find a suitable global folder. Tried these: "/usr/local, /.yarn"
warning Cannot find a suitable global folder. Tried these: "/usr/local, /.yarn"
warning Cannot find a suitable global folder. Tried these: "/usr/local, /.yarn"
warning Cannot find a suitable global folder. Tried these: "/usr/local, /.yarn"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
error /go/src/github.com/gravitational/teleport/node_modules/electron: Command failed.
Exit code: 1
Command: node install.js
Arguments:
Directory: /go/src/github.com/gravitational/teleport/node_modules/electron
Output:
Error: EACCES: permission denied, mkdir '/.cache'
make: *** [teleterm] Error 1

I also get this message:

warning Skipping preferred cache folder "/.cache/yarn" because it is not writable.
warning Selected the next writable cache folder in the list, will be "/tmp/.yarn-cache-501".

I think it might all be related to the -u flag that we're passing to docker run, -u $(id -u):$(id -g). All the folders set up by the node image are set up for the node user but -u causes the command to run as a different user perhaps?

[codingllama]

If we can drop custom users shenanigans for the connect image, +1 to it.

[jakule]

I tried to run this locally. make -C build.assets teleterm fails when doing yarn install because some Electron script tries to save something to a global node_modules folder (instead of the default local location) and the global folder is not writable.

Error log
I also get this message:

warning Skipping preferred cache folder "/.cache/yarn" because it is not writable.
warning Selected the next writable cache folder in the list, will be "/tmp/.yarn-cache-501".

I think it might all be related to the -u flag that we're passing to docker run, -u $(id -u):$(id -g). All the folders set up by the node image are set up for the node user but -u causes the command to run as a different user perhaps?

@ravicious Thanks for checking that. I think that this image "was designed" only to work as node user with UID=1000. I hardcoded the value 😐

[jakule]

If we can drop custom users shenanigans for the connect image, +1 to it.

@codingllama I don't think we can. We use that image on Drone. We should not run this image as root for a few reasons:

1. Artifacts generated as root can't be removed, copied etc. AFAIR Drone runs as a non-root user.
2. Potential privilege escalation and other security-related issues
3. This is something to check, but this image comes with the assumption that you will use it as a node user with UID=1000. I'm not sure what are the consequences of using root.

In this case, I vote for just hardcoding the UID. It will work on macs, as the Mac driver re-map users anyway. On Linux everyone should be fine, as of course everyone builds on Linux as the default user 🙈

[jakule]

Currently, I'm getting some x86/arm issues on MacOS

  ⨯ cannot execute  cause=exit status 1
                    errorOut=/home/node/.cache/electron-builder/fpm/fpm-1.9.3-2.3.1-linux-x86/lib/ruby/bin/ruby: line 6: /home/node/.cache/electron-builder/fpm/fpm-1.9.3-2.3.1-linux-x86/lib/ruby/bin.real/ruby: cannot execute binary file: Exec format error
    /home/node/.cache/electron-builder/fpm/fpm-1.9.3-2.3.1-linux-x86/lib/ruby/bin/ruby: line 6: /home/node/.cache/electron-builder/fpm/fpm-1.9.3-2.3.1-linux-x86/lib/ruby/bin.real/ruby: Success

I'll take a look at this tomorrow, but looks like this is a common problem: jordansissel/fpm#1801

[codingllama]

@jakule hardcoding the UID seems alright to me, as at least we have a deterministic user. It's the "binding the local user" shenanigans that I tend to dislike. (Of course, not having to bother with users at all would be the ideal.)

[ravicious]

Re fpm issues, I just checked and it looks like make -C build.assets teleterm has never actually worked on arm64 Macs.

I completely forgot about it when I was writing my comment, but whenever we build Connect in Linux VMs to test stuff, we have to remember about setting USE_SYSTEM_FPM=1. I'm fine with not addressing this in this PR.

[gzdunek]

Tested the dev build on Ubuntu 20.04, no problems.

I really like the idea of using node image as a base.

[ravicious]

I really like the idea of using node image as a base.

Oh yeah, I forgot to give a thumbs up to this. I feel like 80% of build failures of Connect is caused by the Node.js CDN returning an error.