Want better loadup / release method with Docker integration

[masinter]

Is your feature request related to a problem? Please describe.

The Docker integration is a good demo of the idea, but the build stopped working with maiko not a subproject.

Describe the solution you'd like

1. Separate out 'building linux.x86_64 maiko container (which just needs a dev environment) from a minimal 'run medley' container
2. The run medley container should have more run-time files like fonts and dinfo
3. instructions on how to mount a folder in your file system if you're docker desktop host
4. ability to use a local X server as well as vnc

Describe alternatives you've considered
Maybe we'll be able to run in a browser rather than in the cloud

Additional context

[waywardmonkeys]

I've been learning Docker recently for other things and have some ideas on how we can improve upon this.

For item 1, we can do a multi-stage build where the first stage is a full image with dev tools, but then we copy the required build products to the second stage which is the actual image that gets deployed.

Items 2, 3, 4 are things I don't know about yet.

The alternative ... I'm working on it slowly.

[AbeJellinek]

Trying to decide how to proceed on this. Our current build file does use a multi-stage build, and it works (worked) well. The main issue is that a Dockerfile can only copy files from the directory that it is in into the container, i.e. COPY ./foo.txt / works, but COPY ../other-dir/foo.txt / doesn't. Now that Maiko is no longer a submodule (and thus doesn't exist inside medley), we're left with two options for how to run the build:

1. Have two Dockerfiles, one in maiko, one in medley. maiko will build with full dev tools, medley will copy only the relevant files out of the maiko image and build a final (slim) release image. Anyone wanting to build a new version of the Docker image will need to run two separate Docker builds in the right order - inconvenient and fragile.
2. Have one multi-stage Dockerfile like we have now. First stage builds the maiko image, second stage copies the built artifacts out of the maiko image and builds the medley image. To get around the Dockerfile directory issue, create a symlink in medley pointing to maiko that we clean up at the end. Better because we only need one Dockerfile and we don't have to tag the large intermediate (maiko) image, worse because it requires a wrapper script to create/dispose of the symlink, and because the symlink approach itself seems suboptimal. (But what is optimal here?)

@waywardmonkeys, curious to hear your thoughts about this.

[masinter]

Is there something special about the name Dockerfile? It would seem limiting to only allow one per repo.

[nbriggs]

It's like Makefile to make. Documentation suggests that you can do docker build -f /path/to/a/Dockerfile with an arbitrary name.

[masinter]

@AbeJellinek did you see the proposed fixes in #233? you file gave me some errors, for which i found work-arounds, mainly ln -s ../maiko plus updating .dockerignore. I wonder if we can do releases with automation, including whatever is needed. Will we need something more complex?

[AbeJellinek]

I left a comment on your changes a while ago, but I guess it got lost! On my system, the code seems to create the wrong directory structure. I'll give it another try.

[AbeJellinek]

Hmm. I actually don't even get that far. The output:

[+] Building 16.3s (11/17)
 => [internal] load build definition from Dockerfile                                           0.3s
 => => transferring dockerfile: 681B                                                           0.0s
 => [internal] load .dockerignore                                                              0.5s
 => => transferring context: 121B                                                              0.0s
 => [internal] load metadata for docker.io/library/ubuntu:focal                                1.6s
 => [stage-0 1/6] FROM docker.io/library/ubuntu:focal@sha256:b4f9e18267eb98998f6130342baacaeb  0.0s
 => [internal] load build context                                                             13.9s
 => => transferring context: 1.27GB                                                           13.4s
 => CACHED [stage-0 2/6] RUN apt-get update && apt-get install -y make clang libx11-dev        0.0s
 => ERROR [stage-0 3/6] COPY ./maiko/ /build/                                                  0.0s
 => CACHED [stage-1 2/8] RUN apt-get update && apt-get install -y tightvncserver               0.0s
 => CACHED [stage-1 3/8] RUN mkdir -p /app/maiko                                               0.0s
 => CACHED [stage-1 4/8] WORKDIR /app                                                          0.0s
 => ERROR [stage-1 5/8] COPY ./* ./                                                            0.0s
------
 > [stage-0 3/6] COPY ./maiko/ /build/:
------
------
 > [stage-1 5/8] COPY ./* ./:
------
failed to compute cache key: too many symlinks: /maiko

All I did was ln -s ../maiko. Could it be a Mac-specific issue?

[AbeJellinek]

Interesting. This seems explicitly not to work! Yet it works on your machine? Which folder are you building in?

When I build in maiko/ it fails and complains about the symlink. In the parent directory that contains both medley/ and maiko/, it fails with a less specific error: cannot replace to directory /var/lib/docker/overlay2/tqgfqikjsm3ablek0xxh778gh/merged/app/maiko with file

[masinter]

I think the idea now is that the Docker build stage is unnecessary. All you need is a release and load the maiko.llinux and medley compressed tar files, into a kind of bare Docker distro.

[masinter]

Also #300 for a possible delivery of docker method