Update README

[aeaton-overleaf]

This updates the README with instructions for running the CLSI in Docker using the texlive/texlive image for sibling containers.

[mans0954]

3009 has never been the CLSI port - it's the filestore port. I must have been confused when I added that to the notes!

[mans0954]

There are a number of environment variables missing from the list. These are the ones I found going through the setting file:

ALLOWED_COMPILE_GROUPS
ALLOWED_IMAGES
CATCH_ERRORS
COMPILE_GROUP_DOCKER_CONFIGS
COMPILES_HOST_DIR
COMPILE_SIZE_LIMIT
DOCKER_RUNNER
DOCKER_RUNTIME
FILESTORE_DOMAIN_OVERRIDE
FILESTORE_PARALLEL_FILE_DOWNLOADS
FILESTORE_PARALLEL_SQL_QUERY_LIMIT
LISTEN_ADDRESS
PROCESS_LIFE_SPAN_LIMIT_MS
SENTRY_DSN
SMOKE_TEST
SQLITE_PATH
SYNCTEX_BIN_HOST_PATH
TEXLIVE_IMAGE
TEX_LIVE_IMAGE_NAME_OVERRIDE
TEXLIVE_IMAGE_USER
TEXLIVE_OPENOUT_ANY

[mans0954]

These instructions don't work as is on Linux, because the root user in the texlive image gets mapped to the root user on the host. A workaround is to do something like:

sudo chown -R mans0954:root compiles/
sudo chmod g+w -R compiles/

See this internal issue: https://github.com/overleaf/dev-environment/issues/79
And this upstream issue: moby/moby#7198

[mans0954]

Thanks very much for picking this up! I think the path to the default settings file should be changed. If you've got the time it would be great to add the extra env vars in to the README.

I've never found an entirely satisfactory solution to the Linux file permission mapping. In the dev environment we fudged around it by changing the uid of my Ubuntu account from 1001 to 1000 to match the tex user in our gcr.io/overleaf-ops/texlive-full image.

[aeaton-overleaf]

These instructions don't work as is on Linux, because the root user in the texlive image gets mapped to the root user on the host.

I couldn't find a more appropriate user in the texlive/texlive image, but if there's a suggestion that works here I'd be happy to update the command.

[aeaton-overleaf]

it would be great to add the extra env vars in to the README.

If you wouldn't mind adding them in a separate PR that might be better, as I'm not sure of the description needed for each one.

[jdleesmiller]

sudo chown -R mans0954:root compiles/

It may be worth asking... do we need to bind mount compiles at all? I think it's basically ephemeral data. Keeping it in a volume instead of a bind mount might avoid some complexity here. (Also, I think it would be fine for the /compiles data to live in /tmp for the same reason, and making a volume for /tmp is a fairly standard thing to do.) But, that could all be separate to these README improvements!

[mans0954]

It may be worth asking... do we need to bind mount compiles at all? I think it's basically ephemeral data. Keeping it in a volume instead of a bind mount might avoid some complexity here. (Also, I think it would be fine for the /compiles data to live in /tmp for the same reason, and making a volume for /tmp is a fairly standard thing to do.)

Currently a folder on the host (e.g. $PWD/compiles) gets mounted into CLSI and then project subfolders (e.g. $PWD/compiles/1234) get mounted into the TeX Live container. Presumably DockerRunner would need to create per-project volumes in order to achieve the same level of security as we have at the moment, as I don't think you can mount a path within a volume into a container? Possibly sharing a single volume between the CLSI and all the TeX Live containers would be acceptable in the dev environment?

But, that could all be separate to these README improvements!

Is it worth writing up an issue for this?

[aeaton-overleaf]

I don't think you can mount a path within a volume into a container?

Right, it doesn't seem to be possible. moby/moby#32582

You could in theory mount the whole named volume somewhere without read access and symlink the project folder into somewhere with read access, but that would be a bit risky.

[jdleesmiller]

mounted into the TeX Live container

Good points, thanks! I think we can forget about my comment, in that case. I have given the upstream issue a 👍 .

[mans0954]

Good points, thanks! I think we can forget about my comment, in that case. I have given the upstream issue a .

Okay, not quite sure where to go next with this. There are some good improvements to the README in the PR, but I feel we should at least acknowledge that the instructions won't work if followed literally on most Linux setups.

[aeaton-overleaf]

To make it work on Linux, is it enough to comment out the -e TEXLIVE_IMAGE_USER=root \ line? I could do that and add a comment about enabling it for macOS.

[aeaton-overleaf]

Updated to move the TEXLIVE_IMAGE_USER line into the macOS-specific instructions.

[mans0954]

To make it work on Linux, is it enough to comment out the -e TEXLIVE_IMAGE_USER=root \ line? I could do that and add a comment about enabling it for macOS.

Unfortunately that gives the error (HTTP code 500) server error - linux spec user: unable to find user tex: no matching entries in passwd file because TEXLIVE_IMAGE_USER defaults to tex and there is no tex user in the TeX Live image.

Possibly one could provide instructions for building an image based on Tex Live which contained an additional user. I think you would still need to make the uids of the users in the images match with the uid of the host user.

[aeaton-overleaf]

If it's not currently possible to make this work on Linux then I'll propose merging this and updating it when there's a solution.

[aeaton-overleaf]

@mans0954 Would documenting this workaround that you proposed earlier make sense instead?

sudo chown -R mans0954:root compiles/
sudo chmod g+w -R compiles/

[mans0954]

@mans0954 Would documenting this workaround that you proposed earlier make sense instead?

sudo chown -R mans0954:root compiles/
sudo chmod g+w -R compiles/

Yes, I think that was what I originally had in mind!

[aeaton-overleaf]

Updated with instructions for changing permissions on the compiles folder on Linux.

[mans0954]

Updated with instructions for changing permissions on the compiles folder on Linux.

To be honest, it's a bit more complicated than that. I'll try to write something up.

[mans0954]

I've written something about the permissions issue on Linux. It's complicated as some distros use 1000 as the uid for the first normal user and some use 1001.