Patch plumbing of docker-compose UID/GID build args #4527

stoooops · 2022-06-24T10:57:33Z

Issue

The UID/GID docker build args are only partially plumbed. For docker-compose build flow, they were hard-coded to 1000/1000 in multiple places.

Expected Behavior

I can use a dedicated user with UID != 1000 and/or GID != 1000 without permissions errors in any service defined in docker-compose.yml

Symptoms

At least 3 docker services fail to come up if you use a user with UID != 1000 or GID != 1000. The services that fail are rpcdaemon, prometheus, and grafana (file permissions).

Fix

Add services > erigon > build > args in the docker-compose.yml, so the args are passed in to the docker process from docker-compose at build time
Add .env.example with ${DOCKER_UID} and ${DOCKER_GID} specified
Replace hard-coded 1000/1000 with ${DOCKER_UID} and ${DOCKER_GID} for prometheus, grafana services

These fixes allow me to use a user on my host OS with UID != 1000 and/or GID != 1000 without the docker-compose bringup failing.

Nice-to-have

Improved debugging for future users:

Detailed README including description around permissions pitfalls
More clearly renamed envvars PUID/PGID to DOCKER_UID/DOCKER_GID (on host OS) and UID/GID (in Dockerfile)
Echo user, uid, gid in docker-compose Makefile target
Add a fail-fast Makefile step if the host OS does not have an "erigon*" user with the correct UID/GID

Author

Patched written by cory.eth (Cory Gabrielsen) (@cory_eth).

AskAlexSharov · 2022-06-24T12:14:09Z

Great description

docker-compose.yml

AskAlexSharov · 2022-06-25T09:56:54Z

shell usually has UID and GID vars. means we can do:
UID=${UID} GID=${GID} docker compose up or UID=$(id -u) GID=$(id -g) docker-compose up
and UID=${UID} GID=${GID} docker compose build erigon
and user: "${UID:-1000}:${GID:-1000}"

let's add user: "${UID:-1000}:${GID:-1000}" to x-erigon-service

maybe also rename PUID to UID? Even that it's not backward compatible

How do you think?

stoooops · 2022-06-25T14:28:02Z

Thanks for the quick review. I will try your suggestions tomorrow. Hiking today. ⛰️

AskAlexSharov · 2022-06-28T07:57:43Z

Looks great.
@stoooops , DOCKER_UID also can rename to UID?

stoooops · 2022-06-28T08:14:20Z

Ok, so here's what I ended up doing since first commit:

Added a small .env
Default uid/gid to 3473 in examples (random number) since 1000/1000 will almost always be reserved by non-erigon user on host OS
Rename the PUID/GUID -> UID, GID
In the README, I went with ERIGON_* environment variable names.
More robust fail-fast build checks
Adjust Makefile logic to not print error if Go not installed on host OS (relying on docker)

I went heavy on comments throughout everywhere because this stuff is so easy to get wrong and the file permissions are usually pretty opaque error messages if you're not experienced. Have dealt with this exact issue multiple times across several professional/personal projects now that I favor docker.

stoooops · 2022-06-28T08:20:09Z

Looks great. @stoooops , DOCKER_UID also can rename to UID?

I am actually not sure this is the correct decision. What if the caller is not the erigon user and $UID resolves to the wrong/unexpected/surprising value? What if it's a power user running some docker-compose manually? I think it makes it easier to mess up and not be sure why. I prefer the explicitness.

There can be three user/hosts pairs involved:

"dev" user on host OS
"erigon" user on host OS
"erigon" user in docker

both the erigon users must be in sync. But the dev user (or any other user on the host OS) should be able to invoke make/docker-compose easily without needing to think critically about what UID environment variable will resolve to. Having a separate DOCKER_UID allows this "dev" user to manage the concepts independently.

I do prefer it this way. I think I would need a bit more convincing to change it based on above reasoning.

stoooops · 2022-06-28T23:09:40Z

Can you trigger the CI again when you get a chance? I think I patched it. Thank you.

AskAlexSharov

make erigon doesn't work on MacOS. It doesn't store user in /etc/passwd https://superuser.com/questions/191330/users-dont-appear-in-etc-passwd-on-mac-os-x
CI also red by some reason: https://github.com/ledgerwatch/erigon/runs/7104458854?check_suite_focus=true#step:3:13
Maybe build on hub.docker.com will also fail, maybe need define DOCKER_UID in ./hooks/build

stoooops · 2022-06-29T06:22:12Z

Ok thanks, I will fix MacOS. I know we can add MacOS as a github runner too.

Will try to get it working via my own repo so I don't need to keep triggering failed runs here.

stoooops · 2022-06-29T13:19:46Z

Ok, I have added fixes and then updated the README again to reflect the changes:

patched the Makefile validation to support Mac OS.
include DOCKER_UID, DOCKER_GID in hooks/build
updated README
created Makefile targets for creating "erigon" user on either linux/macos

I tested the CI on my own fork repo, so it should work if we try running it.

AskAlexSharov · 2022-06-29T14:20:33Z

looks great. I will run test on hub.docker.com then merge

stoooops · 2022-06-29T14:48:44Z

I see that it failed. If you can share the logs with me, I can try to patch it. (there was a login prompt so I'm assuming I don't have permissions)

I have also pushed one more change needed to make some README changes I made play nice together even when the env vars aren't setup yet. This doesn't affect the existing CI, just the new targets I made for helping create the "erigon" user.

AskAlexSharov · 2022-06-29T15:49:07Z

from hub.docker.com

Executing build hook...
Docker build args:
DOCKER_UID: 0
DOCKER_GID: 0
Ensuring host OS user exists with specified UID/GID...
✔️ host OS user exists: root
Updating git submodules
...
#17 [stage-1 3/7] COPY --from=builder /app/build/bin/* /usr/local/bin/
#17 sha256:7fd450444c8760819b499bec1a58998f494a8ec1bf2dfd2c29776bac0f075b68
#17 DONE 4.2s
#18 [stage-1 4/7] RUN adduser -H -u 0 -g 0 -D erigon
#18 sha256:30125699a05a0d9ae56d5500f7525ece800845a9f91b874ac644c3f488a1d774
#18 0.378 adduser: uid '0' in use
#18 ERROR: executor failed running [/bin/sh -c adduser -H -u ${UID} -g ${GID} -D erigon]: exit code: 1
------
> [stage-1 4/7] RUN adduser -H -u 0 -g 0 -D erigon:
------
executor failed running [/bin/sh -c adduser -H -u ${UID} -g ${GID} -D erigon]: exit code: 1
Makefile:55: recipe for target 'docker' failed
make: *** [docker] Error 1
build hook failed! (2)

stoooops · 2022-06-29T16:00:30Z

Ok thank you, I need to handle the root user case. I will work on a fix.

stoooops · 2022-06-29T19:21:20Z

Updated:

Set DOCKER_UID=3473 in hooks/build
Set DOCKER_GID=3473 in hooks/build
Add CI test that make docker succeeds when ran as root (after non-root builds succeeds and is cached)

Notably, on my local workspace, when calling sudo DOCKER_UID=$(id -u) DOCKER_GID=$(id -g) make docker, the existing git-submodules target would fail as git doesn't let root user stomp over a non-root users git repository. This is fixed by allowing the git submodule sync to fail (albeit noisely).

-    @git submodule sync --quiet --recursive
+    @git submodule sync --quiet --recursive || true

This case was not hit on Docker Hub since I believe the repo gets checked out with root file permissions.

This case was not hit in CI because the docker builds were ran as non-root user.

With these changes, root user can build the image in an existing, checked out repo, as long as they pass the DOCKER_UID and DOCKER_GID correctly. It will not fail fast from a git file permission check.

stoooops · 2022-06-30T06:02:15Z

I think the CI that failed is nondeterministic / inconsistent and unrelated to these changes. If you re-run the one job, I'd expect it to work. I notice that a couple times on my own fork.

Thanks for your help.

0xakihiko · 2022-07-07T03:52:22Z

Ok thank you, I need to handle the root user case. I will work on a fix.

Also getting this on my server while on root.

 => ERROR [stage-1 4/5] RUN adduser -D -u 0 -g 0 erigon                                                                                                                   0.2s
------
 > [stage-1 4/5] RUN adduser -D -u 0 -g 0 erigon:
#16 0.206 adduser: uid '0' in use
------
executor failed running [/bin/sh -c adduser -D -u $UID -g $GID erigon]: exit code: 1
make: *** [Makefile:55: docker] Error 1```

stoooops · 2022-07-07T04:34:26Z

@0xakihiko Looks like you're running your build as root. Specify DOCKER_UID and DOCKER_GID to be non-zero so that the adduser command succeeds in the dockerfile.

You can see an example of this in the .github/workflows/ci.yaml where it tests that it can build with sudo.

That said, we could patch the Dockerfile "RUN adduser ..." layer so it gracefully ignore creating the user if both are passed in as 0, but usually it's better practice to use a dedicated user, especially as it will try to volume-mount a directory from your host OS and the written files are permissioned by the docker user uid/gid.

0xakihiko · 2022-07-07T04:46:58Z

May

@0xakihiko Looks like you're running your build as root. Specify DOCKER_UID and DOCKER_GID to be non-zero so that the adduser command succeeds in the dockerfile.

You can see an example of this in the .github/workflows/ci.yaml where it tests that it can build with sudo.

That said, we could patch the Dockerfile "RUN adduser ..." layer so it gracefully ignore creating the user if both are passed in as 0, but usually it's better practice to use a dedicated user, especially as it will try to volume-mount a directory from your host OS and the written files are permissioned by the docker user uid/gid.

Yeah this makes sense.
Maybe another alternative is to print a message if 0 is passed through to guide the user in another direction that they may have missed a step if it wasn't intentional?

easeev · 2022-07-13T05:55:55Z

@stoooops I suggest bringing back default values for UID and GID as now Docker Hub builds produces images with value 3473 which breaks many things since before whey were set to 1000

https://hub.docker.com/layers/erigon/thorax/erigon/v2022.07.02/images/sha256-34ae0723539b09f6df51a7c0349acdc8744ed307ee36631be910339345019e15?context=explore

AskAlexSharov · 2022-07-13T07:31:06Z

I think it make sense to return 1000 as default for backward compatibility. done by: #4702

JanRuettinger · 2022-07-19T01:12:32Z

I think there is something missing in the readme.

My setup:
user 1: dev
user 2: erigon (created with password disabled using make user_linux)

When I run:
make user_linux
ERIGON_USER=erigon
sudo -u ${ERIGON_USER} DOCKER_UID=$(id -u ${ERIGON_USER}) DOCKER_GID=$(id -g ${ERIGON_USER}) XDG_DATA_HOME=~${ERIGON_USER}/.ethereum DOCKER_BUILDKIT=1 COMPOSE_DOCKER_CLI_BUILD=1 make docker-compose

I get the following error/issue. I am supposed to type in a password for erigon. But the password is disabled.

Docker build args:
    DOCKER_UID: 3473
    DOCKER_GID: 3473

Ensuring host OS user exists with specified UID/GID...
✔️ host OS user exists: erigon
mkdir -p ~erigon/.ethereum/erigon ~erigon/.ethereum/erigon-grafana ~erigon/.ethereum/erigon-prometheus
ls -aln ~erigon/.ethereum | grep -E "472.*0.*erigon-grafana" || sudo chown -R 472:0 ~erigon/.ethereum/erigon-grafana
[sudo] password for erigon:

Any ideas?

Patch plumbing of docker-compose UID/GID build args

fb70d08

stoooops force-pushed the devel branch from 73d955b to fb70d08 Compare June 24, 2022 11:19

AskAlexSharov requested changes Jun 25, 2022

View reviewed changes

docker-compose.yml Outdated Show resolved Hide resolved

stoooops added 7 commits June 28, 2022 06:40

Fallback to 1000/1000 if DOCKER_(U|G)ID not set

c1d032d

Revise README.md instructions for docker further

0ab02a2

Fix existing typo forc 'servie' -> 'service'

b1330b0

Rename PUID/GUID -> UID/GID

5e45423

Specify user in erigon docker service

00d887e

Rely on .env instead of specifying :-1000

ea1c614

Polish Makefile for docker use case

17f7eff

one more helpful comment

5ecfbac

stoooops added 2 commits June 28, 2022 08:24

make docker should use UID/GID --build-arg

770a6a2

Fix make docker and more fail-fast if envvar set incorrect

8ac3e2d

stoooops requested a review from AskAlexSharov June 28, 2022 09:01

mv .env->.env.example to not intefere existing workflows

e905840

stoooops force-pushed the devel branch from d1cbc4e to e905840 Compare June 28, 2022 21:52

Specify envvars in docker CI

b5a8ae4

AskAlexSharov requested changes Jun 29, 2022

View reviewed changes

stoooops added 4 commits June 29, 2022 00:03

Adjust validate_docker_build_args to permit non-erigon user

6a50d0d

Also run docker CI target on macos-11 os

8b988df

Add DOCKER_UID, DOCKER_GID in hooks/build

f57ee93

Patch docker build arg validation for macos

86956e1

stoooops added 8 commits June 29, 2022 03:40

More Makefile cleanup, debugging still

3c039e4

Typo fix

9ba206d

Create subdirs before calling ls

52f9b7a

Get rid of flaky validation

7ddf093

DOCKER_UID, DOCKER_GID init to runner if not set

a6301af

Get rid of unnecessary variable for now

a2c0902

Improved README based on new changes

966a846

CI, Mac OS fixes, update README

36307e8

stoooops requested a review from AskAlexSharov June 29, 2022 14:12

Proper uid/gid make user_* when no envars set

30fce40

Fix typo in Makefile comment

5eb4f60

Fix make docker as sudo user

b10125a

AskAlexSharov closed this Jun 30, 2022

AskAlexSharov reopened this Jun 30, 2022

AskAlexSharov approved these changes Jun 30, 2022

View reviewed changes

AskAlexSharov merged commit 74cf984 into erigontech:devel Jun 30, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Patch plumbing of docker-compose UID/GID build args #4527

Patch plumbing of docker-compose UID/GID build args #4527

stoooops commented Jun 24, 2022 •

edited

Loading

AskAlexSharov commented Jun 24, 2022

AskAlexSharov commented Jun 25, 2022 •

edited

Loading

stoooops commented Jun 25, 2022 •

edited

Loading

AskAlexSharov commented Jun 28, 2022

stoooops commented Jun 28, 2022

stoooops commented Jun 28, 2022 •

edited

Loading

stoooops commented Jun 28, 2022

AskAlexSharov left a comment

stoooops commented Jun 29, 2022

stoooops commented Jun 29, 2022

AskAlexSharov commented Jun 29, 2022

stoooops commented Jun 29, 2022 •

edited

Loading

AskAlexSharov commented Jun 29, 2022

stoooops commented Jun 29, 2022 •

edited

Loading

stoooops commented Jun 29, 2022

stoooops commented Jun 30, 2022 •

edited

Loading

0xakihiko commented Jul 7, 2022 •

edited

Loading

stoooops commented Jul 7, 2022

0xakihiko commented Jul 7, 2022

easeev commented Jul 13, 2022

AskAlexSharov commented Jul 13, 2022

JanRuettinger commented Jul 19, 2022

Patch plumbing of docker-compose UID/GID build args #4527

Patch plumbing of docker-compose UID/GID build args #4527

Conversation

stoooops commented Jun 24, 2022 • edited Loading

Issue

Expected Behavior

Symptoms

Fix

Nice-to-have

Author

AskAlexSharov commented Jun 24, 2022

AskAlexSharov commented Jun 25, 2022 • edited Loading

stoooops commented Jun 25, 2022 • edited Loading

AskAlexSharov commented Jun 28, 2022

stoooops commented Jun 28, 2022

stoooops commented Jun 28, 2022 • edited Loading

stoooops commented Jun 28, 2022

AskAlexSharov left a comment

Choose a reason for hiding this comment

stoooops commented Jun 29, 2022

stoooops commented Jun 29, 2022

AskAlexSharov commented Jun 29, 2022

stoooops commented Jun 29, 2022 • edited Loading

AskAlexSharov commented Jun 29, 2022

stoooops commented Jun 29, 2022 • edited Loading

stoooops commented Jun 29, 2022

stoooops commented Jun 30, 2022 • edited Loading

0xakihiko commented Jul 7, 2022 • edited Loading

stoooops commented Jul 7, 2022

0xakihiko commented Jul 7, 2022

easeev commented Jul 13, 2022

AskAlexSharov commented Jul 13, 2022

JanRuettinger commented Jul 19, 2022

stoooops commented Jun 24, 2022 •

edited

Loading

AskAlexSharov commented Jun 25, 2022 •

edited

Loading

stoooops commented Jun 25, 2022 •

edited

Loading

stoooops commented Jun 28, 2022 •

edited

Loading

stoooops commented Jun 29, 2022 •

edited

Loading

stoooops commented Jun 29, 2022 •

edited

Loading

stoooops commented Jun 30, 2022 •

edited

Loading

0xakihiko commented Jul 7, 2022 •

edited

Loading