New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] Secret mounts in /run/secrets
throws an error: Could not find the file / in container <container_id>
#10663
Comments
Tried to reproduce but don't get the same error:
Which I expected: secret is injected under /run/secrets/grafana_admin_password as owner |
@ndeloof 😮 this is rather interesting that you get a permission denied error. I used to have Would you happen to know if this is because of certain settings in the Docker Daemon that you have? I am okay with the permission error, but the secrets not being created in the first place is what was not expected |
sure, but this one I can't reproduce. |
Stepsusing the compose file in the issue with LogsHere are the logs from the Docker Daemon via journalctl -xu docker.service | tail -f
Beyond the warning of the |
Maybe this seems like a Moby-related Issue? |
can you please try: services:
demo:
image: alpine
secrets:
- foo
user: "1000"
secrets:
foo:
environment: FOO
|
No still the same error. for a FOO=hello docker compose -f docker-compose.test.yml run demo cat /run/secrets/foo provides the logs:
and the daemon logs
|
can you try copying a random file in a test container ?
|
Case 1
services:
demo:
image: alpine
command: sleep 3600 Steps
Case 2
Works |
@ndeloof here is a thorough analysis of the same docker compose files in two distinct Docker Engine Versions. This might need to be discussed also on Moby Code under Test
DiscrepancyDocker Engine Version (v24.0.2)
This only cements my current conclusion that somehow the user needs to be the same name as that of the container image if not Docker Engine Versions (v23.0.6)upon downgrading
|
tested your example repo
still can't reproduce the I wonder: do you have containerd image store enabled ? |
My {
"data-root": "/home/shantanoo/docker",
"insecure-registries" : ["artifactory.internal.org"],
"debug": true,
"features": {
"buildkit": true
},
"dns": ["10.24.64.11", "8.8.8.8"]
} I am currently on my work-machine which is WSL2 on Windows 10, but the same error I got was on my personal machine with Manjaro Linux (although the
No. on neither one of the instances |
Vagrant Boxes as Proof@ndeloof the repository provides two isolated instances of the problem being faced with reproducible environments and working examples with results to back the claim. This is the maximum I can reach when it comes to reproducing the errors I get locally via Vagrant Boxes (VMs) https://github.com/shantanoo-desai/docker-engine-secrets-error |
Thanks for you test setup, I was able to reproduce issue |
This indeed is a moby issue, I logged moby/moby#45719 with my debugging notes |
the reason I was not able to reproduce is I'm running latest codebase which includes #10598. The good news is that this will also bring you a fix (actually, workaround) for this issue |
I'm closing this issue as we can follow up fix in moby/moby#45719 and #10598 already reduces the impact of this bug |
Description
This error started just recently, especially when updating to latest Docker Engine / Docker Compose versions.
When a service is mentioned with
user: "1000"
the container that have explicit users defined to them e.g.grafana
in their images fail to mount the secrets in compose file to the respective/run/secrets
directory in the container on boot.Previously, it was known that a
getent
was performed on the host to match whether theuser
ID matches and a container is spun up in order to make/run/secrets
readable by a the container's user (if it notroot
).Steps To Reproduce
Compose file
Environment Variables
.env
fileSteps
Upon
docker compose up
the following error occurs:Container komponist_grafana Creating Error response from daemon: Could not find the file / in container 9714fd659bd2eb795855f9fa292d7e76f3a06fdd40a16dfd47e5c53f759758a9
Upon forcing an up again using
docker compose up
the following logs show up:Upon removing the
user
value from the Compose file, error still persistsCompose Version
Docker Environment
Anything else?
Work-around / Solution
The only way to get the container up is to figure out from the container what the user is using an
whoami
/id
and place this in theuser
of the compose file.Working
docker-compose.yml
I am not sure if this is a Docker Compose Bug or a Docker Engine thing from the start of Docker Engine v24.x.x.
Happy to help reproduce any other examples with similar logic.
The text was updated successfully, but these errors were encountered: