-
Notifications
You must be signed in to change notification settings - Fork 18.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docker build hangs/crashes when useradd with large UID #5419
Comments
We're probably doing the wrong thing with sparse files which Can you update this the output of I'd also be interested if you can you explain your need for such a large uid? |
Thanks for your prompt answer. These user ids are given to each user (university wide) by our IT overlords. Setting a proper UID is needed to write to NAS mounts.
|
As a work around can you try the
|
Thanks! This worked. |
The underlying issue is that a large sparse file is created (approximately 32 GB), but it's not exactly a Docker bug. Docker could make an attempt to handle large sparse files better, but that's a subject for the #docker-dev mailing list. I'll close this issue now. Please feel free to comment. |
Use --no-log-init when creating a user to avoid moby/moby#5419
So is this an issue with "docker commit" that it can't handle sparse files and blots the file to the full size during commit. If yes, then I guess this is something which should be fixed in docker. I created a 1G sparse file in a container (with overlay backend) and then did docker commit and top most layer size was 1G. So docker did inflate the file to 1G and that seems very inefficient. |
@unclejack Has this issue been discussed since then any where else? I tested this with latest docker so it has not been fixed yet. |
|
It's happening the same on overlay.. I'm going to reopen this to track this bug and see if Docker can better handle this scenario |
golang's |
This issue still occurs on docker
Hangs after printing done. Docker info:
|
I can also confirm this is still occurring with I am trying to create the following image:
And I am building with the following command:
The build freezes after displaying all output from RUN and writes to docker info output:
|
I also can confirm that @pnasrat's workaround fixes this problem. I am now using:
instead of:
And the build succeeded. My $HOST_USER_UID and $HOST_USER_GID contain 10 digit IDs |
linking golang/go#13548 to this issue |
Reproducible as an automated build on Docker Hub: https://hub.docker.com/r/gbraad/issue-dockerfile/builds/bsdmngknf8dhfreh8sgfmyu/ :-s While |
Presumably disk quotas will fix the issue with the host running out of space and just leave a runtime error? |
…user in docker Note: related to moby/moby#5419 Signed-off-by: Petro Karashchenko <petro.karashchenko@caviumnetworks.com>
* Spark jars should have a dedicated artifact id * Use python 3 instead of python 2. This requires to make /usr/bin/python point to python3 * Add -l option to docker useradd command to prevent a bug using long uid moby/moby#5419 * Remove \\\ in the maven settings.xml file in the container otherwise the variables criteo.repo.username|password are not correctly used and lead to 401 issues when uploading files to nexus * Fix hive dependency. Groupid has to be org.spark-project.hive and artifactid version 1.2.1.spark2 * use set -e in the build script to fail immediatly when an error occur in any command * Spark scala is a profile and cannot be activated with -D java option but rather with -P maven option * Use --no-transfer-progress in maven commands to make output readable * All mvn commands were using \\ instead of \ leading to bad command interpreation by bash * Fix tar command used to build the 'jar-only' tgz * Fix mvn jar:jar deploy:deploy parameters which are not exactly the same as the ones of mvn deploy:deploy-file * Upgrade pip in the venv otherwise pyarrow cannot install * Fix altDeploymentRepository declaration * Set pypandoc version to 1.5 as some functions have been removed in newer versions * Set back version of build-helper-maven-plugin and change maven-shade-plugin (error during git apply patch) * Remove leftover 2.4.3-criteo versions in pom.xml * Fix compilation error in sql/core/src/test/scala/org/apache/spark/sql/internal/SQLConfSuite.scala
* Spark jars should have a dedicated artifact id * Use python 3 instead of python 2. This requires to make /usr/bin/python point to python3 * Add -l option to docker useradd command to prevent a bug using long uid moby/moby#5419 * Remove \\\ in the maven settings.xml file in the container otherwise the variables criteo.repo.username|password are not correctly used and lead to 401 issues when uploading files to nexus * Fix hive dependency. Groupid has to be org.spark-project.hive and artifactid version 1.2.1.spark2 * use set -e in the build script to fail immediatly when an error occur in any command * Spark scala is a profile and cannot be activated with -D java option but rather with -P maven option * Use --no-transfer-progress in maven commands to make output readable * All mvn commands were using \\ instead of \ leading to bad command interpreation by bash * Fix tar command used to build the 'jar-only' tgz * Fix mvn jar:jar deploy:deploy parameters which are not exactly the same as the ones of mvn deploy:deploy-file * Upgrade pip in the venv otherwise pyarrow cannot install * Fix altDeploymentRepository declaration * Set pypandoc version to 1.5 as some functions have been removed in newer versions * Set back version of build-helper-maven-plugin and change maven-shade-plugin (error during git apply patch) * Compilation error in org.apache.spark.sql.internal.SQLConfSuite
* Spark jars should have a dedicated artifact id * Use python 3 instead of python 2. This requires to make /usr/bin/python point to python3 * Add -l option to docker useradd command to prevent a bug using long uid moby/moby#5419 * Remove \\\ in the maven settings.xml file in the container otherwise the variables criteo.repo.username|password are not correctly used and lead to 401 issues when uploading files to nexus * Fix hive dependency. Groupid has to be org.spark-project.hive and artifactid version 1.2.1.spark2 * use set -e in the build script to fail immediatly when an error occur in any command * Spark scala is a profile and cannot be activated with -D java option but rather with -P maven option * Use --no-transfer-progress in maven commands to make output readable * All mvn commands were using \\ instead of \ leading to bad command interpreation by bash * Fix tar command used to build the 'jar-only' tgz * Fix mvn jar:jar deploy:deploy parameters which are not exactly the same as the ones of mvn deploy:deploy-file * Upgrade pip in the venv otherwise pyarrow cannot install * Fix altDeploymentRepository declaration * Set pypandoc version to 1.5 as some functions have been removed in newer versions
There is an old bug on docker, which having a large user id will cause hang/crash on build level. --no-log-init added to the useradd command, such that the /var/log/faillog does not take much space. The upstream issue: moby/moby#5419
On macOS, building the "openedx-dev" Docker image resulted in an image that required more than 600 GB of disk space. This was due to the `adduser` command which was called with a user ID of 2x10⁹ (on macOS only). This resulted in a very large /var/log/faillog file, hence the image size. Related upstream discussion: moby/moby#5419 Close openedx-unsupported/wg-developer-experience#178
On macOS, building the "openedx-dev" Docker image resulted in an image that required more than 600 GB of disk space. This was due to the `adduser` command which was called with a user ID of 2x10⁹ (on macOS only). This resulted in a very large /var/log/faillog file, hence the image size. Related upstream discussion: moby/moby#5419 Close openedx-unsupported/wg-developer-experience#178
I had the same issue. It occurred when we changed the user ID in the RUN instruction of the image using usermod command. Since I couldn't manage the user creation, I had to find other way. It took me a couple of days to find out other workaround. And I did it. From man pages of lastlog:
I set it equal to UID_MAX and the problem was gone. |
When I try to add a user during a "docker build ." the process hangs for approx 2-3 min and crashes with a
Dockerfile
If I change the above to
or run
useradd -u 99900000 -g users mcieslik
in a
everything works fine
The text was updated successfully, but these errors were encountered: