-
Notifications
You must be signed in to change notification settings - Fork 18.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cannot specify --storage-opt to "docker build" #34947
Comments
I should mention I'm aware of docker/for-win#999 which also creates a problem since we have no way to increase volume size for |
Docker CE 17.09 (stable) shouldn't be that far of, and will have the fix in the engine. Further, for the users that are consuming your base image, will they also be generating incredibly large layers on top of your base image? |
Possibly. There's basically two goals here. We're containerizing Build Tools for Visual Studio 2017 but its not yet as feature-rich (i.e. doesn't support building all common project types) as Visual Studio 2017 itself, so even if we made a base image of Build Tools available on which customers may build, customers may also need to install Visual Studio 2017 into an image for now. So besides chasing down any blocking bugs to just building a complete image ourselves, I also need to provide guidance for customers building their own. So being able to pass more options (or just specific ones, like |
I agree that this is something we want. We've had multiple asks for this (ex. #33469), and it's much more reasonable to have a base image size specified at build time than being a global daemon option imo. |
After discussing different options for implementation with @jhowardmsft and @johnstep, I'm working on a change to add |
Apologies for bringing up an old topic, but it seems it was never implemented. We've been using VS Build Tools 2017 since August 2017, and never had this problem.
|
@johnstep created PR #35780 but it was rejected after the default size was raised to 127GB (same as the default for Hyper-V). Apart from being rather short-sighted, IMO (using build tools containers for large solutions might require even more space, which means it should be configurable), I don't know if this made it into Docker 4 Windows in either the edge or stable channels. It's been nearly a year so I'd hope so. |
Following this, as I'm also looking to create large base images. |
@crossan007 How large? The default scratch is now 127GB. Bigger than that? |
@jhowardmsft New here. By default scratch do you mean the max size is 127GB for a container being built? I ask because I'm trying to build a simple container following Microsoft's instructions (install Visual Studio Build Tools). When building, after pulling and extracting the images I still have 50GB on the windows host machine remaining. Then it conducts the Visual Studio Build Tools install and starts rapidly consuming disk space (about 20GB consumed) but I still have 30GB to work with. Part way through the install it fails with the out-of-disk-space error:
I am guessing that docker build on Windows at least, does not allow the container to consume 127GB and has a low limit of 20GB? Is there any way around this? |
@jhowardmsft, certain installs I manage are also starting to grow beyond 127GB for a full install of all first- and third-party software. We will need larger sizes - something like my original change would have partly provided. Seems only fitting that if |
I'm working on a CI process for an application where my base layer is will be ~156GB. The 127GB cap is causing the build phase to fail. On top of this base layer, I intend to add components built by the current run of my CI process. I've added the following to my
|
An option that would also fit my scenario (but I haven't really found) would be the ability to mount a volume to a container having "writefilter" mechanism in use so that the volume itself is immutable, and writes to the volume only affect the layer representing the mount to the current container. |
This should be fixed by #41636 in the 21.xx release. There still isn't an implementation of |
20GB limit for COPY was never merged to 20.10.x branch. |
Do you have any kind of plan to solve this, we can not use windows containers with big images. |
Once Docker v22.06 is released, it will be possible to work with large images in The costs of making the config file number quite large are minimal (the VHD file created expands on-demand so builds won't use much more space when run with a larger size in the config) and since plumbing storage-opts through to the builder hasn't been done, I assume it's not seen as particularly high value for any contributors given the config-level setting bug-fix above. I'll note also that #35780 in 2018 attempted to implement this and hit some unresolved issues. I haven't checked to see if those issues still apply though. |
@TBBle it was done but rejected. It's been long enough I imagine it would have to be done again, but I'd be happy to contribute. The problem with the config file is that users have to change it every time they need to increase the build size. The convenience of plumbing through the command line option is being able to change it as needed only - same as with passing storage opts to |
I certainly wish this would be implemented -- ideally in the Dockerfile, but on the |
Description
We are working on containerizing a very large application where the default volume size may be insufficient but cannot pass
--storage-opt
todocker build
. Changing the daemon configuration may be a blocker for customers wishing to build upon our base image we're trying to build. We can change the amount of memory provided to the container(s) which is necessary during build, so it would be nice to be able to specify the volume size as well.But rather than merely support
storage-opt
for thebuild
command, would it make more sense to be allowed to pass through arguments frombuild
to use when running containers for eachRUN
statement?Output of
docker version
:Output of
docker info
:Additional environment details (AWS, VirtualBox, physical, etc.):
Using Hyper-V isolation on Windows 10 (currently all that's supported).
The text was updated successfully, but these errors were encountered: