Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot specify --storage-opt to "docker build" #34947

Open
heaths opened this issue Sep 22, 2017 · 13 comments

Comments

Projects
None yet
10 participants
@heaths
Copy link

commented Sep 22, 2017

Description

We are working on containerizing a very large application where the default volume size may be insufficient but cannot pass --storage-opt to docker build. Changing the daemon configuration may be a blocker for customers wishing to build upon our base image we're trying to build. We can change the amount of memory provided to the container(s) which is necessary during build, so it would be nice to be able to specify the volume size as well.

But rather than merely support storage-opt for the build command, would it make more sense to be allowed to pass through arguments from build to use when running containers for each RUN statement?

Output of docker version:

Client:
 Version:      17.06.2-ce
 API version:  1.30
 Go version:   go1.8.3
 Git commit:   cec0b72
 Built:        Tue Sep  5 19:57:19 2017
 OS/Arch:      windows/amd64

Server:
 Version:      17.06.2-ce
 API version:  1.30 (minimum version 1.24)
 Go version:   go1.8.3
 Git commit:   cec0b72
 Built:        Tue Sep  5 19:59:47 2017
 OS/Arch:      windows/amd64
 Experimental: true

Output of docker info:

Containers: 1
 Running: 1
 Paused: 0
 Stopped: 0
Images: 28
Server Version: 17.06.2-ce
Storage Driver: windowsfilter
 Windows: 
Logging Driver: json-file
Plugins: 
 Volume: local
 Network: l2bridge l2tunnel nat null overlay transparent
 Log: awslogs etwlogs fluentd json-file logentries splunk syslog
Swarm: inactive
Default Isolation: hyperv
Kernel Version: 10.0 15063 (15063.0.amd64fre.rs2_release.170317-1834)
Operating System: Windows 10 Enterprise
OSType: windows
Architecture: x86_64
CPUs: 8
Total Memory: 31.92GiB
Name: heathswrk
ID: PPYQ:6372:4QVR:ERCP:U676:ZHX5:T6FG:RKYE:O4XN:FDP4:7DMT:CQ7P
Docker Root Dir: E:\Docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: -1
 Goroutines: 32
 System Time: 2017-09-22T13:28:45.4667296-07:00
 EventsListeners: 0
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.):

Using Hyper-V isolation on Windows 10 (currently all that's supported).

@heaths

This comment has been minimized.

Copy link
Author

commented Sep 22, 2017

I should mention I'm aware of docker/for-win#999 which also creates a problem since we have no way to increase volume size for docker build until that change is released to the stable build (since customers may not want to run preview bits).

@friism

This comment has been minimized.

Copy link
Contributor

commented Sep 22, 2017

Docker CE 17.09 (stable) shouldn't be that far of, and will have the fix in the engine.

Further, for the users that are consuming your base image, will they also be generating incredibly large layers on top of your base image?

@heaths

This comment has been minimized.

Copy link
Author

commented Sep 22, 2017

Possibly. There's basically two goals here. We're containerizing Build Tools for Visual Studio 2017 but its not yet as feature-rich (i.e. doesn't support building all common project types) as Visual Studio 2017 itself, so even if we made a base image of Build Tools available on which customers may build, customers may also need to install Visual Studio 2017 into an image for now. So besides chasing down any blocking bugs to just building a complete image ourselves, I also need to provide guidance for customers building their own. So being able to pass more options (or just specific ones, like --storage-opt) to control intermediate containers would be very helpful.

@darstahl

This comment has been minimized.

Copy link
Contributor

commented Sep 22, 2017

I agree that this is something we want. We've had multiple asks for this (ex. #33469), and it's much more reasonable to have a base image size specified at build time than being a global daemon option imo.

cc @jhowardmsft @johnstep

@heaths

This comment has been minimized.

Copy link
Author

commented Oct 21, 2017

After discussing different options for implementation with @jhowardmsft and @johnstep, I'm working on a change to add --storage-opt to the build command akin to the run command.

@IvanBoyko

This comment has been minimized.

Copy link

commented Nov 29, 2018

Apologies for bringing up an old topic, but it seems it was never implemented.

We've been using VS Build Tools 2017 since August 2017, and never had this problem.
Our installation works, although I do see in %temp%\dd_client_*.log Errors like this:

2018-11-29T07:35:46 : Error : Visual Studio needs at least 25.63 GB of disk space. You might want to free up space on C:\ or change your target drive.

@heaths

This comment has been minimized.

Copy link
Author

commented Nov 29, 2018

@johnstep created PR #35780 but it was rejected after the default size was raised to 127GB (same as the default for Hyper-V). Apart from being rather short-sighted, IMO (using build tools containers for large solutions might require even more space, which means it should be configurable), I don't know if this made it into Docker 4 Windows in either the edge or stable channels. It's been nearly a year so I'd hope so.

@crossan007

This comment has been minimized.

Copy link

commented May 23, 2019

Following this, as I'm also looking to create large base images.

@jhowardmsft

This comment has been minimized.

Copy link
Contributor

commented Jun 6, 2019

@crossan007 How large? The default scratch is now 127GB. Bigger than that?

@JoshuaKing

This comment has been minimized.

Copy link

commented Jun 28, 2019

@jhowardmsft New here. By default scratch do you mean the max size is 127GB for a container being built? I ask because I'm trying to build a simple container following Microsoft's instructions (install Visual Studio Build Tools).

When building, after pulling and extracting the images I still have 50GB on the windows host machine remaining. Then it conducts the Visual Studio Build Tools install and starts rapidly consuming disk space (about 20GB consumed) but I still have 30GB to work with. Part way through the install it fails with the out-of-disk-space error:

Error 0x80070070: Failed to write to part stream.
Error 0x80070070: Failed to write to archive file stream.
!ERROR: 0x80070070 [There is not enough space on the disk.]

I am guessing that docker build on Windows at least, does not allow the container to consume 127GB and has a low limit of 20GB? Is there any way around this?

@heaths

This comment has been minimized.

Copy link
Author

commented Jun 28, 2019

@jhowardmsft, certain installs I manage are also starting to grow beyond 127GB for a full install of all first- and third-party software. We will need larger sizes - something like my original change would have partly provided. Seems only fitting that if docker run can specify a larger size, docker build should be able to as well.

@crossan007

This comment has been minimized.

Copy link

commented Jul 10, 2019

I'm working on a CI process for an application where my base layer is will be ~156GB. The 127GB cap is causing the build phase to fail.

On top of this base layer, I intend to add components built by the current run of my CI process.

I've added the following to my C:\ProgramData\Docker\Config\Daemon.json file, but seems to have no effect:

"storage-opts": [
    "size=300GB"
  ]
@crossan007

This comment has been minimized.

Copy link

commented Jul 10, 2019

An option that would also fit my scenario (but I haven't really found) would be the ability to mount a volume to a container having "writefilter" mechanism in use so that the volume itself is immutable, and writes to the volume only affect the layer representing the mount to the current container.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.