CLI flag for docker create(run) to change block device size. #19367

shishir-a412ed · 2016-01-15T15:53:15Z

Signed-off-by: Shishir Mahajan shishir.mahajan@redhat.com

shishir-a412ed · 2016-01-15T15:55:31Z

Now that daemon option (--storage-opt dm.basesize) has been introduced upstream, I would like to rekindle the discussion on introducing CLI flag (e.g. --storage-opt size:30G) for docker create/run which would allow a user to set containers rootfs size at creation time (provided it is greater than the base device size)

There were two major problems with the existing implementation (daemon option):

This daemon option would allow the user to expand the basedevice size, however that does not necessarily mean that all new containers would be of this new size. I ll give an example. Lets assume my current docker has 2 images (fedora, ubuntu) and 2 containers (fedora_container, ubuntu_container) with the default basedevice size=10G.
Now I restart the daemon and expanded the base device size to 30G (docker daemon --storage-opt dm.basesize=30G).

If I create a new container on any of the existing images (fedora, ubuntu) the rootfs size of those containers would still be 10G as they are snapshots of the existing image block device and not the basedevice which we just expanded. I feel this would be a little confusing to the user as he might be expecting the new size to show up. To make it work, I would need to remove my existing images (possibly containers associated with it) and repull them in order for my new containers rootfs to be of new base device size (dm.basesize=30G)

With this approach the heaviest application (container) would dictate the size for the rest of the containers. e.g. If I want to have 100 containers on my infrastructure and one of them is a data intensive application requiring 100G of space. I would have to set the base device size (dm.basesize=100G) to 100G. Even though there are other 99 containers which only needed 200 MB of space.

This solution gives more flexibility and solves these problems.

Shishir

shishir-a412ed · 2016-01-25T14:38:36Z

ping @calavera @rhvgoyal @tiborvass

Can we atleast start the design discussions ?

Shishir

shishir-a412ed · 2016-02-02T16:14:48Z

@calavera @rhvgoyal @tiborvass
Any updates on this one ?

Shishir

anusha-ragunathan · 2016-02-04T18:04:29Z

Please fix test failures.

shishir-a412ed · 2016-02-04T21:18:54Z

@anusha-ragunathan Fixed. Please check now.
Don't worry about the vendor failure. That change would eventually go to engine-api and will be vendored into docker. Lets atleast start the design discussions.

Shishir

thaJeztah · 2016-03-23T18:20:39Z

@shishir-a412ed it should be there (see 928ea1e)

shishir-a412ed · 2016-03-23T18:39:57Z

@thaJeztah Please check now.

thaJeztah · 2016-03-23T18:43:53Z

Thanks @shishir-a412ed, LGTM

thaJeztah · 2016-03-23T22:28:18Z

janky hitting #21408 (not related)

Signed-off-by: Shishir Mahajan <shishir.mahajan@redhat.com>

shishir-a412ed · 2016-03-28T14:12:08Z

@thaJeztah any updates on this one ?

thaJeztah · 2016-03-29T02:19:15Z

oh! I think it needs one more LGTM for the documentation

ping @vdemeester @MHBauer ptal

vdemeester · 2016-03-29T06:52:51Z

LGTM 🐮

mYmNeo · 2016-03-29T07:50:37Z

integration-cli/docker_cli_create_test.go

@@ -60,6 +60,27 @@ func (s *DockerSuite) TestCreateArgs(c *check.C) {

 }

+// Make sure we can grow the container's rootfs at creation time.
+func (s *DockerSuite) TestCreateGrowRootfs(c *check.C) {
+	testRequires(c, Devicemapper)


Is there something missing? I can not find any code about Devicemapper testRequirement

@mYmNeo here you go : https://github.com/docker/docker/blob/666563b190b7c97dc9db334ef03768ee8432b20e/integration-cli/requirements.go#L121

…ock device size Signed-off-by: Shishir Mahajan <shishir.mahajan@redhat.com>

tiborvass · 2016-06-17T18:07:05Z

docs/reference/commandline/create.md

+
+    $ docker create -it --storage-opt size=120G fedora /bin/bash
+
+This (size) will allow to set the container rootfs size to 120G at creation time. 


@thaJeztah @SvenDowideit it should be noted that htis is devicemapper specific. The flag is not, but the value is. Followup pr someone?

@tiborvass what are you looking for ?

do all storage drivers have a size option ?

@tiborvass I think this would be easier to discuss on irc. We can discuss this on Monday.

Note that this is no longer devicemapper specific. Windows will have support when #23391 is merged. ZFS also supports this since #21946. There might be more that I missed as well.

@tiborvass #23780

DrewRay · 2016-07-22T19:01:01Z

What about on docker build ? can we somehow get it to create the new images with a larger base size?

shishir-a412ed · 2016-07-23T19:42:41Z

@DrewRay Please see discussions going on at #22701

With this patch, I can create/run a container with a size bigger than base device size. However there were certain limitations.

Even if I start a container with bigger size, I cannot commit it. This affects docker commit.
I cannot build an image bigger than base device size. This affects docker build.
I cannot pull an image bigger than base device size. This affects docker pull

Upvote on the Issue #22701, If you feel this is a useful feature.
I have patches for all 3 of them. I can open a PR if there is enough interest in these features.

Shishir

CloudRAN need this. cherry-pick from moby#19367 Signed-off-by: Shishir Mahajan <shishir.mahajan@redhat.com> Signed-off-by: Lei Jitang <leijitang@huawei.com>

CLI flag for docker create(run) to change block device size CloudRAN need this. cherry-pick from moby#19367 Signed-off-by: Shishir Mahajan <shishir.mahajan@redhat.com> Signed-off-by: Lei Jitang <leijitang@huawei.com> See merge request docker/docker!245

GordonTheTurtle added the status/0-triage label Jan 15, 2016

calavera added status/1-design-review and removed status/0-triage labels Jan 15, 2016

shishir-a412ed force-pushed the rootfs_size_configurable_cli branch 5 times, most recently from 16e6c20 to 2808fff Compare January 22, 2016 19:49

thaJeztah mentioned this pull request Jan 25, 2016

Add disk quota support for btrfs #19651

Merged

shishir-a412ed force-pushed the rootfs_size_configurable_cli branch 8 times, most recently from 5650290 to 4c2efec Compare February 1, 2016 14:31

shishir-a412ed force-pushed the rootfs_size_configurable_cli branch from 4c2efec to 841fec4 Compare February 2, 2016 16:12

shishir-a412ed force-pushed the rootfs_size_configurable_cli branch 2 times, most recently from 8b690f2 to bb686db Compare February 4, 2016 15:00

shishir-a412ed force-pushed the rootfs_size_configurable_cli branch from bb686db to d51e5d3 Compare February 4, 2016 19:39

shishir-a412ed force-pushed the rootfs_size_configurable_cli branch from d51e5d3 to 13489ef Compare February 5, 2016 15:12

tiborvass added the status/needs-attention Calls for a collective discussion during a review session label Feb 5, 2016

shishir-a412ed force-pushed the rootfs_size_configurable_cli branch 2 times, most recently from 10581ff to 0144a96 Compare February 9, 2016 15:47

shishir-a412ed force-pushed the rootfs_size_configurable_cli branch 3 times, most recently from 0651e54 to 7521e1c Compare March 23, 2016 18:38

shishir-a412ed force-pushed the rootfs_size_configurable_cli branch 3 times, most recently from fdf91e4 to 59f002e Compare March 23, 2016 19:55

shishir-a412ed force-pushed the rootfs_size_configurable_cli branch from 59f002e to 9abddb1 Compare March 24, 2016 13:06

CLI flag for docker create(run) to change block device size.

b16decf

Signed-off-by: Shishir Mahajan <shishir.mahajan@redhat.com>

shishir-a412ed force-pushed the rootfs_size_configurable_cli branch from 9abddb1 to b16decf Compare March 28, 2016 14:05

vdemeester merged commit e6aa40a into moby:master Mar 29, 2016

mYmNeo reviewed Mar 29, 2016
View reviewed changes

thaJeztah added this to the 1.12.0 milestone Mar 29, 2016

kenXengineering mentioned this pull request Apr 11, 2016

Add support for setting storage size on ZFS containers #21946

Merged

shishir-a412ed pushed a commit to shishir-a412ed/docker that referenced this pull request May 5, 2016

BACKPORT: PR moby#19367: CLI flag for docker create(run) to change bl…

c7489ff

…ock device size Signed-off-by: Shishir Mahajan <shishir.mahajan@redhat.com>

thaJeztah mentioned this pull request Jun 17, 2016

v1.12.0: some fixes to CHANGELOG #23594

Merged

tiborvass reviewed Jun 17, 2016
View reviewed changes

albers mentioned this pull request Jun 21, 2016

bash completion for docker {run,create} --storage-opt #23803

Merged

jbweston mentioned this pull request Nov 22, 2016

Add support for storage opts in container_create docker/docker-py#1300

Closed

zaunerc mentioned this pull request Dec 23, 2016

Docker image for Oracle Database 12c EE fails to build oracle/docker-images#239

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CLI flag for docker create(run) to change block device size. #19367

CLI flag for docker create(run) to change block device size. #19367

shishir-a412ed commented Jan 15, 2016

shishir-a412ed commented Jan 15, 2016

shishir-a412ed commented Jan 25, 2016

shishir-a412ed commented Feb 2, 2016

anusha-ragunathan commented Feb 4, 2016

shishir-a412ed commented Feb 4, 2016

thaJeztah commented Mar 23, 2016

shishir-a412ed commented Mar 23, 2016

thaJeztah commented Mar 23, 2016

thaJeztah commented Mar 23, 2016

shishir-a412ed commented Mar 28, 2016

thaJeztah commented Mar 29, 2016

vdemeester commented Mar 29, 2016

mYmNeo Mar 29, 2016

vdemeester Mar 29, 2016

tiborvass Jun 17, 2016 •

edited

shishir-a412ed Jun 17, 2016

tiborvass Jun 17, 2016

shishir-a412ed Jun 17, 2016

darstahl Jun 20, 2016

shishir-a412ed Jun 20, 2016

DrewRay commented Jul 22, 2016

shishir-a412ed commented Jul 23, 2016


		$ docker create -it --storage-opt size=120G fedora /bin/bash

		This (size) will allow to set the container rootfs size to 120G at creation time.

CLI flag for docker create(run) to change block device size. #19367

CLI flag for docker create(run) to change block device size. #19367

Conversation

shishir-a412ed commented Jan 15, 2016

shishir-a412ed commented Jan 15, 2016

shishir-a412ed commented Jan 25, 2016

shishir-a412ed commented Feb 2, 2016

anusha-ragunathan commented Feb 4, 2016

shishir-a412ed commented Feb 4, 2016

thaJeztah commented Mar 23, 2016

shishir-a412ed commented Mar 23, 2016

thaJeztah commented Mar 23, 2016

thaJeztah commented Mar 23, 2016

shishir-a412ed commented Mar 28, 2016

thaJeztah commented Mar 29, 2016

vdemeester commented Mar 29, 2016

mYmNeo Mar 29, 2016

Choose a reason for hiding this comment

vdemeester Mar 29, 2016

Choose a reason for hiding this comment

tiborvass Jun 17, 2016 • edited

Choose a reason for hiding this comment

shishir-a412ed Jun 17, 2016

Choose a reason for hiding this comment

tiborvass Jun 17, 2016

Choose a reason for hiding this comment

shishir-a412ed Jun 17, 2016

Choose a reason for hiding this comment

darstahl Jun 20, 2016

Choose a reason for hiding this comment

shishir-a412ed Jun 20, 2016

Choose a reason for hiding this comment

DrewRay commented Jul 22, 2016

shishir-a412ed commented Jul 23, 2016

tiborvass Jun 17, 2016 •

edited