Add quota support for storage backends

[fjakobs]

I understand that the aufs does not support quotas but since the device mapper backend mounts an ext4 partition it should be possible to enable quota support there. If the loopback device with the ext4 partition is mounted with the proper quota options I think it would be possible to use regular linux quota tools inside of a docker container.

[unclejack]

This feature is going to be added at some point, but not right away. It's a bit more difficult to add this functionality right now because a lot of chunks of code are moving from one place to another. After this work is done, it should be much easier to implement this functionality.

Please keep in mind that quota support can't be added as a hack to devicemapper, it has to be implemented for as many storage backends as possible, so it has to be implemented in a way which makes it easy to add quota support for other storage backends.

While I do understand that you need quota support for the container itself, wouldn't using bind mounts with quota help until quota support gets added to Docker?

[fjakobs]

Right now I'm using the fixed ext4 image size as a quota replacement but for this to work well I depend on #3548 being implemented.

If I wanted to hack quota support into my docker clone, where could I change the mount arguments?

[unclejack]

I've updated the title to make it more generic. We have more backends now and it wouldn't make sense to "hack" quota support into a specific backend.

[itsnotvalid]

Is there any updates to this problem?

[mtasic85]

Is there any way for docker to use BTRFS instead of AUFS?

[crosbymichael]

@mtasic85 yes, format your dive on btrfs and docker will use it automatically. or use the flag when starting the daemon docker -d -s btrfs

[mtasic85]

@crosbymichael Thanks. Is there way to know or specify CONTAINER ID before it is created? I'm asking because I want to create btrfs subvolume with quota and mount it to containers working dir.

[cscetbon]

Is there any progress concerning quota support ? I don't see any news since @itsnotvalid question

[sherter]

Since BTRFS support was added, you should be able to manually add quotas (see btrfs wiki page). However, I didn't try it myself.

[cscetbon]

yeah but I guess even device-mapper should support as it uses ext4/xfs as a filesystem. What I was looking for was a way to tell docker to apply it on the fly without having to find sub volumes (for Btrfs) after the creation of a new container

[majst01]

I implemented btrfs quota support here:

majst01/docker@docker:master...master
this is my first golang experience so there might be some stupidities.
This is not ready for a pull request because i am currently talking to original author of the btrfs code for a review. This code is kinda working so far but with limited testing.

So review and comments welcome.

For the long term i would like to have per container quotas. It should be possible to limit disk space the same way cpushares and memory is limited. But this would be a start.

[fza]

@majst01 .. +1, so much!!

[nfnty]

@majst01 Any updates on the btrfs quota support? Very interested in this feature.

[blackfader]

@majst01 Any updates on the btrfs quota support? Very interested in this feature. +1

[denghongcai]

Any updates? Seems nothing changed at all?

[SunWeicheng0001]

Any updates?

[xiaods]

Any updates?

[yijia2413]

ok now ?

[GordonTheTurtle]

USER POLL

The best way to get notified when there are changes in this discussion is by clicking the Subscribe button in the top right.

The people listed below have appreciated your meaningfull discussion with a random +1:

@rbugajewski

[cpuguy83]

Docker 1.8 supports blkio-weight.
#15879 adds blkio iops per device
#14466 adds read/write bps per device
#13959 adds blkio weight per device

[denghongcai]

@cpuguy83 features you listed here is not related with disk space quota...

[cpuguy83]

They are not "disk quota support", but they are related to the topic.

[arkkanoid]

So, it's not possible to use Disk Quotas inside a contanier for now?

[cpuguy83]

@arkkanoid correct

[arkkanoid]

And is there another way to limit the disk size for an specific user inside docker container?

[vote539]

The solution I ended up using, which is far from perfect but gets the job done, is to make a virtual, limited-size file system loop-mounted on the host, and then mount it as a volume on the container. This page has a good tutorial on setting up limited-size filesystems (the quota reporting mechanisms are optional):

http://souptonuts.sourceforge.net/quota_tutorial.html

One downside of this approach is that you need to have root permissions in order to mount the loop filesystem. (That's not really a problem though because you also need root permissions to control Docker. 😉 ) The other downside is that only the one directory is going to be quota'd. If the user inside the container tries to write a file outside of their designated directory, they can still take up space on the host filesystem. Depending on your setup, read/write permissions inside the container could help deal with that potential problem.

This obviously works only if you have control of the host machine.

[tjdett]

@vote539 This is essentially what the devicemapper storage backend does when used in loopback mode. To quote @jasonbrooks from Project Atomic:

Friends Don't Let Friends Run Docker on Loopback in Production

If this is just for dev, then stop reading now. You have a solution that works, just go with it.

If it's for prod, then you will probably get better performance from using:

• LVM logical volumes, or
• LVM thin pool volumes, or
• normal disk partitions, or
• a single BTRFS/ZFS filesystem based on one of the above, using subvolumes/datasets to limit directory space.

You should also consider if devicemapper would be an easier solution depending on the environment you're running Docker in.

[gegere]

Basically the solution for the Docker disk limit stuck at 20gb would be to share the growing file directory to the host system?

https://docs.docker.com/compose/compose-file/#volumes-volume-driver

    volumes:
   - "~/SITES/docker/mysql:/var/lib/mysql"

[tjdett]

@gegre You should store your data in a volume. That part you've definitely got right. The rest isn't good practice, and seems completely unrelated to the topic of this issue.

AFAIK, the only Docker storage backend which limits container disk by default is devicemapper. You can configure the daemon to adjust that size, so it's not "stuck" in any real sense.

Volumes use flat storage rather than layered storage, and use a completely different backend which can be extended via plugins. Simply making /var/lib/mysql a volume would ordinarily remove the 20GB limit from devicemapper, and also improve performance.

You do not need to map it to the host filesystem as you're suggesting - this is a pre-1.9 way of doing things, and was discouraged even then in favour of data only containers. Now that named volumes exist, you should use them for your data instead.

[cpuguy83]

FYI, quotas are now supported in devicemapper, zfs, and btrfs.
Probably won't ever be supported in aufs or overlay.

[mYmNeo]

Although disk quota is supported by ext4, but at most of time the user running in the container is root, it is not limited by disk quota

[treeder]

@cpuguy83 do you have links to any information on this?

[cpuguy83]

@treeder Looks like docs are not ready for that yet, though there is some mention in docs/reference/commandline/dockerd.md

Basically, can set --storage-opt size=1G at either the daemon level or at runtime.

[tobegit3hub]

This is closed but I'm not sure if it works for our requirements.

Are there issues about supporting ext4, aufs or overlayfs? Which version of docker daemon has merged the patch and we can start to use?

[ghost]

I think it isn’t working yet. I haven’t read that this feature already
implemented.

tobe notifications@github.com ezt írta (időpont: 2016. aug. 25., Cs 5:50):

This is closed but I'm not sure if it works for our requirements.

Does this work for btrfs, devicemapper or any storage backend? Which
version of docker daemon has merged the patch and we can start to use?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#3804 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/ANfEFcrBx5A56cxAMWcl5mI9ujFZxjjTks5qjRETgaJpZM4BdHcS
.

[AkihiroSuda]

@tobegit3hub @kovacsa91
PRs are available:
XFS+overlay2: #24771
XFS+overlay: #24807

[tobegit3hub]

Thanks @kovacsa91 and @AkihiroSuda . That's clear to me now 😃

--- Update ---

Refer to docs/reference/commandline/run.md, we can use docker create -it --storage-opt size=120G fedora /bin/bash now.

And after diving into the code, the devicemapper has been implemented in b16decf . And maybe btrfs, windowsfilter, and zfs have been added according to the doc. Please correct me if I'm wrong.

[bklau]

@AkihiroSuda What is --storage-opt size=xxx means for the various storage drivers?. Is 'xxx' means just the max size of the writable(CoW) layer in the container or the sum total of the container base image layers plus the writable(CoW) layer?. It's not clear to me. Also, what this "disk quota" field in the container.HostConfig?. How it is related to --storage-opt size=XXX ?

[timchenxiaoyu]

if I use devicemapper /aufs /overlay ...,all support storage-opt size?

[soullivaneuh]

This is not supported for aufs either vfs:

$ docker create -it --storage-opt size=5G debian /bin/bash
Error response from daemon: --storage-opt is not supported for aufs
(reverse-i-search)` sh': git^Chow HEAD

/ # docker create -it --storage-opt size=5G debian /bin/bash
Error response from daemon: --storage-opt is not supported for vfs

The issue should be re-opened.

[katsar0v]

Any updates on this issue?

[AkihiroSuda]

@katsar0v #24771

[geertschuring]

I get this error:

docker: Error response from daemon: --storage-opt is supported only for overlay over xfs with 'pquota' mount option.

Im quite surpised that setting a max on file storage is not supported by docker.

[cpuguy83]

@geertschuring Setting quotas is highly dependent on the underlying filesystem.

Quotas are support on btrfs, overlay2+xfs, and and devicemapper.

[samip5]

The issue indeed should be re-opened.

$ docker create --storage-opt size=500G -v /opt/docker/appdata/nextcloud:/var/www/html -v /opt/docker/appdata/nextcloud/custom_apps:/var/www/html/custom_apps -v /opt/docker/appdata/nextcloud/config:/var/www/html/config -v /opt/docker/appdata/nextcloud/data:/var/www/html/data nextcloud:apache nextcloud

Error response from daemon: --storage-opt is supported only for overlay over xfs with 'pquota' mount option

[cpuguy83]

@samip5 unless there is some generic tech to target, then no.
Quotas are supported when the underlying storage supports it.

[samip5]

@cpuguy83 To my understanding, ext4 is supposed to support quotas, so why is it not supported by Docker?

[cpuguy83]

Seems like you are looking for #29364 Brian Goff

…

On Feb 16, 2019, at 09:09, Sami Mäntysaari ***@***.***> wrote: @cpuguy83 To my understanding, ext4 is supposed to support quotas, so why is it not supported by Docker? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.