Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add quota support for storage backends #3804

Closed
fjakobs opened this issue Jan 28, 2014 · 60 comments
Closed

Add quota support for storage backends #3804

fjakobs opened this issue Jan 28, 2014 · 60 comments

Comments

@fjakobs
Copy link

@fjakobs fjakobs commented Jan 28, 2014

I understand that the aufs does not support quotas but since the device mapper backend mounts an ext4 partition it should be possible to enable quota support there. If the loopback device with the ext4 partition is mounted with the proper quota options I think it would be possible to use regular linux quota tools inside of a docker container.

@unclejack
Copy link
Contributor

@unclejack unclejack commented Jan 28, 2014

This feature is going to be added at some point, but not right away. It's a bit more difficult to add this functionality right now because a lot of chunks of code are moving from one place to another. After this work is done, it should be much easier to implement this functionality.

Please keep in mind that quota support can't be added as a hack to devicemapper, it has to be implemented for as many storage backends as possible, so it has to be implemented in a way which makes it easy to add quota support for other storage backends.

While I do understand that you need quota support for the container itself, wouldn't using bind mounts with quota help until quota support gets added to Docker?

@fjakobs
Copy link
Author

@fjakobs fjakobs commented Jan 28, 2014

Right now I'm using the fixed ext4 image size as a quota replacement but for this to work well I depend on #3548 being implemented.

If I wanted to hack quota support into my docker clone, where could I change the mount arguments?

@unclejack
Copy link
Contributor

@unclejack unclejack commented Feb 26, 2014

I've updated the title to make it more generic. We have more backends now and it wouldn't make sense to "hack" quota support into a specific backend.

@itsnotvalid
Copy link

@itsnotvalid itsnotvalid commented May 24, 2014

Is there any updates to this problem?

@mtasic85
Copy link

@mtasic85 mtasic85 commented Jun 7, 2014

Is there any way for docker to use BTRFS instead of AUFS?

@crosbymichael
Copy link
Contributor

@crosbymichael crosbymichael commented Jun 7, 2014

@mtasic85 yes, format your dive on btrfs and docker will use it automatically. or use the flag when starting the daemon docker -d -s btrfs

@mtasic85
Copy link

@mtasic85 mtasic85 commented Jun 7, 2014

@crosbymichael Thanks. Is there way to know or specify CONTAINER ID before it is created? I'm asking because I want to create btrfs subvolume with quota and mount it to containers working dir.

@cscetbon
Copy link

@cscetbon cscetbon commented Aug 11, 2014

Is there any progress concerning quota support ? I don't see any news since @itsnotvalid question

@sherter
Copy link

@sherter sherter commented Aug 12, 2014

Since BTRFS support was added, you should be able to manually add quotas (see btrfs wiki page). However, I didn't try it myself.

@cscetbon
Copy link

@cscetbon cscetbon commented Aug 12, 2014

yeah but I guess even device-mapper should support as it uses ext4/xfs as a filesystem. What I was looking for was a way to tell docker to apply it on the fly without having to find sub volumes (for Btrfs) after the creation of a new container

@majst01
Copy link

@majst01 majst01 commented Nov 23, 2014

I implemented btrfs quota support here:

majst01/docker@docker:master...master
this is my first golang experience so there might be some stupidities.
This is not ready for a pull request because i am currently talking to original author of the btrfs code for a review. This code is kinda working so far but with limited testing.

So review and comments welcome.

For the long term i would like to have per container quotas. It should be possible to limit disk space the same way cpushares and memory is limited. But this would be a start.

@fza
Copy link

@fza fza commented Nov 23, 2014

@majst01 .. +1, so much!!

@nfnty
Copy link

@nfnty nfnty commented May 17, 2015

@majst01 Any updates on the btrfs quota support? Very interested in this feature.

@blackfader
Copy link

@blackfader blackfader commented May 28, 2015

@majst01 Any updates on the btrfs quota support? Very interested in this feature. +1

@denghongcai
Copy link

@denghongcai denghongcai commented Jul 23, 2015

Any updates? Seems nothing changed at all?

@SunWeicheng0001
Copy link

@SunWeicheng0001 SunWeicheng0001 commented Aug 3, 2015

Any updates?

1 similar comment
@xiaods
Copy link
Contributor

@xiaods xiaods commented Aug 3, 2015

Any updates?

@yijia2413
Copy link

@yijia2413 yijia2413 commented Aug 24, 2015

ok now ?

@GordonTheTurtle
Copy link

@GordonTheTurtle GordonTheTurtle commented Aug 28, 2015

USER POLL

The best way to get notified when there are changes in this discussion is by clicking the Subscribe button in the top right.

The people listed below have appreciated your meaningfull discussion with a random +1:

@rbugajewski

@cpuguy83
Copy link
Collaborator

@cpuguy83 cpuguy83 commented Aug 28, 2015

Docker 1.8 supports blkio-weight.
#15879 adds blkio iops per device
#14466 adds read/write bps per device
#13959 adds blkio weight per device

@denghongcai
Copy link

@denghongcai denghongcai commented Aug 29, 2015

@cpuguy83 features you listed here is not related with disk space quota...

@cpuguy83
Copy link
Collaborator

@cpuguy83 cpuguy83 commented Aug 29, 2015

They are not "disk quota support", but they are related to the topic.

@gegere
Copy link

@gegere gegere commented Apr 2, 2016

Basically the solution for the Docker disk limit stuck at 20gb would be to share the growing file directory to the host system?

https://docs.docker.com/compose/compose-file/#volumes-volume-driver

    volumes:
   - "~/SITES/docker/mysql:/var/lib/mysql"
@tjdett
Copy link
Contributor

@tjdett tjdett commented Apr 4, 2016

@gegre You should store your data in a volume. That part you've definitely got right. The rest isn't good practice, and seems completely unrelated to the topic of this issue.

AFAIK, the only Docker storage backend which limits container disk by default is devicemapper. You can configure the daemon to adjust that size, so it's not "stuck" in any real sense.

Volumes use flat storage rather than layered storage, and use a completely different backend which can be extended via plugins. Simply making /var/lib/mysql a volume would ordinarily remove the 20GB limit from devicemapper, and also improve performance.

You do not need to map it to the host filesystem as you're suggesting - this is a pre-1.9 way of doing things, and was discouraged even then in favour of data only containers. Now that named volumes exist, you should use them for your data instead.

@cpuguy83
Copy link
Collaborator

@cpuguy83 cpuguy83 commented Jun 27, 2016

FYI, quotas are now supported in devicemapper, zfs, and btrfs.
Probably won't ever be supported in aufs or overlay.

@mYmNeo
Copy link
Contributor

@mYmNeo mYmNeo commented Jun 28, 2016

Although disk quota is supported by ext4, but at most of time the user running in the container is root, it is not limited by disk quota

@treeder
Copy link

@treeder treeder commented Jun 28, 2016

@cpuguy83 do you have links to any information on this?

@cpuguy83
Copy link
Collaborator

@cpuguy83 cpuguy83 commented Jun 28, 2016

@treeder Looks like docs are not ready for that yet, though there is some mention in docs/reference/commandline/dockerd.md

Basically, can set --storage-opt size=1G at either the daemon level or at runtime.

@tobegit3hub
Copy link

@tobegit3hub tobegit3hub commented Aug 25, 2016

This is closed but I'm not sure if it works for our requirements.

Are there issues about supporting ext4, aufs or overlayfs? Which version of docker daemon has merged the patch and we can start to use?

@ghost
Copy link

@ghost ghost commented Aug 25, 2016

I think it isn’t working yet. I haven’t read that this feature already
implemented.

tobe notifications@github.com ezt írta (időpont: 2016. aug. 25., Cs 5:50):

This is closed but I'm not sure if it works for our requirements.

Does this work for btrfs, devicemapper or any storage backend? Which
version of docker daemon has merged the patch and we can start to use?


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#3804 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/ANfEFcrBx5A56cxAMWcl5mI9ujFZxjjTks5qjRETgaJpZM4BdHcS
.

@AkihiroSuda
Copy link
Member

@AkihiroSuda AkihiroSuda commented Aug 25, 2016

@tobegit3hub @kovacsa91
PRs are available:
XFS+overlay2: #24771
XFS+overlay: #24807

@tobegit3hub
Copy link

@tobegit3hub tobegit3hub commented Aug 25, 2016

Thanks @kovacsa91 and @AkihiroSuda . That's clear to me now 😃

--- Update ---

Refer to docs/reference/commandline/run.md, we can use docker create -it --storage-opt size=120G fedora /bin/bash now.

And after diving into the code, the devicemapper has been implemented in b16decf . And maybe btrfs, windowsfilter, and zfs have been added according to the doc. Please correct me if I'm wrong.

@bklau
Copy link

@bklau bklau commented Jun 14, 2017

@AkihiroSuda What is --storage-opt size=xxx means for the various storage drivers?. Is 'xxx' means just the max size of the writable(CoW) layer in the container or the sum total of the container base image layers plus the writable(CoW) layer?. It's not clear to me. Also, what this "disk quota" field in the container.HostConfig?. How it is related to --storage-opt size=XXX ?

@timchenxiaoyu
Copy link
Contributor

@timchenxiaoyu timchenxiaoyu commented Sep 25, 2017

if I use devicemapper /aufs /overlay ...,all support storage-opt size?

@soullivaneuh
Copy link

@soullivaneuh soullivaneuh commented Oct 14, 2017

This is not supported for aufs either vfs:

$ docker create -it --storage-opt size=5G debian /bin/bash
Error response from daemon: --storage-opt is not supported for aufs
(reverse-i-search)` sh': git^Chow HEAD
/ # docker create -it --storage-opt size=5G debian /bin/bash
Error response from daemon: --storage-opt is not supported for vfs

The issue should be re-opened.

@katsar0v
Copy link

@katsar0v katsar0v commented Oct 17, 2018

Any updates on this issue?

@AkihiroSuda
Copy link
Member

@AkihiroSuda AkihiroSuda commented Oct 17, 2018

@katsar0v #24771

@geertschuring
Copy link

@geertschuring geertschuring commented Nov 5, 2018

I get this error:

docker: Error response from daemon: --storage-opt is supported only for overlay over xfs with 'pquota' mount option.

Im quite surpised that setting a max on file storage is not supported by docker.

@cpuguy83
Copy link
Collaborator

@cpuguy83 cpuguy83 commented Nov 5, 2018

@geertschuring Setting quotas is highly dependent on the underlying filesystem.

Quotas are support on btrfs, overlay2+xfs, and and devicemapper.

@samip5
Copy link

@samip5 samip5 commented Feb 16, 2019

The issue indeed should be re-opened.

$ docker create --storage-opt size=500G -v /opt/docker/appdata/nextcloud:/var/www/html -v /opt/docker/appdata/nextcloud/custom_apps:/var/www/html/custom_apps -v /opt/docker/appdata/nextcloud/config:/var/www/html/config -v /opt/docker/appdata/nextcloud/data:/var/www/html/data nextcloud:apache nextcloud

Error response from daemon: --storage-opt is supported only for overlay over xfs with 'pquota' mount option
@cpuguy83
Copy link
Collaborator

@cpuguy83 cpuguy83 commented Feb 16, 2019

@samip5 unless there is some generic tech to target, then no.
Quotas are supported when the underlying storage supports it.

@samip5
Copy link

@samip5 samip5 commented Feb 16, 2019

@cpuguy83 To my understanding, ext4 is supposed to support quotas, so why is it not supported by Docker?

@cpuguy83
Copy link
Collaborator

@cpuguy83 cpuguy83 commented Feb 16, 2019

@andres-fr
Copy link

@andres-fr andres-fr commented Nov 9, 2020

@samip5 unless there is some generic tech to target, then no.
Quotas are supported when the underlying storage supports it.

I don't think that is a sufficient argument. By that token, we wouldn't have cross-platform functionalities on the first place right?

Let me elaborate: I understand and respect that this may require exta abstraction, and that this may look ugly etc. But I specifically mean the idea of "underlying storage supporting a generic tech". Cheers!

@cpuguy83
Copy link
Collaborator

@cpuguy83 cpuguy83 commented Nov 9, 2020

@andres-fr I'm not sure what you are getting at.
Docker supports quotas, but it is totally dependent on the underlying filesystem implementation.

For instance, with xfs you need the pquota mount option.
These are abstracted away as the --storage-opt size=<val> option.

@andres-fr
Copy link

@andres-fr andres-fr commented Nov 10, 2020

@cpuguy83 TLDR: I think I'm at the wrong issue, sorry for the noise.


To give more context, I was under the impression that this issue was about allowing the Docker host to have a disk usage limit (e.g. something in the form images+containers+volumes <= 100GB). After some searching I came to the conclusion that this functionality is not provided by Docker, and that the reason you provide is backend heterogeneity. Maybe at this point I dare to ask, do you know if this is this possible now?

@cpuguy83
Copy link
Collaborator

@cpuguy83 cpuguy83 commented Nov 10, 2020

Docker will not impose such a limit.
You can put docker's root onto a separate disk/partition/quota.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet