Add size option to volumes on linux/unix via xfs pquota

[BtbN]

- What I did

This PR uses the existing quota module to add an option to apply xfs project quotas on volumes.

- How I did it

The first part modifies the existing projectquota code to be ready for handling multiple directories.
It needs a way to avoid projectID re-use between the two, since otherwise a volume and a container could end up sharing a quota.
For this, a global singleton is introduced that manages the next free projectID in a thread-safe manner.
Since the local volume driver uses a subdir of the base dir for the actual storage (volumename/_data), the projectID scanning code in findNextProjectID is extended to take one level of sub directories into account, provided no projectID is set on the upper level dir already.

The second part then adds support for projectquota to the volume/local driver.
Some modifications were needed in order to allow for options being present without forcing a mount.
In turn, the mandatoryOpts now refer to a list of other options, which are all required to be set if the respective option itself is set, thus allowing size to be set without requiring all the other ones, while still maintaining the previous constraints.
Instead of checking if any options are set in order to determine if a mount call is needed, local.go now calls a new helper function specifically for that purpose, which is then implemented in the driver based on the set options.

Finally, the actual quota support is added, by adding a new size option, which when set is applied via the projectquota module on volume mount, utilizing the new postMount function which is called unconditionally after (potentially) mounting the volume.

- How to verify it

docker volume create -o size=1G testvol
docker run -ti --rm -v testvol:/home ubuntu:latest /bin/bash
# df -h /
# df -h /home

This obviously needs dockerd to run with /var/lib/docker being on xfs with pquota support enabled.

- Description for the changelog

Added support for volume size quotas via xfs project quotas.

- TODO

• Add documentation
• Potentially add an option to manually specify a projectID for both volumes and overlay2, to give users an option to intentionally share a quota across multiple containers/volumes. Though that's probably for a future PR.

[BtbN]

This failure of docker-py in ContainerTest.test_remove looks unrelated? At least I don't see how this PR would interfere with that at all.

[AkihiroSuda]

Please use real name for signing

[AkihiroSuda]

Also, could you add integration tests

[AkihiroSuda]

@thaJeztah @cpuguy83

Do you think this is safe to bring in 20.XX? Or better to defer until 21.XX?

[BtbN]

Also, could you add integration tests

I'll look into that, though it's a bit complicated given that it relies on xfs and specific mount options.

[kolyshkin]

LGTM

[thaJeztah]

Discussing: looking at #33903; it may be good to put in the documentation that it's implemented as a quota.

Wondering if the option should be (e.g.) max-size instead of size to make clearer that it's restricting the maximum size that can be consumed, and doesn't reserve the specified size (or guarantee the size)

WDYT?

[cpuguy83]

I'm not very familiar these paths.
How big can this be?

I have a feeling we should read the dir with a fixed buffer size.

[BtbN]

It's not too crazy, given that both overlay2 and local have another layer of subdirs below their top level dir, it's never going to actually reach a users FS to enumerate.
Also, this functions only runs once at daemon startup to build the initial state, and never at runtime.

[thaJeztah]

ping @cpuguy83 ptal ^

[thaJeztah]

Discussing size vs max-size with @cpuguy83 @kolyshkin and given that some drivers already use size, it's probably ok to keep as size 👍

[BtbN]

I can absolutely see the argument that size might be a bit too generic of a name.
My idea would be "quota-size" or something along those lines. Reason I stuck to just "size" was to keep it in line with the already existing size option on the overlay2 graph driver.

[BtbN]

Something that needs changing still? I'm not sure how exactly to adjust the initial scan, but I really don't think it's a performance issue, even if it scans thousands of dirs, it's still fast and only done once.

[thaJeztah]

Reason I stuck to just "size" was to keep it in line with the already existing size option on the overlay2 graph driver.

Yes, I think "size" is ok to keep

[cpuguy83]

Some questions on the implementation. I don't understand the changes to mandatory opts handlings.

Other than that, I tested this both on an unsupported config and a supported one and this all looks good.

[cpuguy83]

Maybe pass quotaEnabled as a bool rather than the whole volume

[BtbN]

The idea here is to keep the function signature more generic, so that potential future options which need info from the volume don't have to add another one.
Passing in quotaEnabled would also just mean the check for v.quotaCtl == nil would move one function up, to then be passed in as bool. I think I prefer the current approach.

Though it does cause a bit of code-duplication between this and the actual quota-application in setOpts.
So maybe the check here can be removed entirely, and setOpts checks for quota support itself and bails if it's missing?

[BtbN]

Refactored this to get rid of the issue entirely and avoid the logic duplication.

[cpuguy83]

Can you explain the reason behind the changes around this?

[BtbN]

Prior to this, as soon as any option on the local_unix driver was set, all options in this map(or rather, set) had to be provided.
This is not desirable for the size option, since it can very well stand on its own.
So the new logic changes this to a map of string-arrays, where the array elements indicate which other options are required if the option which is they key is set.
That way all prior mandatory constraints are still maintained, while size can stand on its own without requiring any other options to be set.

[thaJeztah]

@cpuguy83 ptal

[BtbN]

I slightly refactored the validateOpts part to remove the logic-duplication it had.
The failing tests all look unrelated and seem to be caused by network issues on one test runner.

[cpuguy83]

LGTM

[thaJeztah]

LGTM

[max-wittig]

Is there an option to set a global size limit per container? Like have some sort of setting inside daemon.json?