New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove btrfs quota groups after containers destroyed #29427

Merged
merged 2 commits into from Jun 6, 2017

Conversation

@yongtang
Member

yongtang commented Dec 14, 2016

- What I did

This fix tries to address the issue raised in #29325 where btrfs quota groups are not clean up even after containers have been destroyed.

- How I did it

The reason for the issue is that btrfs quota groups have to be explicitly destroyed. This fix fixes this issue.

- How to verify it

This fix is tested manually in Ubuntu 16.04:

pre-fix behavior with qgroup leftover:

ubuntu@ubuntu:~$ docker pull busybox
Using default tag: latest
latest: Pulling from library/busybox
4b0bc1c4050b: Pull complete
Digest: sha256:817a12c32a39bbe394944ba49de563e085f1d3c5266eb8e9723256bc4448680e
Status: Downloaded newer image for busybox:latest
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         128.00KiB    128.00KiB
0/257       112.00KiB    112.00KiB
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker run --name top -d busybox top
b59e6e3da6ed3e601f081c11c44d3f4dd44aefee5584d34ba913e68207c9694d
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         128.00KiB    128.00KiB
0/257         1.17MiB     64.00KiB
0/258         1.17MiB     16.00KiB
0/259         1.17MiB     16.00KiB
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker rm -f top
top
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         144.00KiB    144.00KiB
0/257         1.17MiB      1.17MiB
0/258           0.00B        0.00B
0/259           0.00B        0.00B

PR with qgroup cleaned (non --live-restore case):

ubuntu@ubuntu:~$ docker pull busybox
Using default tag: latest
latest: Pulling from library/busybox
4b0bc1c4050b: Pull complete
Digest: sha256:817a12c32a39bbe394944ba49de563e085f1d3c5266eb8e9723256bc4448680e
Status: Downloaded newer image for busybox:latest
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         128.00KiB    128.00KiB
0/257       112.00KiB    112.00KiB
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker run --name top -d busybox top
201d444f74943247c7794fae2215f763cad0dd726d03285004c54ffa04882d6b
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         128.00KiB    128.00KiB
0/257         1.17MiB     64.00KiB
0/258         1.17MiB     16.00KiB
0/259         1.17MiB     16.00KiB
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker rm -f top
top
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         144.00KiB    144.00KiB
0/257         1.17MiB      1.17MiB

PR with qgroup cleaned (--live-restore case):

# [sudo dockerd --debug --live-restore -s btrfs --storage-opt btrfs.min_space=1G]

ubuntu@ubuntu:~$ docker pull busybox
Using default tag: latest
latest: Pulling from library/busybox
4b0bc1c4050b: Pull complete
Digest: sha256:817a12c32a39bbe394944ba49de563e085f1d3c5266eb8e9723256bc4448680e
Status: Downloaded newer image for busybox:latest
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         128.00KiB    128.00KiB
0/257       112.00KiB    112.00KiB
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker run --name top -d busybox top
73c2184a4414d47b5f5dc27cd2ee28116aa761f947196618ca66bf05c4705456
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         128.00KiB    128.00KiB
0/257         1.17MiB     64.00KiB
0/258         1.17MiB     16.00KiB
0/259         1.17MiB     16.00KiB
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$

# [Ctrl-C ]
# [sudo dockerd --debug --live-restore -s btrfs]

ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED              STATUS              PORTS               NAMES
73c2184a4414        busybox             "top"               About a minute ago   Up About a minute                       top
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         152.00KiB    152.00KiB
0/257         1.17MiB     64.00KiB
0/258         1.17MiB     48.00KiB
0/259         1.17MiB     48.00KiB
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker rm -f top
top
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         168.00KiB    168.00KiB
0/257         1.17MiB     64.00KiB

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

c03904d6028381a6bd0d1b1bfd1523bb

This fix fixes #29325.

Signed-off-by: Yong Tang yong.tang.github@outlook.com

@yongtang

This comment has been minimized.

Show comment
Hide comment
@yongtang
Member

yongtang commented Dec 19, 2016

@zhuguihua

This comment has been minimized.

Show comment
Hide comment
@zhuguihua

zhuguihua Dec 28, 2016

Contributor

Sorry for late reply. LGTM.

Contributor

zhuguihua commented Dec 28, 2016

Sorry for late reply. LGTM.

@yongtang

This comment has been minimized.

Show comment
Hide comment
@yongtang

yongtang Jan 3, 2017

Member

Previously the PR didn't take into consideration that, if quotaEnabled is false, there is no need to clean up the qgroup (See #29810 for similar issue)

Now the PR has been updated to add the check for quotaEnabled.

Member

yongtang commented Jan 3, 2017

Previously the PR didn't take into consideration that, if quotaEnabled is false, there is no need to clean up the qgroup (See #29810 for similar issue)

Now the PR has been updated to add the check for quotaEnabled.

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Jan 3, 2017

Member

Also wondering the same here as on #29835 (comment)

Member

thaJeztah commented Jan 3, 2017

Also wondering the same here as on #29835 (comment)

@mlaventure

This comment has been minimized.

Show comment
Hide comment
@mlaventure

mlaventure Jan 3, 2017

Contributor

I had a quick look at the code, it looks like @thaJeztah may be correct.

Would wait on @cpuguy83 for confirmation

Contributor

mlaventure commented Jan 3, 2017

I had a quick look at the code, it looks like @thaJeztah may be correct.

Would wait on @cpuguy83 for confirmation

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Jan 3, 2017

Member

if I'm correct, we may need some mechanism to detect if a quota is set (perhaps querying btrfs), otherwise it's possible we don't know when to cleanup / manage quotas

Member

thaJeztah commented Jan 3, 2017

if I'm correct, we may need some mechanism to detect if a quota is set (perhaps querying btrfs), otherwise it's possible we don't know when to cleanup / manage quotas

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Jan 3, 2017

Contributor

I think you are right. On daemon restore it could really be a problem.

Contributor

cpuguy83 commented Jan 3, 2017

I think you are right. On daemon restore it could really be a problem.

@LK4D4

This comment has been minimized.

Show comment
Hide comment
@LK4D4

LK4D4 Jan 26, 2017

Contributor

@cpuguy83 @tonistiigi @yongtang so, what we're going to do with this and #29385
If we have no proper solution for now, I propose to close PRs.
Thanks!

Contributor

LK4D4 commented Jan 26, 2017

@cpuguy83 @tonistiigi @yongtang so, what we're going to do with this and #29385
If we have no proper solution for now, I propose to close PRs.
Thanks!

@yongtang

This comment has been minimized.

Show comment
Hide comment
@yongtang

yongtang Jan 26, 2017

Member

@cpuguy83 @tonistiigi @LK4D4 This PR (#29427) and the other PR (#29835) are related.

I updated the other PR #29835 first to try to address the issue by looking up the qgroup and see if it returns something.

Please take a look at the (#29835) first and see if it address the concerns.

Member

yongtang commented Jan 26, 2017

@cpuguy83 @tonistiigi @LK4D4 This PR (#29427) and the other PR (#29835) are related.

I updated the other PR #29835 first to try to address the issue by looking up the qgroup and see if it returns something.

Please take a look at the (#29835) first and see if it address the concerns.

@yongtang

This comment has been minimized.

Show comment
Hide comment
@yongtang

yongtang Jan 28, 2017

Member

Here is the complete steps to verify that the issue is fixed (Including --live-restore case).

Would be good to setup integration test for btrfs in Jenkins

Note: this PR has been updated to incorporate #30497. If this one is merged then #30497 could be closed

pre-fix behavior with qgroup leftover:

ubuntu@ubuntu:~$ docker pull busybox
Using default tag: latest
latest: Pulling from library/busybox
4b0bc1c4050b: Pull complete
Digest: sha256:817a12c32a39bbe394944ba49de563e085f1d3c5266eb8e9723256bc4448680e
Status: Downloaded newer image for busybox:latest
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         128.00KiB    128.00KiB
0/257       112.00KiB    112.00KiB
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker run --name top -d busybox top
b59e6e3da6ed3e601f081c11c44d3f4dd44aefee5584d34ba913e68207c9694d
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         128.00KiB    128.00KiB
0/257         1.17MiB     64.00KiB
0/258         1.17MiB     16.00KiB
0/259         1.17MiB     16.00KiB
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker rm -f top
top
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         144.00KiB    144.00KiB
0/257         1.17MiB      1.17MiB
0/258           0.00B        0.00B
0/259           0.00B        0.00B

PR with qgroup cleaned (non --live-restore case):

ubuntu@ubuntu:~$ docker pull busybox
Using default tag: latest
latest: Pulling from library/busybox
4b0bc1c4050b: Pull complete
Digest: sha256:817a12c32a39bbe394944ba49de563e085f1d3c5266eb8e9723256bc4448680e
Status: Downloaded newer image for busybox:latest
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         128.00KiB    128.00KiB
0/257       112.00KiB    112.00KiB
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker run --name top -d busybox top
201d444f74943247c7794fae2215f763cad0dd726d03285004c54ffa04882d6b
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         128.00KiB    128.00KiB
0/257         1.17MiB     64.00KiB
0/258         1.17MiB     16.00KiB
0/259         1.17MiB     16.00KiB
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker rm -f top
top
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         144.00KiB    144.00KiB
0/257         1.17MiB      1.17MiB

PR with qgroup cleaned (--live-restore case):

# [sudo dockerd --debug --live-restore -s btrfs --storage-opt btrfs.min_space=1G]

ubuntu@ubuntu:~$ docker pull busybox
Using default tag: latest
latest: Pulling from library/busybox
4b0bc1c4050b: Pull complete
Digest: sha256:817a12c32a39bbe394944ba49de563e085f1d3c5266eb8e9723256bc4448680e
Status: Downloaded newer image for busybox:latest
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         128.00KiB    128.00KiB
0/257       112.00KiB    112.00KiB
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker run --name top -d busybox top
73c2184a4414d47b5f5dc27cd2ee28116aa761f947196618ca66bf05c4705456
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         128.00KiB    128.00KiB
0/257         1.17MiB     64.00KiB
0/258         1.17MiB     16.00KiB
0/259         1.17MiB     16.00KiB
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$

# [Ctrl-C ]
# [sudo dockerd --debug --live-restore -s btrfs]

ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED              STATUS              PORTS               NAMES
73c2184a4414        busybox             "top"               About a minute ago   Up About a minute                       top
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         152.00KiB    152.00KiB
0/257         1.17MiB     64.00KiB
0/258         1.17MiB     48.00KiB
0/259         1.17MiB     48.00KiB
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker rm -f top
top
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         168.00KiB    168.00KiB
0/257         1.17MiB     64.00KiB
Member

yongtang commented Jan 28, 2017

Here is the complete steps to verify that the issue is fixed (Including --live-restore case).

Would be good to setup integration test for btrfs in Jenkins

Note: this PR has been updated to incorporate #30497. If this one is merged then #30497 could be closed

pre-fix behavior with qgroup leftover:

ubuntu@ubuntu:~$ docker pull busybox
Using default tag: latest
latest: Pulling from library/busybox
4b0bc1c4050b: Pull complete
Digest: sha256:817a12c32a39bbe394944ba49de563e085f1d3c5266eb8e9723256bc4448680e
Status: Downloaded newer image for busybox:latest
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         128.00KiB    128.00KiB
0/257       112.00KiB    112.00KiB
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker run --name top -d busybox top
b59e6e3da6ed3e601f081c11c44d3f4dd44aefee5584d34ba913e68207c9694d
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         128.00KiB    128.00KiB
0/257         1.17MiB     64.00KiB
0/258         1.17MiB     16.00KiB
0/259         1.17MiB     16.00KiB
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker rm -f top
top
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         144.00KiB    144.00KiB
0/257         1.17MiB      1.17MiB
0/258           0.00B        0.00B
0/259           0.00B        0.00B

PR with qgroup cleaned (non --live-restore case):

ubuntu@ubuntu:~$ docker pull busybox
Using default tag: latest
latest: Pulling from library/busybox
4b0bc1c4050b: Pull complete
Digest: sha256:817a12c32a39bbe394944ba49de563e085f1d3c5266eb8e9723256bc4448680e
Status: Downloaded newer image for busybox:latest
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         128.00KiB    128.00KiB
0/257       112.00KiB    112.00KiB
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker run --name top -d busybox top
201d444f74943247c7794fae2215f763cad0dd726d03285004c54ffa04882d6b
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         128.00KiB    128.00KiB
0/257         1.17MiB     64.00KiB
0/258         1.17MiB     16.00KiB
0/259         1.17MiB     16.00KiB
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker rm -f top
top
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         144.00KiB    144.00KiB
0/257         1.17MiB      1.17MiB

PR with qgroup cleaned (--live-restore case):

# [sudo dockerd --debug --live-restore -s btrfs --storage-opt btrfs.min_space=1G]

ubuntu@ubuntu:~$ docker pull busybox
Using default tag: latest
latest: Pulling from library/busybox
4b0bc1c4050b: Pull complete
Digest: sha256:817a12c32a39bbe394944ba49de563e085f1d3c5266eb8e9723256bc4448680e
Status: Downloaded newer image for busybox:latest
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         128.00KiB    128.00KiB
0/257       112.00KiB    112.00KiB
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker run --name top -d busybox top
73c2184a4414d47b5f5dc27cd2ee28116aa761f947196618ca66bf05c4705456
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         128.00KiB    128.00KiB
0/257         1.17MiB     64.00KiB
0/258         1.17MiB     16.00KiB
0/259         1.17MiB     16.00KiB
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$

# [Ctrl-C ]
# [sudo dockerd --debug --live-restore -s btrfs]

ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED              STATUS              PORTS               NAMES
73c2184a4414        busybox             "top"               About a minute ago   Up About a minute                       top
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         152.00KiB    152.00KiB
0/257         1.17MiB     64.00KiB
0/258         1.17MiB     48.00KiB
0/259         1.17MiB     48.00KiB
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker rm -f top
top
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$
ubuntu@ubuntu:~$ sudo btrfs qgroup show /var/lib/docker/btrfs
qgroupid         rfer         excl
--------         ----         ----
0/5         168.00KiB    168.00KiB
0/257         1.17MiB     64.00KiB
@mlaventure

This comment has been minimized.

Show comment
Hide comment
@mlaventure
Contributor

mlaventure commented Feb 10, 2017

@yongtang

This comment has been minimized.

Show comment
Hide comment
@yongtang

yongtang Feb 13, 2017

Member

Thanks @mlaventure for the review. The PR has been updated with isQuotaEnabled() and lock added. Please take a look and let me know if there are any issues.

Member

yongtang commented Feb 13, 2017

Thanks @mlaventure for the review. The PR has been updated with isQuotaEnabled() and lock added. Please take a look and let me know if there are any issues.

@vdemeester vdemeester added this to the 1.14.0 milestone Feb 13, 2017

@mlaventure

This comment has been minimized.

Show comment
Hide comment
@mlaventure

mlaventure Mar 3, 2017

Contributor

ping @cpuguy83 @tonistiigi this is waiting for reviews 👼

Contributor

mlaventure commented Mar 3, 2017

ping @cpuguy83 @tonistiigi this is waiting for reviews 👼

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah
Member

thaJeztah commented Mar 15, 2017

@thaJeztah thaJeztah modified the milestones: 17.05.0, 17.04.0 Mar 15, 2017

@tonistiigi

This comment has been minimized.

Show comment
Hide comment
@tonistiigi

tonistiigi Mar 15, 2017

Member

I think implementation could be cleaner with sync.Once but LGTM

Member

tonistiigi commented Mar 15, 2017

I think implementation could be cleaner with sync.Once but LGTM

@yongtang

This comment has been minimized.

Show comment
Hide comment
@yongtang

yongtang Mar 17, 2017

Member

Thanks @tonistiigi for the review. The PR has been updated to use sync.Once. Please take a look.

Member

yongtang commented Mar 17, 2017

Thanks @tonistiigi for the review. The PR has been updated to use sync.Once. Please take a look.

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Apr 3, 2017

Member

@sargun its too late for docker 17.04, so will end up in 17.05, which will be released a month from now. Of course, it's possible to build it yourself

Member

thaJeztah commented Apr 3, 2017

@sargun its too late for docker 17.04, so will end up in 17.05, which will be released a month from now. Of course, it's possible to build it yourself

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Apr 5, 2017

Contributor

This is not re-applying qutoas on daemon restart.

$ docker run -d --name test --storage-opt size=100m busybox top
$ docker exec test sh -c 'dd if=/dev/urandom of=/data bs=1 count=200' # errors around 100M
$ # restart docker
$ # note that `btrfs qgroup show` does not find the qgroup
$ docker start test
$ # note still no qgroup
$ docker exec test sh -c 'dd if=/dev/urandom of=/data bs=1 count=200'  # write successful, no qutoa
Contributor

cpuguy83 commented Apr 5, 2017

This is not re-applying qutoas on daemon restart.

$ docker run -d --name test --storage-opt size=100m busybox top
$ docker exec test sh -c 'dd if=/dev/urandom of=/data bs=1 count=200' # errors around 100M
$ # restart docker
$ # note that `btrfs qgroup show` does not find the qgroup
$ docker start test
$ # note still no qgroup
$ docker exec test sh -c 'dd if=/dev/urandom of=/data bs=1 count=200'  # write successful, no qutoa
@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Apr 5, 2017

Contributor

It looks like we need to rescan for quota in Get()... or not disable quota on Cleanup()

Contributor

cpuguy83 commented Apr 5, 2017

It looks like we need to rescan for quota in Get()... or not disable quota on Cleanup()

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Apr 5, 2017

Contributor

Or I guess write out a file to trigger an auto-enable of the quota on restart.

Contributor

cpuguy83 commented Apr 5, 2017

Or I guess write out a file to trigger an auto-enable of the quota on restart.

@mlaventure

This comment has been minimized.

Show comment
Hide comment
@mlaventure

mlaventure Apr 25, 2017

Contributor

ping @yongtang 👼

Contributor

mlaventure commented Apr 25, 2017

ping @yongtang 👼

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah May 12, 2017

Member

ping @yongtang do you still have time to work on this?

Member

thaJeztah commented May 12, 2017

ping @yongtang do you still have time to work on this?

@thaJeztah thaJeztah modified the milestones: 17.06.0, 17.05.0 May 12, 2017

@yongtang

This comment has been minimized.

Show comment
Hide comment
@yongtang

yongtang May 13, 2017

Member

@thaJeztah Sorry for the late response. Let me spend the next several days to get this one fixed.

Member

yongtang commented May 13, 2017

@thaJeztah Sorry for the late response. Let me spend the next several days to get this one fixed.

@yongtang

This comment has been minimized.

Show comment
Hide comment
@yongtang

yongtang May 25, 2017

Member

@cpuguy83 The PR has been updated. Now the quota is saved in /lib/var/docker/btrfs/quotas and recovered in case of a daemon restart. Not sure this is the best place to persist so any suggestions would be greatly appreciated. Please take a look.

Member

yongtang commented May 25, 2017

@cpuguy83 The PR has been updated. Now the quota is saved in /lib/var/docker/btrfs/quotas and recovered in case of a daemon restart. Not sure this is the best place to persist so any suggestions would be greatly appreciated. Please take a look.

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah May 30, 2017

Member

ping @cpuguy83

also @cyphar, who may be interested in this 😃

Member

thaJeztah commented May 30, 2017

ping @cpuguy83

also @cyphar, who may be interested in this 😃

@sargun

This comment has been minimized.

Show comment
Hide comment
@sargun

sargun May 30, 2017

Contributor

Alternatively, you can just wait for this patchset to land: http://www.spinics.net/lists/linux-btrfs/msg65928.html

Contributor

sargun commented May 30, 2017

Alternatively, you can just wait for this patchset to land: http://www.spinics.net/lists/linux-btrfs/msg65928.html

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah May 31, 2017

Member

@sargun nice! Unfortunately we don't control the host, so cannot guarantee that patch is present on each distro / version, so not sure we can rely on that

Member

thaJeztah commented May 31, 2017

@sargun nice! Unfortunately we don't control the host, so cannot guarantee that patch is present on each distro / version, so not sure we can rely on that

Show outdated Hide outdated daemon/graphdriver/btrfs/btrfs.go Outdated
Show outdated Hide outdated daemon/graphdriver/btrfs/btrfs.go Outdated
Show outdated Hide outdated daemon/graphdriver/btrfs/btrfs.go Outdated
Show outdated Hide outdated daemon/graphdriver/btrfs/btrfs.go Outdated

yongtang added some commits Dec 14, 2016

Remove btrfs quota groups after containers destroyed
This fix tries to address the issue raised in 29325 where
btrfs quota groups are not clean up even after containers
have been destroyed.

The reason for the issue is that btrfs quota groups have
to be explicitly destroyed. This fix fixes this issue.

This fix is tested manually in Ubuntu 16.04,
with steps specified in 29325.

This fix fixes 29325.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Persist the quota size for btrfs so that daemon restart keeps quota
This commit is an extension of fix for 29325 based on the review comment.
In this commit, the quota size for btrfs is kept in `/var/lib/docker/btrfs/quotas`
so that a daemon restart keeps quota.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
@yongtang

This comment has been minimized.

Show comment
Hide comment
@yongtang

yongtang Jun 2, 2017

Member

Thanks @cpuguy83 for the review. The PR has been updated. Please take a look.

Member

yongtang commented Jun 2, 2017

Thanks @cpuguy83 for the review. The PR has been updated. Please take a look.

@cpuguy83

LGTM

@yongtang

This comment has been minimized.

Show comment
Hide comment
@yongtang

yongtang Jun 6, 2017

Member

All tests passed now, merging...

Member

yongtang commented Jun 6, 2017

All tests passed now, merging...

@yongtang yongtang merged commit 54dacaf into moby:master Jun 6, 2017

6 checks passed

dco-signed All commits are signed
experimental Jenkins build Docker-PRs-experimental 34679 has succeeded
Details
janky Jenkins build Docker-PRs 43275 has succeeded
Details
powerpc Jenkins build Docker-PRs-powerpc 3664 has succeeded
Details
windowsRS1 Jenkins build Docker-PRs-WoW-RS1 14562 has succeeded
Details
z Jenkins build Docker-PRs-s390x 3498 has succeeded
Details

@yongtang yongtang deleted the yongtang:29325-cleanup-btrfs-quota-groups branch Jun 6, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment