Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: readonly volume should be bind mounted readonly on the host #1062

Merged
merged 3 commits into from Nov 6, 2020

Conversation

bergwolf
Copy link
Member

So that even if the guest is compromised, we still ensure that readonly volumes cannot be modified.

@bergwolf bergwolf added needs-backport Changes need to be applied to an older branch / repository no-forward-port-needed Changed do not need to be applied to a newer branch / repository labels Oct 30, 2020
@bergwolf
Copy link
Member Author

/test

@bergwolf
Copy link
Member Author

/test

Copy link
Contributor

@jodh-intel jodh-intel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bergwolf.

Any way you could create a new test explicitly to check for this behaviour?

@amshinde amshinde added the do-not-merge PR has problems or depends on another label Oct 30, 2020
@amshinde
Copy link
Member

@awprice PTAL

bergwolf added a commit to bergwolf/kata-tests that referenced this pull request Nov 2, 2020
To make sure readonly volumes are mounted readonly both inside the guest
and on the host.

Depends-on: github.com/kata-containers/kata-containers#1062
Fixes: kata-containers#3022
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
bergwolf added a commit to bergwolf/kata-tests that referenced this pull request Nov 2, 2020
To make sure readonly volumes are mounted readonly both inside the guest
and on the host.

Depends-on: github.com/kata-containers/kata-containers#1062
Fixes: kata-containers#3022
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
@bergwolf
Copy link
Member Author

bergwolf commented Nov 2, 2020

@jodh-intel sure, pls see kata-containers/tests#3023

bergwolf added a commit to bergwolf/kata-tests that referenced this pull request Nov 2, 2020
To make sure readonly volumes are mounted readonly both inside the guest
and on the host.

Depends-on: github.com/kata-containers/kata-containers#1062
Fixes: kata-containers#3022
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
@awprice
Copy link
Contributor

awprice commented Nov 2, 2020

Thanks for raising this so quickly @bergwolf! It looks good to me, but I haven't had a chance to test this as I'm still on 1.11.3

@bergwolf
Copy link
Member Author

bergwolf commented Nov 2, 2020

@awprice ah, I see. I just backported the PR here kata-containers/runtime#3042
Please try the 1.x version. The two repo codes are almost identical and it is pretty easy to port things between them ;)

@awprice
Copy link
Contributor

awprice commented Nov 2, 2020

@bergwolf Thanks! I'll test out the 1.x code tomorrow

if err := bindMount(c.ctx, m.Source, mountDest, m.ReadOnly, "private"); err != nil {
return "", false, err
}
// Save HostPath mount value into the mount list of the container.
c.mounts[idx].HostPath = mountDest
// bindmount remount event is not propagated to mount subtrees, so we have to remount the shared dir mountpoint directly.
if m.ReadOnly {
mountDest = filepath.Join(hostSharedDir, filename)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bergwolf I suppose this is a newly added commit. Will this allow the guest to perform an unmount in case of a container escape, the vulnerability CVE-2020-2024 that we addressed before?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nop, this is host mount. guest cannot change it and only sees the readonly directory.

@awprice
Copy link
Contributor

awprice commented Nov 3, 2020

I've tested the 1.x code in my environment and it fixes the issue.

bergwolf added a commit to bergwolf/kata-tests that referenced this pull request Nov 3, 2020
To make sure readonly volumes are mounted readonly both inside the guest
and on the host.

Depends-on: github.com/kata-containers/kata-containers#1062
Fixes: kata-containers#3022
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
Copy link

@devimc devimc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

othanks @bergwolf - one nit

src/runtime/virtcontainers/mount.go Outdated Show resolved Hide resolved
@bergwolf
Copy link
Member Author

bergwolf commented Nov 3, 2020

/test

@bergwolf
Copy link
Member Author

bergwolf commented Nov 4, 2020

/test

So that we get protected at the VM boundary not just the guest kernel.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
bindmount remount events are not propagated through mount subtrees,
so we have to remount the shared dir mountpoint directly.

E.g.,
```
mkdir -p source dest foo source/foo

mount -o bind --make-shared source dest

mount -o bind foo source/foo
echo bind mount rw
mount | grep foo
echo remount ro
mount -o remount,bind,ro source/foo
mount | grep foo
```
would result in:
```
bind mount rw
/dev/xvda1 on /home/ubuntu/source/foo type ext4 (rw,relatime,discard,data=ordered)
/dev/xvda1 on /home/ubuntu/dest/foo type ext4 (rw,relatime,discard,data=ordered)
remount ro
/dev/xvda1 on /home/ubuntu/source/foo type ext4 (ro,relatime,discard,data=ordered)
/dev/xvda1 on /home/ubuntu/dest/foo type ext4 (rw,relatime,discard,data=ordered)
```

The reason is that bind mount creats new mount structs and attaches them to different mount subtrees.
However, MS_REMOUNT only looks for existing mount structs to modify and does not try to propagate the
change to mount structs in other subtrees.

Fixes: kata-containers#1061
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
@bergwolf
Copy link
Member Author

bergwolf commented Nov 4, 2020

/test

@bergwolf
Copy link
Member Author

bergwolf commented Nov 4, 2020

There is a regression introduced by #1037 and it is causing crio + k8s configmap to fail a lot on multiple platforms in different PRs. I've added an extra commit to revert the offending commit:

commit 87848e874e10b692c0efdf0709158fb1f9df72a3 (origin/pr/1037)
Author: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
Date:   Tue Oct 27 10:50:09 2020 -0600

    versions: Update crio version

    This PR updates the crio version from 1.18.3 to 1.18.4 in order to include
    the fix https://github.com/cri-o/cri-o/pull/4284.

    Fixes #1036

    Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>

ref: #1080

@bergwolf
Copy link
Member Author

bergwolf commented Nov 4, 2020

/test-vfio

@bergwolf
Copy link
Member Author

bergwolf commented Nov 4, 2020

vfio test is failing at

19:44:58 kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl kubectl exec [POD] -- [COMMAND] instead.
19:44:58 Unable to use a TTY - input is not a terminal or the right kind of file

Should we just run kubectl exec without the -i option? /cc @devimc

The same is failing on other unrelated PRs as well, e.g., http://jenkins.katacontainers.io/job/kata-containers-2.0-vfio-PR/184/console

@devimc
Copy link

devimc commented Nov 4, 2020

@bergwolf yeah I'm on that kata-containers/tests#3026

@bergwolf
Copy link
Member Author

bergwolf commented Nov 5, 2020

/test

This reverts commit 87848e8 as it is
breaking the k8s configMap test.

Fixex: kata-containers#1080
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
@bergwolf
Copy link
Member Author

bergwolf commented Nov 5, 2020

/test

1 similar comment
@amshinde
Copy link
Member

amshinde commented Nov 5, 2020

/test

@bergwolf
Copy link
Member Author

bergwolf commented Nov 6, 2020

/test-vfio

@bergwolf bergwolf removed the do-not-merge PR has problems or depends on another label Nov 6, 2020
@liubin liubin merged commit a68e200 into kata-containers:2.0-dev Nov 6, 2020
11 checks passed
@bergwolf bergwolf deleted the ro-volume branch November 6, 2020 02:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-backport Changes need to be applied to an older branch / repository no-forward-port-needed Changed do not need to be applied to a newer branch / repository
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants