Move root filesystem from rootfs to tmpfs #5133

afbjorklund · 2019-08-19T17:47:03Z

This moves the root filesystem of the ISO, from rootfs to tmpfs.

This allows us to stop having to use --no-pivot flag with runc.

The Docker configuration is flexible enough to work with both...

Thanks to @kfox1111 for suggesting a much easier way to do it.

Issue #3512

k8s-ci-robot · 2019-08-19T17:47:15Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: afbjorklund

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [afbjorklund]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

kfox1111 · 2019-08-19T20:34:00Z

Looks good to me. :)

pkg/provision/buildroot.go

tstromberg · 2019-08-19T22:37:12Z

deploy/iso/minikube-iso/board/coreos/minikube/rootfs-overlay/init

@@ -0,0 +1,11 @@
+#!/bin/sh
+mkdir /sysroot
+mount -t tmpfs -o size=90% tmpfs /sysroot


Add a comment explaining how you arrived at 90% being the correct choice here.

That’s easy, I stole it from tinycore (and boot2docker). 😀
I think it needs some space left, so using 100% won’t work. But we could probably get away with 95% since our VM is twice as big ? Can experiment with it

That's fine. Just mention that you have copied the number from elsewhere then. I was afraid it might have some magical principle to it.

I'll stick with 90%, until I know better. Will leave a reference to the original

tstromberg

This looks great! Minor nits, but please feel free to merge once you feel it is ready.

tstromberg · 2019-08-19T22:38:15Z

deploy/iso/minikube-iso/board/coreos/minikube/rootfs-overlay/init

@@ -0,0 +1,11 @@
+#!/bin/sh


For future archeologists, do you mind adding a quick comment describing why this file exists, and what it's intent is?

future archeologists hahaha ! love it ! hello future archeologists from 2019 !

Explaining init(1) ?

Or maybe what is the difference between this init and the normal one would be more appropriate. It doesn’t really help that the final /sbin/init is actually not init but just Lennart (systemd) posing

I for one didn't realize this took the place of the system init, though it makes sense given the location.

It'd be nice to have an explanation of why data is being copied from one directory to another before exec'ing out to another init. =)

Settled for adding a reference to switch_root(8)

DESCRIPTION
switch_root moves already mounted /proc, /dev, /sys and /run to newroot and makes newroot the new root filesystem and starts init process.

WARNING: switch_root removes recursively all files and directories on the current root filesystem.

leseb · 2019-08-20T14:59:00Z

This should also close #4072. Thanks!

pkg/provision/buildroot.go

TravisBuddy · 2019-08-22T18:31:31Z

Travis tests have failed

Hey @afbjorklund,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

1st Build

View build log

make test

which go-bindata || GO111MODULE=off GOBIN=/home/travis/gopath/bin go get github.com/jteeuwen/go-bindata/...
PATH="/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.12.9.linux.amd64/bin:/home/travis/bin:/home/travis/bin:/home/travis/.local/bin:/usr/local/lib/jvm/openjdk11/bin:/opt/pyenv/shims:/home/travis/.phpenv/shims:/home/travis/perl5/perlbrew/bin:/home/travis/.nvm/versions/node/v8.12.0/bin:/home/travis/.rvm/gems/ruby-2.5.3/bin:/home/travis/.rvm/gems/ruby-2.5.3@global/bin:/home/travis/.rvm/rubies/ruby-2.5.3/bin:/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.11.1.linux.amd64/bin:/usr/local/maven-3.6.0/bin:/usr/local/cmake-3.12.4/bin:/usr/local/clang-7.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/home/travis/.rvm/bin:/home/travis/.phpenv/bin:/opt/pyenv/bin:/home/travis/.yarn/bin:/home/travis/gopath/bin" go-bindata -nomemcopy -o pkg/minikube/assets/assets.go -pkg assets deploy/addons/...
gofmt -s -w pkg/minikube/assets/assets.go
which go-bindata || GO111MODULE=off GOBIN=/home/travis/gopath/bin go get github.com/jteeuwen/go-bindata/...
/home/travis/gopath/bin/go-bindata
PATH="/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.12.9.linux.amd64/bin:/home/travis/bin:/home/travis/bin:/home/travis/.local/bin:/usr/local/lib/jvm/openjdk11/bin:/opt/pyenv/shims:/home/travis/.phpenv/shims:/home/travis/perl5/perlbrew/bin:/home/travis/.nvm/versions/node/v8.12.0/bin:/home/travis/.rvm/gems/ruby-2.5.3/bin:/home/travis/.rvm/gems/ruby-2.5.3@global/bin:/home/travis/.rvm/rubies/ruby-2.5.3/bin:/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.11.1.linux.amd64/bin:/usr/local/maven-3.6.0/bin:/usr/local/cmake-3.12.4/bin:/usr/local/clang-7.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/home/travis/.rvm/bin:/home/travis/.phpenv/bin:/opt/pyenv/bin:/home/travis/.yarn/bin:/home/travis/gopath/bin" go-bindata -nomemcopy -o pkg/minikube/translate/translations.go -pkg translate translations/...
gofmt -s -w pkg/minikube/translate/translations.go
./test.sh
= go mod ================================================================
ok
= make lint =============================================================
golangci/golangci-lint info checking GitHub for tag 'v1.17.1'
golangci/golangci-lint info found version: 1.17.1 for v1.17.1/linux/amd64
golangci/golangci-lint info installed out/linters/golangci-lint
test/integration/a_download_only_test.go:68:35: GetKubeadmCachedBinaries not declared by package constants (typecheck)
				for _, bin := range constants.GetKubeadmCachedBinaries() {
				                              ^
Makefile:323: recipe for target 'lint-ci' failed
make[1]: *** [lint-ci] Error 1
= boilerplate ===========================================================
ok
= schema_check ==========================================================
ok
= go test ===============================================================
ok  	k8s.io/minikube/cmd/minikube/cmd	0.074s	coverage: 15.7% of statements
ok  	k8s.io/minikube/cmd/minikube/cmd/config	0.051s	coverage: 18.9% of statements
ok  	k8s.io/minikube/pkg/drivers	0.009s	coverage: 19.6% of statements
ok  	k8s.io/minikube/pkg/drivers/kvm	0.066s	coverage: 2.3% of statements
ok  	k8s.io/minikube/pkg/minikube/assets	0.029s	coverage: 61.8% of statements
ok  	k8s.io/minikube/pkg/minikube/bootstrapper	3.087s	coverage: 72.9% of statements
ok  	k8s.io/minikube/pkg/minikube/bootstrapper/kubeadm	0.079s	coverage: 29.5% of statements
ok  	k8s.io/minikube/pkg/minikube/cluster	0.814s	coverage: 54.2% of statements
ok  	k8s.io/minikube/pkg/minikube/config	0.020s	coverage: 76.0% of statements
ok  	k8s.io/minikube/pkg/minikube/cruntime	0.009s	coverage: 62.4% of statements
ok  	k8s.io/minikube/pkg/minikube/extract	0.018s	coverage: 56.7% of statements
ok  	k8s.io/minikube/pkg/minikube/kubeconfig	0.060s	coverage: 75.6% of statements
ok  	k8s.io/minikube/pkg/minikube/logs	0.024s	coverage: 1.5% of statements
ok  	k8s.io/minikube/pkg/minikube/machine	0.031s	coverage: 12.0% of statements
ok  	k8s.io/minikube/pkg/minikube/notify	0.024s	coverage: 77.8% of statements
ok  	k8s.io/minikube/pkg/minikube/out	0.009s	coverage: 70.3% of statements
ok  	k8s.io/minikube/pkg/minikube/problem	0.020s	coverage: 42.9% of statements
ok  	k8s.io/minikube/pkg/minikube/proxy	0.010s	coverage: 67.3% of statements
ok  	k8s.io/minikube/pkg/minikube/registry	0.012s	coverage: 81.8% of statements
ok  	k8s.io/minikube/pkg/minikube/service	0.035s	coverage: 35.9% of statements
ok  	k8s.io/minikube/pkg/minikube/sshutil	0.225s	coverage: 75.0% of statements
ok  	k8s.io/minikube/pkg/minikube/translate	0.010s	coverage: 8.4% of statements
ok  	k8s.io/minikube/pkg/minikube/tunnel	1.842s	coverage: 64.5% of statements
ok  	k8s.io/minikube/pkg/util	0.958s	coverage: 61.6% of statements
ok  	k8s.io/minikube/pkg/util/retry	0.003s	coverage: 0.0% of statements
ok
Makefile:229: recipe for target 'test' failed
make: *** [test] Error 4

TravisBuddy Request Identifier: 0cc8f740-c50b-11e9-8712-75d78f9b457f

afbjorklund · 2019-08-22T18:35:37Z

@TravisBuddy : Build failed due to #5176

kfox1111 · 2019-08-23T16:34:50Z

Looks like the travis fix may be merged.
/retest

medyagh · 2019-08-23T17:12:54Z

@TravisBuddy : Build failed due to #5176

@TravisBuddy sorry it was my fault ! should had pulled upstream

pkg/provision/buildroot.go

The docker configuration is determined at runtime, so make it work with both old rootfs and new tmpfs.

It is only intended for compatibility with the old rootfs ISO, and not needed with the new tmpfs ISO.

tstromberg · 2019-08-24T17:58:31Z

PR looks good, but I'm concerned about the apparently reliable Gvisor failure:

FAIL: TestContainerd/GvisorRestart (613.33s)
...
08:44:04 | ! [kubelet-check] Initial timeout of 40s passed.

We'll need to confirm if this issue occurs at master without this PR.

UPDATE: opened #5191

afbjorklund · 2019-08-25T07:17:19Z

The test results look fairly non-obvious to me.

They seem to fail, no matter if the code is broken (7b5f648) or if it is working (edea8f9) ?
So either I will override and go on pushing broken code, or be afraid to submit working code.

That being said, it works here (don't use gvisor)

leseb · 2019-08-26T14:29:58Z

What's the target release for this? Thanks

afbjorklund · 2019-08-26T14:35:04Z

v1.4.0

afbjorklund requested a review from tstromberg August 19, 2019 17:47

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Aug 19, 2019

k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Aug 19, 2019

k8s-ci-robot requested review from RA489 and sharifelgamal August 19, 2019 17:47

tstromberg reviewed Aug 19, 2019

View reviewed changes

pkg/provision/buildroot.go Outdated Show resolved Hide resolved

tstromberg reviewed Aug 19, 2019

View reviewed changes

tstromberg suggested changes Aug 19, 2019

View reviewed changes

leseb approved these changes Aug 20, 2019

View reviewed changes

cyphar reviewed Aug 22, 2019

View reviewed changes

pkg/provision/buildroot.go Outdated Show resolved Hide resolved

afbjorklund force-pushed the tmpfs branch from b474c31 to 3e07256 Compare August 22, 2019 18:21

afbjorklund force-pushed the tmpfs branch from 3e07256 to c3eb17b Compare August 22, 2019 19:11

cyphar reviewed Aug 24, 2019

View reviewed changes

pkg/provision/buildroot.go Show resolved Hide resolved

afbjorklund added 4 commits August 24, 2019 16:12

Only use DOCKER_RAMDISK on fstype rootfs

edea8f9

The docker configuration is determined at runtime, so make it work with both old rootfs and new tmpfs.

Move root filesystem from rootfs to tmpfs

a4c5992

Stop using no_pivot when not using rootfs

ae8894c

Add warning that using --no-pivot is not secure

eec1a06

It is only intended for compatibility with the old rootfs ISO, and not needed with the new tmpfs ISO.

afbjorklund force-pushed the tmpfs branch from c3eb17b to eec1a06 Compare August 24, 2019 14:20

afbjorklund requested a review from tstromberg August 24, 2019 14:21

tstromberg mentioned this pull request Aug 24, 2019

TestContainerd/GvisorRestart: [kubelet-check] timeout / failed to create containerd container: error unpacking image: content digest not found #5191

Closed

tstromberg merged commit b59a8b8 into kubernetes:master Aug 26, 2019

kfox1111 mentioned this pull request Aug 28, 2019

Enabling bidirectional mounts on pod kills docker #4072

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move root filesystem from rootfs to tmpfs #5133

Move root filesystem from rootfs to tmpfs #5133

afbjorklund commented Aug 19, 2019

k8s-ci-robot commented Aug 19, 2019

kfox1111 commented Aug 19, 2019

tstromberg Aug 19, 2019

afbjorklund Aug 20, 2019

tstromberg Aug 21, 2019

afbjorklund Aug 22, 2019

tstromberg left a comment

tstromberg Aug 19, 2019

medyagh Aug 20, 2019

afbjorklund Aug 20, 2019

tstromberg Aug 21, 2019

afbjorklund Aug 22, 2019

leseb commented Aug 20, 2019

TravisBuddy commented Aug 22, 2019

afbjorklund commented Aug 22, 2019

kfox1111 commented Aug 23, 2019

medyagh commented Aug 23, 2019

tstromberg commented Aug 24, 2019 •

edited

afbjorklund commented Aug 25, 2019

leseb commented Aug 26, 2019

afbjorklund commented Aug 26, 2019

Move root filesystem from rootfs to tmpfs #5133

Move root filesystem from rootfs to tmpfs #5133

Conversation

afbjorklund commented Aug 19, 2019

k8s-ci-robot commented Aug 19, 2019

kfox1111 commented Aug 19, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tstromberg left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

leseb commented Aug 20, 2019

TravisBuddy commented Aug 22, 2019

Travis tests have failed

1st Build

TravisBuddy Request Identifier: 0cc8f740-c50b-11e9-8712-75d78f9b457f

afbjorklund commented Aug 22, 2019

kfox1111 commented Aug 23, 2019

medyagh commented Aug 23, 2019

tstromberg commented Aug 24, 2019 • edited

afbjorklund commented Aug 25, 2019

leseb commented Aug 26, 2019

afbjorklund commented Aug 26, 2019

tstromberg commented Aug 24, 2019 •

edited