Customized mounts in chroot to GA

[rkouj]

Description

With the default image changing to Container-Optimized OS (aka gci image) starting Google Container Engine version 1.4, some volume plugins including NFS and Gluster could not work because the image does not install the necessary utilities to mount the file system.

To solve this issue, we are working on an approach to use mount in a chroot environment. By creating a customized image with all the utility installed and mounting inside of chroot, we can overcome the issue and be able to share mount from chroot to host. The following lists the basic steps for this approach

1. Create a docker image has storage tools (nfs and glusterfs) pre-installed.
2. Export the image to a tar file and upload to the google cloud storage
3. When starting a k8s cluster, download the tar file and untar to a directory as the rootfs of chroot command
4. Setup the chroot environment by bind mounting /var/lib/kubelet, /proc, /dev ...
5. In the kubelet mount code, change to use chroot to mount if it is gci image

Progress Tracker

TBD

• Alpha
  • Write and maintain draft quality doc
    • During development keep a doc up-to-date about the desired experience of the feature and how someone can try the feature in its current state. Think of it as the README of your new feature and a skeleton for the docs to be written before the Kubernetes release. Paste link to Google Doc: DOC-LINK
  • Design Approval
    • Design Proposal. This goes under design-proposals. Doing a proposal as a PR allows line-by-line commenting from community, and creates the basis for later design documentation. Paste link to merged design proposal here: PROPOSAL-NUMBER
    • Decide which repo this feature's code will be checked into. Not everything needs to land in the core kubernetes repo. REPO-NAME
    • Initial API review (if API). Maybe same PR as design doc. PR-NUMBER
      • Any code that changes an API (/pkg/apis/...)
      • cc @kubernetes/api
    • Identify shepherd (your SIG lead and/or kubernetes-pm@googlegroups.com will be able to help you). My Shepherd is: replace.me@replaceme.com (and/or GH Handle)
      • A shepherd is an individual who will help acquaint you with the process of getting your feature into the repo, identify reviewers and provide feedback on the feature. They are not (necessarily) the code reviewer of the feature, or tech lead for the area.
      • The shepherd is not responsible for showing up to Kubernetes-PM meetings and/or communicating if the feature is on-track to make the release goals. That is still your responsibility.
    • Identify secondary/backup contact point. My Secondary Contact Point is: replace.me@replaceme.com (and/or GH Handle)
  • Write (code + tests + docs) then get them merged. ALL-PR-NUMBERS
    • Code needs to be disabled by default. Verified by code OWNERS
    • Minimal testing
    • Minimal docs
      • cc @kubernetes/docs on docs PR
      • cc @kubernetes/feature-reviewers on this issue to get approval before checking this off
      • New apis: Glossary Section Item in the docs repo: kubernetes/kubernetes.github.io
    • Update release notes
• Beta
  • Testing is sufficient for beta
  • User docs with tutorials
    - Updated walkthrough / tutorial in the docs repo: kubernetes/kubernetes.github.io
    - cc @kubernetes/docs on docs PR
    - cc @kubernetes/feature-reviewers on this issue to get approval before checking this off
  • Thorough API review
    • cc @kubernetes/api
• Stable
  • docs/proposals/foo.md moved to docs/design/foo.md
    - cc @kubernetes/feature-reviewers on this issue to get approval before checking this off
  • Soak, load testing
  • detailed user docs and examples
    • cc @kubernetes/docs
    • cc @kubernetes/feature-reviewers on this issue to get approval before checking this off

FEATURE_STATUS is used for feature tracking and to be updated by @kubernetes/feature-reviewers.
FEATURE_STATUS: IN_DEVELOPMENT

More advice:

Design

• Once you get LGTM from a @kubernetes/feature-reviewers member, you can check this checkbox, and the reviewer will apply the "design-complete" label.

Coding

• Use as many PRs as you need. Write tests in the same or different PRs, as is convenient for you.
• As each PR is merged, add a comment to this issue referencing the PRs. Code goes in the http://github.com/kubernetes/kubernetes repository,
  and sometimes http://github.com/kubernetes/contrib, or other repos.
• When you are done with the code, apply the "code-complete" label.
• When the feature has user docs, please add a comment mentioning @kubernetes/feature-reviewers and they will
  check that the code matches the proposed feature and design, and that everything is done, and that there is adequate
  testing. They won't do detailed code review: that already happened when your PRs were reviewed.
  When that is done, you can check this box and the reviewer will apply the "code-complete" label.

Docs

• Write user docs and get them merged in.
• User docs go into http://github.com/kubernetes/kubernetes.github.io.
• When the feature has user docs, please add a comment mentioning @kubernetes/docs.
• When you get LGTM, you can check this checkbox, and the reviewer will apply the "docs-complete" label.

[jsafrane]

Is there a design proposal for this feature?

[idvoretskyi]

@jingxu97 any update on this feature? Docs and release notes are required (please, provide them to the features spreadsheet.

[idvoretskyi]

@jingxu97 can you provide us with the link of the submitted PR with documentation? /cc @devin-donnelly

[jingxu97]

Hi Ihor, I put the submitted PR link in the doc already. Do you need a separate documentation for this feature too? Thanks! Jing

…

On Wed, Mar 15, 2017 at 5:21 PM, Ihor Dvoretskyi ***@***.***> wrote: @jingxu97 <https://github.com/jingxu97> can you provide us with the link of the submitted PR with documentation? /cc @devin-donnelly <https://github.com/devin-donnelly> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#176 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ASSNxShpU1oY3bksHU9VIy2ZiVGJhlGfks5rmIBvgaJpZM4Lrqak> .

-- - Jing

[idvoretskyi]

Do you need a separate documentation for this feature too?

@jingxu97 yes, please.

[jingxu97]

I added the doc PR in the excel file. Please let me know if this is ok. Thanks! Best, Jing

…

On Wed, Mar 15, 2017 at 5:53 PM, Ihor Dvoretskyi ***@***.***> wrote: Do you need a separate documentation for this feature too? @jingxu97 <https://github.com/jingxu97> yes, please. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#176 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ASSNxZHCvvUX1PiZgKHu29Q6JJBHMqy0ks5rmIgIgaJpZM4Lrqak> .

-- - Jing

[idvoretskyi]

@jingxu97 it seems to be a design doc proposal. Do you have any documentation to be submitted to https://github.com/kubernetes/kubernetes.github.io?

[jingxu97]

Hi Ihor The feature is very internal. Basically with this feature, user can continue to use NFS and Gluster as before on GKE. Without it, it will fail to mount NFS or GlusterFS. We already have some example document on how to use NFS and GlusterFS. Please let me know if you think any other document we need to add. Thanks! Jing

…

On Thu, Mar 16, 2017 at 1:01 PM, Ihor Dvoretskyi ***@***.***> wrote: @jingxu97 <https://github.com/jingxu97> it seems to be a design doc proposal. Do you have any documentation to be submitted to https://github.com/kubernetes/kubernetes.github.io? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#176 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ASSNxasiU79Q3NaN8jk2AD6v24zo5QMTks5rmZUAgaJpZM4Lrqak> .

-- - Jing

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle rotten
/remove-lifecycle stale

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close