Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Kubelet] Use the custom mounter script for Nfs and Glusterfs only #35821

Merged
merged 4 commits into from Nov 3, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
7 changes: 7 additions & 0 deletions cluster/gce/gci/configure-helper.sh
Expand Up @@ -473,6 +473,7 @@ function start-kubelet {
flags+=" --cluster-dns=${DNS_SERVER_IP}"
flags+=" --cluster-domain=${DNS_DOMAIN}"
flags+=" --config=/etc/kubernetes/manifests"
flags+=" --experimental-mounter-path=${KUBE_HOME}/bin/mounter"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where's this coming from?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


if [[ -n "${KUBELET_PORT:-}" ]]; then
flags+=" --port=${KUBELET_PORT}"
Expand Down Expand Up @@ -1171,6 +1172,10 @@ function override-kubectl {
echo "export PATH=${KUBE_HOME}/bin:\$PATH" > /etc/profile.d/kube_env.sh
}

function pre-warm-mounter {
${KUBE_HOME}/bin/mounter &> /dev/null
}

########### Main Function ###########
echo "Start to configure instance for kubernetes"

Expand Down Expand Up @@ -1205,6 +1210,8 @@ else
fi

override-kubectl
# Run the containerized mounter once to pre-cache the container image.
pre-warm-mounter
assemble-docker-flags
load-docker-images
start-kubelet
Expand Down
17 changes: 14 additions & 3 deletions cluster/gce/gci/mounter/mounter
Expand Up @@ -23,14 +23,25 @@ MOUNTER_USER=root
RKT_BINARY=/home/kubernetes/bin/rkt

function gc {
${RKT_BINARY} gc --grace-period=0s &> /dev/null
# Attempt to garbage collect rkt pods with 5 retries.
# Rkt pods end up creating new copies of mounts on the host. Hence it is ideal to clean them up right away.
attempt=0
until [ $attempt -ge 5 ]; do
${RKT_BINARY} gc --grace-period=0s && break
attempt=$[$attempt+1]
sleep 1
done
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just want to double check 5 retry won't cause any issue or interfere other ongoing mounter command?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should only garbage collect completed pods. I'd expect rkt to synchronize across parallel GC operations. This is an issue to consider even without this retry.

}

# Garbage collect old rkt containers on exit
trap gc EXIT

echo "Running mount using a rkt fly container"

${RKT_BINARY} run --stage1-name="coreos.com/rkt/stage1-fly:1.18.0" \
--insecure-options=image \
--volume=rootfs,kind=host,source=/,readOnly=false,recursive=true \
--mount volume=rootfs,target=/media/root \
--volume=kubelet,kind=host,source=/var/lib/kubelet,readOnly=false,recursive=true \
--mount volume=kubelet,target=/var/lib/kubelet \
docker://${MOUNTER_DOCKER_IMAGE}:${MOUNTER_DOCKER_VERSION} --user=${MOUNTER_USER} --exec /bin/mount -- "$@"

echo "Successfully ran mount using a rkt fly container"
2 changes: 1 addition & 1 deletion cmd/kube-proxy/app/conntrack.go
Expand Up @@ -67,7 +67,7 @@ func (realConntracker) SetTCPEstablishedTimeout(seconds int) error {
func isSysFSWritable() (bool, error) {
const permWritable = "rw"
const sysfsDevice = "sysfs"
m := mount.New()
m := mount.New("" /* default mount path */)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would prefer NewDefault()

mountPoints, err := m.List()
if err != nil {
glog.Errorf("failed to list mount points: %v", err)
Expand Down
1 change: 0 additions & 1 deletion cmd/kubelet/app/options/options.go
Expand Up @@ -200,7 +200,6 @@ func (s *KubeletServer) AddFlags(fs *pflag.FlagSet) {
fs.BoolVar(&s.ExitOnLockContention, "exit-on-lock-contention", s.ExitOnLockContention, "Whether kubelet should exit upon lock-file contention.")
fs.StringVar(&s.RktPath, "rkt-path", s.RktPath, "Path of rkt binary. Leave empty to use the first rkt in $PATH. Only used if --container-runtime='rkt'.")
fs.StringVar(&s.ExperimentalMounterPath, "experimental-mounter-path", s.ExperimentalMounterPath, "[Experimental] Path of mounter binary. Leave empty to use the default mount.")
fs.StringVar(&s.ExperimentalMounterRootfsPath, "experimental-mounter-rootfs-path", s.ExperimentalMounterRootfsPath, "[Experimental] Absolute path to the root filesystem for the mounter binary.")
fs.StringVar(&s.RktAPIEndpoint, "rkt-api-endpoint", s.RktAPIEndpoint, "The endpoint of the rkt API service to communicate with. Only used if --container-runtime='rkt'.")
fs.StringVar(&s.RktStage1Image, "rkt-stage1-image", s.RktStage1Image, "image to use as stage1. Local paths and http/https URLs are supported. If empty, the 'stage1.aci' in the same directory as '--rkt-path' will be used.")
fs.MarkDeprecated("rkt-stage1-image", "Will be removed in a future version. The default stage1 image will be specified by the rkt configurations, see https://github.com/coreos/rkt/blob/master/Documentation/configuration.md for more details.")
Expand Down
2 changes: 1 addition & 1 deletion cmd/kubelet/app/server.go
Expand Up @@ -118,7 +118,7 @@ func UnsecuredKubeletDeps(s *options.KubeletServer) (*kubelet.KubeletDeps, error
return nil, err
}

mounter := mount.NewCustomMounter(s.ExperimentalMounterPath, s.ExperimentalMounterRootfsPath)
mounter := mount.New(s.ExperimentalMounterPath)
var writer kubeio.Writer = &kubeio.StdWriter{}
if s.Containerized {
glog.V(2).Info("Running kubelet in containerized mode (experimental)")
Expand Down
29 changes: 0 additions & 29 deletions docs/user-guide/kubectl/kubectl_cp.md
@@ -1,32 +1,3 @@
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->

<!-- BEGIN STRIP_FOR_RELEASE -->

<img src="http://kubernetes.io/kubernetes/img/warning.png" alt="WARNING"
width="25" height="25">
<img src="http://kubernetes.io/kubernetes/img/warning.png" alt="WARNING"
width="25" height="25">
<img src="http://kubernetes.io/kubernetes/img/warning.png" alt="WARNING"
width="25" height="25">
<img src="http://kubernetes.io/kubernetes/img/warning.png" alt="WARNING"
width="25" height="25">
<img src="http://kubernetes.io/kubernetes/img/warning.png" alt="WARNING"
width="25" height="25">

<h2>PLEASE NOTE: This document applies to the HEAD of the source tree</h2>

If you are using a released version of Kubernetes, you should
refer to the docs that go with that version.

Documentation for other releases can be found at
[releases.k8s.io](http://releases.k8s.io).
</strong>
--

<!-- END STRIP_FOR_RELEASE -->

<!-- END MUNGE: UNVERSIONED_WARNING -->

This file is autogenerated, but we've stopped checking such files into the
repository to reduce the need for rebases. Please run hack/generate-docs.sh to
populate this file.
Expand Down
1 change: 0 additions & 1 deletion hack/verify-flags/known-flags.txt
Expand Up @@ -188,7 +188,6 @@ experimental-allowed-unsafe-sysctls
experimental-bootstrap-kubeconfig
experimental-keystone-url
experimental-mounter-path
experimental-mounter-rootfs-path
experimental-nvidia-gpus
experimental-prefix
experimental-runtime-integration-type
Expand Down