New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

performance issue during k8s starting a pod container #68672

Open
Cherishty opened this Issue Sep 14, 2018 · 4 comments

Comments

Projects
None yet
4 participants
@Cherishty

Cherishty commented Sep 14, 2018

Is this a BUG REPORT or FEATURE REQUEST?:
BUG

/kind bug

What happened:
When I start a simple nginx pod by kubectl create, k8s always spend 10 second to "Created" a container after "SuccessfulMountVolume", how can it be SO slow ? What happen during this period?
What you expected to happen:
Start a pod in 5 second after mounted successfully
How to reproduce it (as minimally and precisely as possible):
kubectl create -f nginx.yaml (a quite simple pod with nginx)
kubectl describe pods nginx

Anything else we need to know?:
This issue can be easily repro on centos 7 + kubelet 1.9.5/1.10.x + docker 1.12/1.13
However on docker 18.06 it only spends 1-2 second to start a pod ! unbelievable !
Environment:

  • Kubernetes version (use kubectl version):
    1.9.5
  • Cloud provider or hardware configuration:
  • OS (e.g. from /etc/os-release):
    CentOS 7.3
  • Kernel (e.g. uname -a):
    Linux stcvl-115 3.10.0-862.9.1.el7.x86_64 #1 SMP Mon Jul 16 16:29:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
  • Install tools:
  • Others:
    docker info:
    Server Version: 1.13.1
    Storage Driver: overlay2
    Backing Filesystem: xfs
    Supports d_type: true
    Native Overlay Diff: true
    Logging Driver: journald
    Cgroup Driver: systemd
    Plugins:
    Volume: local
    Network: bridge host macvlan null overlay
    Swarm: inactive
    docker version:
    Server:
    Version: 1.13.1
    API version: 1.26 (minimum version 1.12)
    Package version: docker-1.13.1-68.gitdded712.el7.centos.x86_64
    Go version: go1.9.4
    Git commit: dded712/1.13.1
    Built: Tue Jul 17 18:34:48 2018
    OS/Arch: linux/amd64
    Experimental: false
    cat /etc/docker/daemon.json
    {
    "insecure-registries": [],
    "storage-driver": "overlay2",
    "exec-opts": ["native.cgroupdriver=systemd"],
    "graph": "/var/lib/docker"
    }
@Cherishty

This comment has been minimized.

Show comment
Hide comment
@Cherishty

Cherishty Sep 14, 2018

/sig scheduling

Cherishty commented Sep 14, 2018

/sig scheduling

@qzyse2017

This comment has been minimized.

Show comment
Hide comment
@qzyse2017

qzyse2017 commented Sep 15, 2018

/sig node

@k8s-ci-robot

This comment has been minimized.

Show comment
Hide comment
@k8s-ci-robot

k8s-ci-robot Sep 16, 2018

Contributor

@qzyse2017: GitHub didn't allow me to assign the following users: qzyse2017.

Note that only kubernetes members and repo collaborators can be assigned.
For more information please see the contributor guide

In response to this:

/assign

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Contributor

k8s-ci-robot commented Sep 16, 2018

@qzyse2017: GitHub didn't allow me to assign the following users: qzyse2017.

Note that only kubernetes members and repo collaborators can be assigned.
For more information please see the contributor guide

In response to this:

/assign

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k82cn

This comment has been minimized.

Show comment
Hide comment
@k82cn

k82cn Sep 16, 2018

Member

@Cherishty Can you help to show Pod's status? It will give some guidance on which phase consume most .

Member

k82cn commented Sep 16, 2018

@Cherishty Can you help to show Pod's status? It will give some guidance on which phase consume most .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment