Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP][DO NOT REVIEW] debug ipv6 jobs flakiness #85727

Closed
wants to merge 5 commits into from
Closed

[WIP][DO NOT REVIEW] debug ipv6 jobs flakiness #85727

wants to merge 5 commits into from
+45 −3

Commits on Dec 8, 2019

  1. Increase log level in kube-proxy

    @aojea
    aojea committed Dec 8, 2019
    Configuration menu
    Copy the full SHA
    42c7dcb View commit details
    Browse the repository at this point in the history

  2. Increase kubelet log verbosity

    @aojea
    aojea committed Dec 8, 2019
    Configuration menu
    Copy the full SHA
    a666272 View commit details
    Browse the repository at this point in the history

  3. Be more agressive acquiring the iptables lock 

    iptables has two options to modify the behaviour trying to
acquire the lock.

--wait  -w [seconds]    maximum wait to acquire xtables lock
                        before give up
--wait-interval -W [usecs]  wait time to try to acquire xtables
                            lock
                            interval to wait for xtables lock
                            default is 1 second

Kubernetes uses -w 5 that means that wait 5 seconds to try to
acquire the lock. If we are not able to acquire it, kube-proxy
fails and retries in 30 seconds, that is an important penalty
on sensitive applications.
We can be a bit more aggresive and try to acquire the lock every
100 msec, that means that we have to fail 50 times to not being
able to succeed.
    @aojea
    aojea committed Dec 8, 2019
    Configuration menu
    Copy the full SHA
    8f25f06 View commit details
    Browse the repository at this point in the history

  4. Kubelet monitor the iptables lock

    @aojea
    aojea committed Dec 8, 2019
    Configuration menu
    Copy the full SHA
    22665fc View commit details
    Browse the repository at this point in the history

  5. Don't return if StopPodSandbox fails

    @aojea
    aojea committed Dec 8, 2019
    Configuration menu
    Copy the full SHA
    022eacf View commit details
    Browse the repository at this point in the history