Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically recreate pvc when sts pod is stuck in pending #1

Open
wants to merge 1,970 commits into
base: master
Choose a base branch
from

Conversation

rrangith
Copy link
Owner

@rrangith rrangith commented Oct 18, 2022

based on mucahitkurt@1ab4b97

there was some discussion on that change ^ here kubernetes#74374

I added in a check to see if the pod was pending for > 30 seconds (arbitrary value) so that we dont do anything if its a transient pending pod.

The code will check the informer for the pod's pvcs and if a pvc is not found, it will recreate it. since it is using the informer, this won't make a call directly to the api server each time. And the pvc create call should be idempotent (need to confirm through testing). So even if the informer is out of date, and it says a pvc is missing even though it exists, the create call should just do nothing

@rrangith rrangith force-pushed the fix/create-pvc-for-pending-pod branch 3 times, most recently from 5de3112 to 5a9b8da Compare October 21, 2022 21:05
@rrangith rrangith force-pushed the fix/create-pvc-for-pending-pod branch from f3243a0 to 3f21c6c Compare October 31, 2022 20:53
@rrangith rrangith force-pushed the fix/create-pvc-for-pending-pod branch from 06a7a98 to b7120e6 Compare November 10, 2022 15:11
@rrangith rrangith force-pushed the fix/create-pvc-for-pending-pod branch from 8d28e93 to 34da5a1 Compare November 17, 2022 21:45
@rrangith rrangith force-pushed the fix/create-pvc-for-pending-pod branch 2 times, most recently from e0ee169 to 1031ed0 Compare December 2, 2022 16:00
@rrangith rrangith force-pushed the fix/create-pvc-for-pending-pod branch from f3dbc4d to fe0652b Compare December 19, 2022 16:23
249043822 and others added 21 commits January 5, 2023 09:10
…hould correctly account for terminated pods after restart
Signed-off-by: Dave Chen <dave.chen@arm.com>
This is mostly because the path, even for built-in, can change from test
to test, so it's easier to make the whole thing configurable anyway.
This reverts commit 0b93994.

Signed-off-by: aimuz <mr.imuz@gmail.com>
`genCSRConfig.kubeadmConfig` is possible to be nil if there any error
from the config loading, so access the field should only be done if
there is no error in the previous step.

Signed-off-by: Dave Chen <dave.chen@arm.com>
kubeadm: unittest might run as non-root
Before, in RunPostFilterPlugins, we didn't distinguish between unschedulable and unresolvable
because we only have one postFilterPlugin by default, now, we have at least two, we should
make sure that once a postFilterPlugin returns unresolvable, we'll return directly

Signed-off-by: Kante Yin <kerthcet@gmail.com>
…tainer update should trigger a policy evaluation
…e-group-name-registration

code-generator: refer to the API package for GV{R,K}
Signed-off-by: TommyStarK <thomasmilox@gmail.com>
Fix:[Flake] [sig-node] Restart [Serial] [Slow] [Disruptive] Kubelet hould correctly account for terminated pods after restart
…ndler

Drop unused EndpointsHandler / EndpointsConfig from pkg/proxy/config
kubeadm: fix the nil pointer dereference in testcase
It should be possible to import `kms/api` for `kms/service` package.

Signed-off-by: Krzysztof Ostrowski <kostrows@redhat.com>
…tstop

Stop Cache in scheduler when tests tear down
smarterclayton and others added 29 commits January 14, 2023 14:50
While refactoring the backoff manager to simplify and unify the code
in wait a race condition was encountered in
TestSharedInformerWatchDisruption. The new implementation failed
because the fake clock was not propagated to the backoff managers
when the reflector was used in a controller. After ensuring the
mangaers, reflector, controller, and informer shared the same
clock the test needed was updated to avoid the race condition by
advancing the fake clock and adding real sleeps to wait for
asynchronous propagation of the various goroutines in the controller.

Due to the deep structure of informers it is difficult to inject
hooks to avoid having to perform sleeps. At a minimum the FakeClock
interface should allow a caller to determine the number of waiting
timers (to avoid the first sleep).
hack/pin-dependency.sh github.com/moby/ipvs v1.1.0

- go to a fixed tag for `vishvananda/netns`
- no more references to `pkg/errors`

Signed-off-by: Davanum Srinivas <davanum@gmail.com>
…ture/UpdateOrCreateTokenAddErrProcessing

Revert "UpdateOrCreateToken get secrets err handling optimization"
use --null instead of -Z in grep invocation
…pvs-v1.1.0

Update to new release of moby/ipvs
…ck_clock

cache: Reflector should have the same injected clock as its informer
…dition-for-terminated-pods

PodGC should not add DisruptionTarget condition for pods which are in terminal phase
…ontainer annotation

Signed-off-by: Paco Xu <paco.xu@daocloud.io>
add e2e test for kubectl default container annotation
…gv0-controller

use klog instead of klog.V(0)--controller manager part
…nused-functions

delete unused functions in pkg/util directory
…-unused-functions

delete unused functions in pkg/volume directory
…-functions

delete unused functions in pkg directory
…lumebinding

feature(volumebinding): return Skip in PreFilter
kubectl: use autoscaling v2 for hpa
cleanup(kubectl-e2e): remove invalid comments
Wait for logs pod to complete before counting lines
@rrangith rrangith force-pushed the fix/create-pvc-for-pending-pod branch from 8c9867f to 2f0eb54 Compare January 17, 2023 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet