Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kube informer does not support jobs because it only looks for pods? #253

Closed
Cryptophobia opened this issue Sep 13, 2021 · 0 comments · Fixed by #293
Closed

Kube informer does not support jobs because it only looks for pods? #253

Cryptophobia opened this issue Sep 13, 2021 · 0 comments · Fixed by #293
Assignees

Comments

@Cryptophobia
Copy link
Contributor

Cryptophobia commented Sep 13, 2021

TLDR: Restart control loop when KFO finds event that a job has started a pod in a namespace. Watch jobslist similar to podlist and namespaces. If a namespace has only jobs and no pods, we currently miss informing on new pods from those jobs in that namespaces, unless control loop runs again meanwhile.

factory.Batch().V1().Jobs().Informer()

If factory informer finds new pods for job. Then c.RunOnce() and make controller run again (main control loop).

KubeInformer needs to use batch api v1 to list Jobs and create MiniContainers.
Then we need to use CronJob name as label selector to get the pods for that namespace. If a namespace has only jobs and no running pods from jobs at the time of execution, then KFO will not find it.

Implement KubernetersInformer to find when cronjob starts a pod in a given namespace and make config-reloader restart main loop when a new pod from a job starts?

Note: The job name and pod name are different.

Example code that we need:

	var kubeInformerFactory kubeinformers.SharedInformerFactory
	// Sync event only
	if namespace == "" {
		kubeInformerFactory = kubeinformers.NewSharedInformerFactory(kubeClient, 0)
	} else {
		kubeInformerFactory = kubeinformers.NewSharedInformerFactoryWithOptions(kubeClient, 0, kubeinformers.WithNamespace(namespace))
	}

	controller := NewController(kubeClient, kubeInformerFactory.Batch().V1().Jobs())

	kubeInformerFactory.Start(stopCh)

Update the code below in kube_informer to query jobs and to rerun main control loop of KFO if it finds pods that a job has started?

Need to find new pods per job informer event:
https://github.com/vmware/kube-fluentd-operator/blob/master/config-reloader/datasource/kube_informer.go#L26-L33

@Cryptophobia Cryptophobia self-assigned this Sep 13, 2021
Cryptophobia pushed a commit to Cryptophobia/kube-fluentd-operator that referenced this issue Nov 24, 2021
 - add kube pod informer for pods add/rm in namespace we monitor, rerun preprocess configs
 - fixes vmware#289
 - fixes vmware#253
 - fixes issue with respecting `c.cfg.AnnotConfigmapName` on configmap namespaces discovery
 - introduces `hashstructure` package for smarter hashing of changing objects
 - make smarter hashing for `allConfigsHash` metahashing (without `reflect.DeepEqual()`)

Signed-off-by: Anton Ouzounov <aouzounov@vmware.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant