Revisit Pdb calls during the reconciles while job is completed

[jian-he]

Since, the reconcileTFJob is continuously invoked, this DeletePdb code will be continuously invoked even when job is completed.

That will cause the following FailedDeletePdb/SuccessfulDeletePdb events continuously get generated after job completes.

[jian-he]

It's fine to continuously invoke reconcileTFJob while the job is running even with no updates, but I think it's better to NOT invoke it continuously for completed jobs with no updates.

@gaocegege @johnugeorge @jlewi @ScorpioCPH , your opinion ?

[gaocegege]

SGTM. I think we definitely should do it. While it is not in high priority since the performance is not the primary goal in this release.

@jian-he Are you facing some problems with the performance?

[gaocegege]

It is dup with #804 , IMO. @jian-he WDYT? We could close this one and discuss about it in #804 .

There are some other resources like pods and services which also have the problem

[jlewi]

I agree once a TFJob reaches a terminal state we no longer need to call reconcile periodically; but how would we implement that?

We rely on the informer to periodically generate update events for all TFJobs to ensure reconcile gets called periodically.

[jian-he]

hmm, I guess we can have a cache to store that whether the job is already completed and reconciled once. Then next time, it can skip the reconcilation.

[johnugeorge]

This should be fixed in #1012
closing the issue
/close

[k8s-ci-robot]

@johnugeorge: Closing this issue.

In response to this:

This should be fixed in #1012
closing the issue
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.