Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run job on each node once to help with setup #64623

Open
mitchellmaler opened this issue Jun 1, 2018 · 22 comments

Comments

@mitchellmaler
Copy link

commented Jun 1, 2018

Hello,

I am looking to see if it is possible to have a job run on each node in the cluster once. Right now our cluster is dynamically provisioned and scaled and was looking to use the kubernetes job and cronjob feature to run things to setup the node once it is provisioned or have a cron make sure something is cleaned up on each node.

@mitchellmaler

This comment has been minimized.

Copy link
Author

commented Jun 1, 2018

/sig apps

@k8s-ci-robot k8s-ci-robot added sig/apps and removed needs-sig labels Jun 1, 2018
@CaoShuFeng

This comment has been minimized.

Copy link
Member

commented Jun 4, 2018

@mitchellmaler

This comment has been minimized.

Copy link
Author

commented Jun 4, 2018

@CaoShuFeng I guess a daemonset would work but I was looking more towards the job/cron-job api to have something scheduled from the kubernetes side. I guess I could create a container with a cronjob which is ran from a daemonset.

@CaoShuFeng

This comment has been minimized.

Copy link
Member

commented Jun 5, 2018

I guess I could create a container with a cronjob which is ran from a daemonset.

👍

@kow3ns kow3ns added this to Backlog in Workloads Jun 5, 2018
@fejta-bot

This comment has been minimized.

Copy link

commented Sep 3, 2018

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@fejta-bot

This comment has been minimized.

Copy link

commented Oct 3, 2018

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

@mitchellmaler

This comment has been minimized.

Copy link
Author

commented Oct 3, 2018

/remove-lifecycle rotten

@mitchellmaler

This comment has been minimized.

Copy link
Author

commented Oct 3, 2018

Currently we got around this by using a third party tool to run automation on the nodes. It would be useful if jobs (daemon jobs) could be ran once on each node and even if based on node selector.

@fejta-bot

This comment has been minimized.

Copy link

commented Jan 1, 2019

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@fejta-bot

This comment has been minimized.

Copy link

commented Jan 31, 2019

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

@luisdavim

This comment has been minimized.

Copy link

commented Feb 21, 2019

/remove-lifecycle rotten
/remove-lifecycle stale

@luisdavim

This comment has been minimized.

Copy link

commented Feb 21, 2019

I've seen this being requested many times and I think that DaemonSets should allow for a RestartPolicy of OnFailure and jobs should allow using that to execute tasks on all nodes once.

@draveness

This comment has been minimized.

Copy link
Member

commented Mar 25, 2019

Hi @luisdavim, do we still need to support this feature? Maybe I could submit a PR to add OnFailure policy to DeamonSet.

@luisdavim

This comment has been minimized.

Copy link

commented Mar 25, 2019

Sure, for now I've a workaround using metacontroller and a CRD but having this being supported natively would be great.

@draveness

This comment has been minimized.

Copy link
Member

commented Mar 26, 2019

@luisdavim OK. Is it necessary to open a proposal for this change? And if it is, where should I raise it. :)

@draveness

This comment has been minimized.

Copy link
Member

commented Mar 27, 2019

I found out there are multiple issues which discussed the support of run a once off task with daemonset.

#36601
#50689
#69001

Support OnFailure policy in daemonset policy seems quite reasonable. It could support run once job to some extent, though it cannot ensure the task run precisely once on each node.

@kubernetes/sig-apps-feature-requests

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

commented Mar 27, 2019

@draveness: Reiterating the mentions to trigger a notification:
@kubernetes/sig-apps-feature-requests

In response to this:

I found out there are multiple issues which discussed the support of run a once off task with daemonset.

#36601
#50689
#69001

Support OnFailure policy in daemonset policy seems quite reasonable. It could support run once job to some extent, though it cannot ensure the task run precisely once on each node.

@kubernetes/sig-apps-feature-requests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@draveness draveness referenced this issue Apr 4, 2019
6 of 6 tasks complete
@fejta-bot

This comment has been minimized.

Copy link

commented Jun 25, 2019

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@draveness

This comment has been minimized.

Copy link
Member

commented Jun 25, 2019

/remove-lifecycle stale

@draveness

This comment has been minimized.

Copy link
Member

commented Jun 25, 2019

/assign

@Neumann-Nils Neumann-Nils referenced this issue Jul 15, 2019
5 of 5 tasks complete
mudler added a commit to cloudfoundry-community/eirini-bosh-release that referenced this issue Jul 23, 2019
We need the certificates on each nodes, but we cannot have a job on each
nodes, and daemonsets can't be run as a oneshots.

As a workaround keep the process alive and do nothing.

See kubernetes/kubernetes#64623
@fejta-bot

This comment has been minimized.

Copy link

commented Sep 23, 2019

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@OJFord

This comment has been minimized.

Copy link

commented Sep 23, 2019

/remove-lifecycle stale

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Workloads
  
Backlog
7 participants
You can’t perform that action at this time.