Executors keep running if scheduler is shutdown #177

Closed
frankscholten opened this Issue Jul 21, 2015 · 6 comments

Comments

Projects
None yet
3 participants
@frankscholten
Member

frankscholten commented Jul 21, 2015

No description provided.

@philwinder philwinder added the bug label Jul 23, 2015

@frankscholten frankscholten modified the milestone: 0.2 Jul 28, 2015

@philwinder philwinder self-assigned this Jul 28, 2015

@philwinder philwinder modified the milestones: 0.2, 0.3 Aug 7, 2015

@mwl

This comment has been minimized.

Show comment
Hide comment
@mwl

mwl Aug 7, 2015

Contributor

By shutdown you mean gracefully shutdown in a situation where you want to get rid of the whole ElasticSearch cluster?

Contributor

mwl commented Aug 7, 2015

By shutdown you mean gracefully shutdown in a situation where you want to get rid of the whole ElasticSearch cluster?

@philwinder

This comment has been minimized.

Show comment
Hide comment
@philwinder

philwinder Aug 7, 2015

Contributor

Correct. Like when you issue a destroy command from marathon. The scheduler is killed gracefully, but it doesn't pass that message on to the executors.

Contributor

philwinder commented Aug 7, 2015

Correct. Like when you issue a destroy command from marathon. The scheduler is killed gracefully, but it doesn't pass that message on to the executors.

@frankscholten frankscholten modified the milestones: 0.3, 0.2.1, 0.2.2 Aug 20, 2015

@frankscholten

This comment has been minimized.

Show comment
Hide comment
@frankscholten

frankscholten Aug 20, 2015

Member

Asked on the service-dev channel

Member

frankscholten commented Aug 20, 2015

Asked on the service-dev channel

@frankscholten frankscholten modified the milestones: 0.2.2, Backlog Aug 20, 2015

@philwinder

This comment has been minimized.

Show comment
Hide comment
@philwinder

philwinder Aug 24, 2015

Contributor

I can confirm that DCOS uninstall package DOES shut down the executors correctly.

Contributor

philwinder commented Aug 24, 2015

I can confirm that DCOS uninstall package DOES shut down the executors correctly.

@philwinder

This comment has been minimized.

Show comment
Hide comment
@philwinder

philwinder Aug 25, 2015

Contributor

This is how they shut down the executors:

This is the code that does it in the CLI: https://github.com/mesosphere/dcos-cli/blob/master/dcos/mesos.py#L194
I haven’t tested this but that maps to curl -X POST http://<dcos-url>/master/shutdown -d "frameworkId=<framework-id>" with curl.
Also note that /master/shutdown has been renamed to /master/teardown in recent versions of Mesos.

Contributor

philwinder commented Aug 25, 2015

This is how they shut down the executors:

This is the code that does it in the CLI: https://github.com/mesosphere/dcos-cli/blob/master/dcos/mesos.py#L194
I haven’t tested this but that maps to curl -X POST http://<dcos-url>/master/shutdown -d "frameworkId=<framework-id>" with curl.
Also note that /master/shutdown has been renamed to /master/teardown in recent versions of Mesos.

@philwinder

This comment has been minimized.

Show comment
Hide comment
@philwinder

philwinder Aug 27, 2015

Contributor

Ok, confirmed by Mesosphere.
To kill the executors and framework state in Mesos, call /master/shutdown.
To kill the scheduler, destroy the app in Marathon.

The DCOS CLI does both of these things, so from a user perspective, it appears to cleanly shutdown the framework.

Unfortunately, since this is the responsibility of the entity that starts the framework (e.g. the user or the DCOS CLI) this is out of our control.

I recommend that we close this issue as a wontfix. Starting and stopping is largely a Mesos/Marathon issue. If it doesn't do this in a way that the user likes, that is a Mesos/Marathon problem, not an ES problem.

Contributor

philwinder commented Aug 27, 2015

Ok, confirmed by Mesosphere.
To kill the executors and framework state in Mesos, call /master/shutdown.
To kill the scheduler, destroy the app in Marathon.

The DCOS CLI does both of these things, so from a user perspective, it appears to cleanly shutdown the framework.

Unfortunately, since this is the responsibility of the entity that starts the framework (e.g. the user or the DCOS CLI) this is out of our control.

I recommend that we close this issue as a wontfix. Starting and stopping is largely a Mesos/Marathon issue. If it doesn't do this in a way that the user likes, that is a Mesos/Marathon problem, not an ES problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment