How to run this entirely under mesos

[SEJeff]

So following the instructions, it says to edit the storm.yaml file and add in storm.zookeeper.servers and nimbus.host.

So in a perfect world here is how this would work:

1. The storm-mesos project would read /etc/mesos/zk for the mesos.master.url attr if that isn't specified.
2. The framework would ensure nimbus runs somewhere, and you could run multiple instances to keep it up
3. The supervisors would lookup nimbus from zookeeper, so it doesn't have to be hardcoded
4. Magic

Or is it preferred to run this under something like marathon?

[chengweiv5]

The storm-mesos project would read /etc/mesos/zk for the mesos.master.url attr if that isn't specified.

/etc/mesos/zk is introduced by mesosphere rpm package, not a work for all solution.

The framework would ensure nimbus runs somewhere, and you could run multiple instances to keep it up

Not sure this is necessary, see storm fault tolerance

The supervisors would lookup nimbus from zookeeper, so it doesn't have to be hardcoded

Does storm stores nimbus into zookeeper? I'm not sure.

[SEJeff]

I'm not sure. At least the framework could check if /etc/mesos/zk exists and use it if it does.

Ultimately, you shouldn't need to hardcode things if you're using mesos. That is after all, one of the biggest features, is that you don't care what server it runs on, so long as it does run :)

[SEJeff]

I'll submit a PR as a python script which does this for you, is that ok? I'll register nimbus in zookeeper and then configure the UI to look at it.

[tnachen]

I think having it passed in to the framework and/or optionally looking at /etc/mesos/zk is fine.

[SEJeff]

I got it all working, by adding a python wrapper script. The concept is documented here.

What of this makes sense for the framework to natively do?

[erikdw]

I think PR #65 fixes this too -- it has a marathon launching script that will launch everything. @brndnmtthws : think we can close this?

[erikdw]

I've also spent a bit of time on a docker-compose config for launching everything, I need to come back to that.

[brndnmtthws]

Yeah, I think we can close this.