Marathon pluggable #138

enxebre · 2015-05-06T14:02:12Z

No description provided.

enxebre · 2015-05-06T14:05:14Z

site.yml

@@ -5,6 +5,11 @@
    - consul

 - hosts: mesos_masters


This will be changed so not promping all the time when we have more frameworks

Is it expected that we would change this in a future PR, or now? I'd rather standardise on the approach now so we can just rinse and repeat for each "add-on".

At a high-level I guess this means setting an environment variable for marathon_enabled (via the bootstrap scripts aws/do/etc) and setting its default to true?

Maybe we come up with some convention - framework_FRAMEWORKNAME_enabled (or something like that). then we can just have env vars like -

framework_marathon_enabled
framework_jenkins_enabled

thoughts?

we also need to consider other "add-ons" which are not mesos frameworks. e.g. #134, #55

yeh lets set this up now, so we can easily add more later. definitely seems like a thing that would go in the bootstrap scripts. That naming convention looks simple enough.

tayzlor · 2015-05-07T08:14:12Z

I think as part of this PR we are going to want some docs around how to create an Add-on (for future add-ons), and how to contribute them / developing for apollo in the main readme

wallies · 2015-05-07T08:58:08Z

bootstrap/aws/config-default.sh

@@ -13,7 +13,8 @@ ZONE=${APOLLO_AWS_ZONE:-eu-west-1b}
 MASTER_SIZE=${MASTER_SIZE:-m1.medium}
 SLAVE_SIZE=${SLAVE_SIZE:-m1.medium}
 NUM_SLAVES=${NUM_SLAVES:-1}
-
+FRAMEWORK_MARATHON_ENABLED=${FRAMEWORK_MARATHON_ENABLED:-'Y'}


seems to be alot of duplication in the aws and digitalocean bootstrap scripts, can we make a common bootstrap, so when we add more providers or frameworks later its easier

Completely agree. I'll move to its own file the plugins settings. We can do more refactor later.
Ideally/eventually the only settings inside /some-cloud/config-default.sh should be the credential ones

For common settings / environment variables we can either use the existing boostrap/apollo-env.sh (which is included everywhere) or add some other common ones that get included so we don't need to repeat ourselves

apollo-env.sh seems like the rational place for it

tayzlor · 2015-05-08T13:08:29Z

.gitignore

@@ -9,3 +9,4 @@ packer/build
 .wercker
 *.cache
 ssh.config
+apollo-plugins


would just call this 'plugins'

tayzlor · 2015-05-08T14:21:07Z

to fix the test failures you'll need to rebase off master

tayzlor · 2015-05-08T16:18:07Z

CONTRIBUTING.md

+
+We want Apollo core to be as much slim as possible providing a cloud agnostic mesos cluster with an autodiscovery system based on consul for multi-service tasks.
+
+This reduce the impact of core changes allowing to the user customize Apollo behaviour via plugins for satisfying the requirements of a given project.


"reduces"

"allowing the user"

"customise"

tayzlor · 2015-05-08T20:09:21Z

Can we keep the scope of this PR to just providing the plugin (with a static variable will be ok for backwards compatibility). For handling environment variables i've opened #152

tayzlor · 2015-05-11T11:11:08Z

Vagrantfile

@@ -45,6 +46,7 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
    }
  end
  mesos_zk_url = "zk://"+master_infos.map{|master| master[:ip]+":2181"}.join(",")+"/mesos"
+  marathon_zk_url = "zk://"+master_infos.map{|master| master[:ip]+":2181"}.join(",")


does this need +"/marathon" on the end?

that's handled inside the role as for the same ips it needs /marathon and /mesos

ah ok, just found it a bit weird that we seem to do it for the mesos one but not for the other - do we want to make the behaviour consistent here (one way or other)?

tayzlor · 2015-05-31T12:22:53Z

Giving this branch a spin on my machine. My initial beef with it is -

root@apollo-mesos-master-1:~# docker images
mesosphere/marathon      v0.8.1              69c22351652e        2 days ago          1.118 GB

The image is massive and it has to be downloaded 3 times (on each master) - so the provisioning time is increased enormously :'(

Aside from that, it functionally works well.

tayzlor · 2015-06-01T15:30:05Z

Added some code here to deal with #270

… ansible provision

tayzlor · 2015-06-22T09:03:22Z

This isn't going far, probably because the PR is huge. Maybe we should chunk it up to make it a bit easier to review / merge?

We could split it into 3 PRs -

1 PR for the packer box updates (should be backward compatible anyway)
1 PR that just provides the plugin mechanism with no plugins
1 PR for the marathon plugin (would be dependent on the 2nd one, but we could review the plugin and get it in ahead paving the way for some other plugins)

Thoughts ?

wallies · 2015-06-22T09:52:37Z

would agree with @tayzlor. will make it easier to get in, if we split it up

enxebre · 2015-06-22T10:01:59Z

The main thing stopping this is the packer images having to be rebuild. And decisions around plugin system.
I'll move out from this PR the packer related stuff so we can create a separate PR with it and others packer updates

enxebre · 2015-06-24T14:27:00Z

this is being split in to #338 and #323
This PR will be use to provided the plugin mechanism

enxebre · 2015-07-01T09:32:46Z

Closing this one in favour of #357

enxebre reviewed May 6, 2015
View reviewed changes

wallies reviewed May 7, 2015
View reviewed changes

This was referenced May 7, 2015

Deploy cadvisor to the cluster for monitoring docker containers #55

Closed

Provision a docker registry for use #117

Open

tayzlor reviewed May 8, 2015
View reviewed changes

.gitignore

@@ -9,3 +9,4 @@ packer/build

.wercker

*.cache

ssh.config

apollo-plugins

Copy link

Member

tayzlor May 8, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would just call this 'plugins'

tayzlor reviewed May 8, 2015
View reviewed changes

enxebre mentioned this pull request May 9, 2015

Handle environment variables more dynamically #152

Closed

enxebre force-pushed the marathon-pluggable branch 2 times, most recently from 44ec3d9 to fecb37e Compare May 10, 2015 18:05

tayzlor reviewed May 11, 2015
View reviewed changes

enxebre force-pushed the marathon-pluggable branch 5 times, most recently from 286cf62 to 1df0b22 Compare May 12, 2015 14:31

enxebre force-pushed the marathon-pluggable branch 3 times, most recently from 4bb9073 to a009e26 Compare May 15, 2015 16:20

enxebre added 2 commits May 31, 2015 12:52

moving marathon out to a pluggable container

28e15f1

Adding mesos_local_address on vagrant.

5546917

tayzlor force-pushed the marathon-pluggable branch from 4e4dca7 to 5546917 Compare May 31, 2015 12:28

#270 Download docker images to the packer image

5aa0b98

tayzlor added 2 commits June 12, 2015 13:52

Merge branch 'master' into marathon-pluggable

8a4cbba

Updating versions of components/plugins

8872422

tayzlor mentioned this pull request Jun 12, 2015

Upgrade marathon to 0.8.2 #295

Closed

tayzlor added 2 commits June 12, 2015 14:21

Remove example containers from packer, remove docker-py download from…

c7d6843

… ansible provision

Fixup a bad merge on the Vagrantfile

55c60cf

enxebre added the needs work label Jun 24, 2015

enxebre closed this Jul 1, 2015

enxebre deleted the marathon-pluggable branch July 1, 2015 14:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Marathon pluggable #138

Marathon pluggable #138

enxebre commented May 6, 2015

enxebre May 6, 2015

tayzlor May 6, 2015

wallies May 6, 2015

tayzlor commented May 7, 2015

wallies May 7, 2015

enxebre May 7, 2015

tayzlor May 7, 2015

wallies May 7, 2015

tayzlor May 8, 2015

tayzlor commented May 8, 2015

tayzlor May 8, 2015

tayzlor commented May 8, 2015

tayzlor May 11, 2015

enxebre May 11, 2015

tayzlor May 11, 2015

tayzlor commented May 31, 2015

tayzlor commented Jun 1, 2015

tayzlor commented Jun 22, 2015

wallies commented Jun 22, 2015

enxebre commented Jun 22, 2015

enxebre commented Jun 24, 2015

enxebre commented Jul 1, 2015


		We want Apollo core to be as much slim as possible providing a cloud agnostic mesos cluster with an autodiscovery system based on consul for multi-service tasks.

		This reduce the impact of core changes allowing to the user customize Apollo behaviour via plugins for satisfying the requirements of a given project.

Marathon pluggable #138

Marathon pluggable #138

Conversation

enxebre commented May 6, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tayzlor commented May 7, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tayzlor commented May 8, 2015

Choose a reason for hiding this comment

tayzlor commented May 8, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tayzlor commented May 31, 2015

tayzlor commented Jun 1, 2015

tayzlor commented Jun 22, 2015

wallies commented Jun 22, 2015

enxebre commented Jun 22, 2015

enxebre commented Jun 24, 2015

enxebre commented Jul 1, 2015