Suggestion: get rid of the CoreOS / "buddy"

[arkadijs]

While flexible, the buddy / hosts file things complicates the deployment.
I ended up moving Fleet logic into Heapster and getting rid of noticeable chunk of the code. As there are no currently other types of the buddies (Mesos?), I suggest doing something like the following:
arkadijs@7c4fc6c

1. If there is Kubernetes - use listMinions
2. Otherwise try Fleet.

The code is little bit messy - coupling KubeSource to ExternalSource, but my baseline is CoreOS with Deis, with optional Kubernetes, thus such practical solution.

[vishh]

@arkadijs I am curious to know the deployment issues with using the buddy container?
In general, we intend to move towards a side-car based model for discovery, sharding and storage instead of a single monolithic binary.

[arkadijs]

1. +1 container to deploy and explain to newcomer. I prefer to code a solution that just works. Please avoid microservices envy.
2. Exchanging live data via JSON formatted data on the filesystem? Really?
3. The buddy is overwriting the hosts file in-place, thus a race condition with the reader (heapster). Use rename().
4. There is code in Heapster that queries Kubernetes nodes API. following the logic it should be removed and placed into separate container.
   https://github.com/GoogleCloudPlatform/heapster/blob/master/sources/kube.go#L110

[vishh]

On Wed, Feb 11, 2015 at 12:26 AM, arkadijs notifications@github.com wrote:

1. +1 container to deploy and explain to newcomer. I prefer to code a
   solution that just works. Please avoid microservices envy.

Agreed. Although cluster management tools like Kubernetes or (maybe) Fig,
should solve this problem.

1. Exchanging live data via JSON formatted data on the filesystem?
   Really?

The data being exchanged was expected to change very in-frequently and in
some cases not change at all.

1. The buddy is overwriting the hosts file in-place, thus a race
   condition with the reader (heapster). Use rename().

Valid point. Needs to be fixed.

1. There is code in Heapster that queries Kubernetes nodes API.
   following the logic it should be removed and placed into separate
   container.
   https://github.com/GoogleCloudPlatform/heapster/blob/master/sources/kube.go#L110

That is the intent in the long run.
We want to limit the number of platforms that we support natively. Why?
Every new platform being added needs to be tested and maintained as part of
the core binary. Separating out node discovery helps to shard and scale
heapster easily. But limiting to CoreOS in addition to Kubernetes is fine.