discussion / plan LXC integration

[mswart]

As discussed in https://groups.google.com/forum/#!topic/google-containers/aWZ0U_SIA6A supporting plain LXC containers (or lmctfy / lxd) requires some or a lot of changes.

I am very interested in managing LXC containers with kubernetes and also implementing this. But I would like to get some hints what needs to be done/how much work this would be:

• What changes are needed/what components need to be extended?
• What architectural decision needs to be made before the implementation is reasonable?
• What are the minimal changes?
• Are there any changes that should/need to be done before a integration is practical?

[thockin]

The ongoing work to support Rocket should be clearing a lot of the
underbrush for you. I think the biggest issues are

1. What parts of the API need to be changed/extended/removed to capture an
   LXC runtime?
2. What parts of Kubelet " " " " ...

Dawn is overseeing the Rocket work, so I'll let her be more detailed, but
I'd be happy to see LXC support.

On Wed, Apr 15, 2015 at 7:27 AM, Malte Swart notifications@github.com
wrote:

As discussed in
https://groups.google.com/forum/#!topic/google-containers/aWZ0U_SIA6A
supporting plain LXC containers (or lmctfy / lxd) requires some or a lot of
changes.

I am very interested in managing LXC containers with kubernetes and also
implementing this. But I would like to get some hints what needs to be
done/how much work this would be:

• What changes are needed/what components need to be extended?
• What architectural decision needs to be made before the
  implementation is reasonable?
• What are the minimal changes?
• Are there any changes that should/need to be done before a
  integration is practical?

—
Reply to this email directly or view it on GitHub
GoogleCloudPlatform#6862.

[erictune]

@dchen1107

[monokal]

LXC/LXD support in Kubernetes would be awesome. Forming containers around a single process (hey Docker) is obviously the ideal in terms of a micro-service infrastructure, but a lot of us don't yet have this pleasure (e.g, those of us tasked with moving an organisation in to 'DevOps' but with legacy services to support in the meantime).

This is obviously not trivial to implement, but it shouldn't be the goliath task it was pre-LXD now that we have a means of nicely transferring and migrating LXC containers around hosts, given, we're early days. I've been looking at Mesos/Marathon, CoreOS, etc, etc, but nothing seems to solve the problem of actually orchestrating a cluster of non-Docker containers.

I believe the first task would be to implement the runtime abstraction, which is currently in progress with rkt in mind.

[vmarmol]

The runtime abstraction is complete and rkt is using it today. A similar thing could be done with LXC.

[monokal]

Brilliant, good work. For anyone following up on LXC/LXD support, please head over here: https://groups.google.com/forum/#!topic/google-containers/aWZ0U_SIA6A

[sl1pm4t]

+1

[jgnagy]

+1

[bj916b]

@thockin have there been any other discussions/work around this issue? i noticed it's a little old in kubernetes-land, but it would be sort of a nice feature to have for existing organizations who have LXC-based deployments. i'm only curious.

[tebanep]

+1

[bprashanth]

@kubernetes/sig-node

[yujuhong]

We are in the process (#22964) of redefining the container runtime interface, which serves as the main interface for runtime integration, to improve feature velocity and extensibility.
Now would be a great time to join the discussions If you have any input/suggestions/concerns.

[malix0]

+1 to have LXC/LXD support

[vishh]

It would help if someone can enumerate the reasons for wanting lxc support...

[v1k0d3n]

@vishh would this be similar to PR #13079?

application containers are not the same as machine containers from a security perspective, since there is no control over declaring the kernel security posture. i'm not claiming to know what it would take to make this happen in kubernetes-land, but I definitely see a lot of interesting brewing on our side for additional options.

[vishh]

@v1k0d3n Nope. There will be a proposal out (soon) to define a more user-friendly interface against which lxd can be integrated pretty easily.
It would help if you can enumerate a few advanced/additional options that you need from Kubelet.

[bactisme]

@vishh I am not saying that docker's one-process-per-container is broken ... but some people do 😊

If your application talk to a log daemon or simple mail server or ntp, have cron tasks etc. Docker require you to spin as much container. Of course it is better for horizontal scalability and process management ... but it also bring a damageable complexity ...

Lxc can bring simpler deployment: careffully build your box, ship.

[monokal]

I think the majority of people want LXD/LXC support in order to allow for
legacy services. Not everyone's fortune enough to have a completely
green-field environment, myself included.

Mesosphere's DC/OS is now open-source so that's an option, but Kubernetes
support would allow for a single unified platform.
On 14 May 2016 7:46 p.m., "Baptiste M." notifications@github.com wrote:

@vishh https://github.com/vishh I am not saying that docker's
one-process-per-container is broken ... but some people do 😊

If your application talk to a log daemon or simple mail server or ntp,
have cron tasks etc. Docker require you to spin as much container. Of
course it is better for horizontal scalability and process management ...
but it also bring a damageable complexity ...

Lxc can bring simpler deployment: careffully build your box, ship.

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#6862 (comment)

[vishh]

I'm convinced. Once #22964 takes some shape (hopefully very soon), we can take in community contributions for lxc/lxd integration. As part of #22964, we are attempting to define a client-server model for integrating with alternate runtimes and it should be (hopefully) straightforward to integrate with lxc/lxd.

[bj916b]

@vishh this is great news! very much looking forward to this. i'll start watching #22964 more closely.

[dragon9783]

+1

[nick-walt]

LXD is the way forward in the full system-container use case and it definitely has a place there with Rkt and Docker. With LXD's natural API (ugh, might even be better than Docker's?) and its capability to also be used as an incredibly lightweight single app-container, as well as a full system-container, we can expect to see increased usage of this technology. So it would be awesome to have native support in Kuber.

[abcfy2]

+1

[sequoiar]

+1

[dronnix]

+1

[pouledodue]

+111

[techtonik]

Is anyone hiring to work on that?

[SpiLLeR]

+1

[thockin]

EVERYONE PLEASE STOP +1ing this. GitHub has a sentiment button - look at the first message, click the thumbs-up. Every time you +1 it in a message, dozens or hundreds of people get email.

Like so many issues, this requires someone to act as the driver for it. We're doing everything we can to make it possible to integrate things like this, but we (Google team and the larger Kube-team) do not have the bandwidth to do every cool idea.

[pouledodue]

@vishh for orchestration of stateful containers like database, replicas and their associated data

see: https://news.ycombinator.com/item?id=10609247

[malix0]

Canonical release is own distribution of kubernetes
https://jujucharms.com/canonical-kubernetes/bundle/
but does not add LXC/LXD support

[malix0]

The google group discussion has an updated URL https://groups.google.com/forum/m/#!topic/kubernetes-users/aWZ0U_SIA6A

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale