Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

discussion / plan LXC integration #6862

Closed
mswart opened this issue Apr 15, 2015 · 35 comments
Closed

discussion / plan LXC integration #6862

mswart opened this issue Apr 15, 2015 · 35 comments

Comments

@mswart
Copy link

@mswart mswart commented Apr 15, 2015

As discussed in https://groups.google.com/forum/#!topic/google-containers/aWZ0U_SIA6A supporting plain LXC containers (or lmctfy / lxd) requires some or a lot of changes.

I am very interested in managing LXC containers with kubernetes and also implementing this. But I would like to get some hints what needs to be done/how much work this would be:

  • What changes are needed/what components need to be extended?
  • What architectural decision needs to be made before the implementation is reasonable?
  • What are the minimal changes?
  • Are there any changes that should/need to be done before a integration is practical?
@thockin
Copy link
Member

@thockin thockin commented Apr 15, 2015

The ongoing work to support Rocket should be clearing a lot of the
underbrush for you. I think the biggest issues are

  1. What parts of the API need to be changed/extended/removed to capture an
    LXC runtime?
  2. What parts of Kubelet " " " " ...

Dawn is overseeing the Rocket work, so I'll let her be more detailed, but
I'd be happy to see LXC support.

On Wed, Apr 15, 2015 at 7:27 AM, Malte Swart notifications@github.com
wrote:

As discussed in
https://groups.google.com/forum/#!topic/google-containers/aWZ0U_SIA6A
supporting plain LXC containers (or lmctfy / lxd) requires some or a lot of
changes.

I am very interested in managing LXC containers with kubernetes and also
implementing this. But I would like to get some hints what needs to be
done/how much work this would be:

  • What changes are needed/what components need to be extended?
  • What architectural decision needs to be made before the
    implementation is reasonable?
  • What are the minimal changes?
  • Are there any changes that should/need to be done before a
    integration is practical?


Reply to this email directly or view it on GitHub
#6862.

@erictune
Copy link
Member

@erictune erictune commented Apr 15, 2015

@monokal
Copy link

@monokal monokal commented May 10, 2015

LXC/LXD support in Kubernetes would be awesome. Forming containers around a single process (hey Docker) is obviously the ideal in terms of a micro-service infrastructure, but a lot of us don't yet have this pleasure (e.g, those of us tasked with moving an organisation in to 'DevOps' but with legacy services to support in the meantime).

This is obviously not trivial to implement, but it shouldn't be the goliath task it was pre-LXD now that we have a means of nicely transferring and migrating LXC containers around hosts, given, we're early days. I've been looking at Mesos/Marathon, CoreOS, etc, etc, but nothing seems to solve the problem of actually orchestrating a cluster of non-Docker containers.

I believe the first task would be to implement the runtime abstraction, which is currently in progress with rkt in mind.

@vmarmol
Copy link
Contributor

@vmarmol vmarmol commented May 11, 2015

The runtime abstraction is complete and rkt is using it today. A similar thing could be done with LXC.

@monokal
Copy link

@monokal monokal commented May 11, 2015

Brilliant, good work. For anyone following up on LXC/LXD support, please head over here: https://groups.google.com/forum/#!topic/google-containers/aWZ0U_SIA6A

@sl1pm4t
Copy link

@sl1pm4t sl1pm4t commented Nov 5, 2015

+1

1 similar comment
@jgnagy
Copy link

@jgnagy jgnagy commented Apr 5, 2016

+1

@bj916b
Copy link

@bj916b bj916b commented May 6, 2016

@thockin have there been any other discussions/work around this issue? i noticed it's a little old in kubernetes-land, but it would be sort of a nice feature to have for existing organizations who have LXC-based deployments. i'm only curious.

@tebanep
Copy link

@tebanep tebanep commented May 10, 2016

+1

@bprashanth
Copy link

@bprashanth bprashanth commented May 10, 2016

@kubernetes/sig-node

@yujuhong
Copy link
Member

@yujuhong yujuhong commented May 10, 2016

We are in the process (#22964) of redefining the container runtime interface, which serves as the main interface for runtime integration, to improve feature velocity and extensibility.
Now would be a great time to join the discussions If you have any input/suggestions/concerns.

@malix0
Copy link

@malix0 malix0 commented May 13, 2016

+1 to have LXC/LXD support

@vishh
Copy link
Member

@vishh vishh commented May 13, 2016

It would help if someone can enumerate the reasons for wanting lxc support...

@v1k0d3n
Copy link

@v1k0d3n v1k0d3n commented May 13, 2016

@vishh would this be similar to PR #13079?

application containers are not the same as machine containers from a security perspective, since there is no control over declaring the kernel security posture. i'm not claiming to know what it would take to make this happen in kubernetes-land, but I definitely see a lot of interesting brewing on our side for additional options.

@vishh
Copy link
Member

@vishh vishh commented May 13, 2016

@v1k0d3n Nope. There will be a proposal out (soon) to define a more user-friendly interface against which lxd can be integrated pretty easily.
It would help if you can enumerate a few advanced/additional options that you need from Kubelet.

@bactisme
Copy link

@bactisme bactisme commented May 14, 2016

@vishh I am not saying that docker's one-process-per-container is broken ... but some people do 😊

If your application talk to a log daemon or simple mail server or ntp, have cron tasks etc. Docker require you to spin as much container. Of course it is better for horizontal scalability and process management ... but it also bring a damageable complexity ...

Lxc can bring simpler deployment: careffully build your box, ship.

@monokal
Copy link

@monokal monokal commented May 15, 2016

I think the majority of people want LXD/LXC support in order to allow for
legacy services. Not everyone's fortune enough to have a completely
green-field environment, myself included.

Mesosphere's DC/OS is now open-source so that's an option, but Kubernetes
support would allow for a single unified platform.
On 14 May 2016 7:46 p.m., "Baptiste M." notifications@github.com wrote:

@vishh https://github.com/vishh I am not saying that docker's
one-process-per-container is broken ... but some people do 😊

If your application talk to a log daemon or simple mail server or ntp,
have cron tasks etc. Docker require you to spin as much container. Of
course it is better for horizontal scalability and process management ...
but it also bring a damageable complexity ...

Lxc can bring simpler deployment: careffully build your box, ship.


You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#6862 (comment)

@vishh
Copy link
Member

@vishh vishh commented May 18, 2016

I'm convinced. Once #22964 takes some shape (hopefully very soon), we can take in community contributions for lxc/lxd integration. As part of #22964, we are attempting to define a client-server model for integrating with alternate runtimes and it should be (hopefully) straightforward to integrate with lxc/lxd.

@bj916b
Copy link

@bj916b bj916b commented May 18, 2016

@vishh this is great news! very much looking forward to this. i'll start watching #22964 more closely.

@dragon9783
Copy link

@dragon9783 dragon9783 commented Jun 20, 2016

+1

@nick-walt
Copy link

@nick-walt nick-walt commented Jul 11, 2016

LXD is the way forward in the full system-container use case and it definitely has a place there with Rkt and Docker. With LXD's natural API (ugh, might even be better than Docker's?) and its capability to also be used as an incredibly lightweight single app-container, as well as a full system-container, we can expect to see increased usage of this technology. So it would be awesome to have native support in Kuber.

@abcfy2
Copy link

@abcfy2 abcfy2 commented Sep 16, 2016

+1

@sequoiar
Copy link

@sequoiar sequoiar commented Oct 9, 2016

+1

@dronnix
Copy link

@dronnix dronnix commented Oct 19, 2016

+1

@pouledodue
Copy link

@pouledodue pouledodue commented Oct 20, 2016

+111

@techtonik
Copy link

@techtonik techtonik commented Oct 20, 2016

Is anyone hiring to work on that?

@SpiLLeR
Copy link

@SpiLLeR SpiLLeR commented Oct 22, 2016

+1

@thockin
Copy link
Member

@thockin thockin commented Oct 22, 2016

EVERYONE PLEASE STOP +1ing this. GitHub has a sentiment button - look at the first message, click the thumbs-up. Every time you +1 it in a message, dozens or hundreds of people get email.

Like so many issues, this requires someone to act as the driver for it. We're doing everything we can to make it possible to integrate things like this, but we (Google team and the larger Kube-team) do not have the bandwidth to do every cool idea.

@pouledodue
Copy link

@pouledodue pouledodue commented Oct 22, 2016

@vishh for orchestration of stateful containers like database, replicas and their associated data

see: https://news.ycombinator.com/item?id=10609247

@malix0
Copy link

@malix0 malix0 commented Jan 25, 2017

Canonical release is own distribution of kubernetes
https://jujucharms.com/canonical-kubernetes/bundle/
but does not add LXC/LXD support

@malix0
Copy link

@malix0 malix0 commented Mar 2, 2017

The google group discussion has an updated URL https://groups.google.com/forum/m/#!topic/kubernetes-users/aWZ0U_SIA6A

@fejta-bot
Copy link

@fejta-bot fejta-bot commented Dec 21, 2017

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

@fejta-bot
Copy link

@fejta-bot fejta-bot commented Jan 20, 2018

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle rotten
/remove-lifecycle stale

@fejta-bot
Copy link

@fejta-bot fejta-bot commented Feb 19, 2018

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

@malix0
Copy link

@malix0 malix0 commented Jul 16, 2019

The LXD shim is under development https://github.com/automaticserver/lxe

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.