New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove the Qemu in Docker functionality #3178

Open
wants to merge 1 commit into
base: master
from

Conversation

Projects
None yet
4 participants
@justincormack
Copy link
Collaborator

justincormack commented Sep 4, 2018

This was introduced before qemu had full acceleration support
on OSX. Now the Qemu platform supports most platforms it does
not make som much sense, and allows us to simplify the code.

Signed-off-by: Justin Cormack justin@specialbusservice.com

cats on sheep

Remove the Qemu in Docker functionality
This was introduced before qemu had full acceleration support
on OSX. Now the Qemu platform supports most platforms it does
not make som much sense, and allows us to simplify the code.

Signed-off-by: Justin Cormack <justin@specialbusservice.com>
@justinbarrick

This comment has been minimized.

Copy link

justinbarrick commented Sep 4, 2018

One nice thing about Qemu in Docker is that it removes any risk of version differences and also means that users do not need to separately install Qemu (for example, on Mac OS, that's an extra brew install).

Not sure if this is is important for you, but thought I'd mention it as I've found it useful to package Qemu in Docker myself.

@justincormack

This comment has been minimized.

Copy link
Collaborator

justincormack commented Sep 4, 2018

@justinbarrick since Qemu started supporting Hypervisor.Framework on OSX its actually much faster to run it natively than in a container as you get full hardware acceleration, so I kind of changed my mind about this. But if it is useful we can leave it (my thought it was more confusing as it might run differently without being obvious why not).

@justincormack

This comment has been minimized.

Copy link
Collaborator

justincormack commented Sep 4, 2018

Alternatively we could make it opt in only, so you need to specify --containerized

@rn

This comment has been minimized.

Copy link
Member

rn commented Sep 4, 2018

We used to use the containerized version on CI (until we installed qemu in the CI machines and also enabled nested virt). I've seen CI failures which were not reproducible with accelerated qemu (both on KVM and HVF) and saw mixed results with TCG. So having a known baseline version might be useful. OTOH, we had issues/regressions with running qemu containerised in the past and we don't really test it. Removing this option would clean up the code a bit too.

So I'm in two minds about removing it, but wouldn't object merging this unless someone has a strong opinion otherwise.

@justinbarrick

This comment has been minimized.

Copy link

justinbarrick commented Sep 5, 2018

Accelerated versus non-accelerated is a good point on Mac OS as that could cause unexpected behavior. Do you know if it is possible to nest HVF virtualization and run it in Docker for Mac still?

My ideal situation would be containerized Qemu that still supports acceleration. This is how we deploy our images - Qemu inside Docker in Marathon. Inside that image, before running Qemu, we setup the disk, TAP interface, metadata disk, and run the image in a loop. But we’re not easily able to test the image in that environment on a Mac because our options are run directly in HyperKit or an unaccellerated Qemu container.

My container doesn’t actually invoke Linuxkit to run Qemu at runtime (we call Qemu from a bash script), but for users that deploy their images to Qemu and dev on a Mac, the closer it can be to a standard environment that people would use Qemu in the better (i.e. a Linux-like host).

@rn

This comment has been minimized.

Copy link
Member

rn commented Sep 5, 2018

you can't run "nested" on macOS. The qemu in the (linux) container almost certainly does not have the acceleration compiled in for HVF (and wn't be able to use it even if one could map this in somehow).

If we run qemu containerised on Linux we mount in /dev/kvm if present so you'd get acceleration there even if run in a container

@justinbarrick

This comment has been minimized.

Copy link

justinbarrick commented Sep 5, 2018

Bummer. That does remove much of the utility there for me. Maybe someday!

@justincormack

This comment has been minimized.

Copy link
Collaborator

justincormack commented Sep 5, 2018

@justinbarrick VMWare Fusion is the only reliable way of getting nested virt on OSX. You should be able to set it up for testing; we do have LinuxKit support for it too.

Wondering if I should just split qemu in docker to a different target instead of removing it?

@rn

This comment has been minimized.

Copy link
Member

rn commented Sep 5, 2018

Wondering if I should just split qemu in docker to a different target instead of removing it?

That might be a cleaner solution, depending on how much code (apart from cmdline parsing and a few other bits) is shared between them. I do find the current code a little convoluted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment