New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate how to port toro to firecracker #273

Open
MatiasVara opened this Issue Dec 11, 2018 · 7 comments

Comments

Projects
None yet
1 participant
@MatiasVara
Copy link
Owner

MatiasVara commented Dec 11, 2018

Firecracker is techno to build microVMs. It is based on KVM. It provides a simple device model based on virtio. The idea of this ticket is to investigate the changes needed to boot toro by using firecracker.

@MatiasVara MatiasVara self-assigned this Dec 11, 2018

@MatiasVara

This comment has been minimized.

Copy link
Owner

MatiasVara commented Dec 13, 2018

I could make toro boots. However, it crashes when try to get the apic id. I think the problem may be 1) the apic_base has changed or 2) memory region is not mapped. I think problem is 2).

@MatiasVara

This comment has been minimized.

Copy link
Owner

MatiasVara commented Dec 13, 2018

To turn off firecracker, reboot by using the keyboard.

@MatiasVara

This comment has been minimized.

Copy link
Owner

MatiasVara commented Dec 13, 2018

Work in progress in branch pocfor273.

@MatiasVara

This comment has been minimized.

Copy link
Owner

MatiasVara commented Dec 17, 2018

I will stop to work on this until I can understand better how firecracker works. I observed that GetApicId() does not work. It makes firecraker to crash. The reason might be that region of memory is not mapped. I could't verify that.

@MatiasVara

This comment has been minimized.

Copy link
Owner

MatiasVara commented Jan 18, 2019

Some extra info:

Here are the things about firecracker which are not documented very clearly and I have learned the hard way:
firecracker implements virtio but based on virtio-mmio devices model so there is no PCI; in other words instead of PCI devices we have the mmio devices (read here - https://www.kraxel.org/virtio/virtio-v1.0-cs03-virtio-gpu.html#x1-1080002)
there is no ACPI which means there is no MADT table to parse vcpu information (firecracker provides this information through MP table)
OSv implements pre-finalized virtio spec and firecracker implements the finalized one which means that there are some subtle differences around "legacy interface, devices" (see here - https://www.kraxel.org/virtio/virtio-v1.0-cs03-virtio-gpu.html#x1-60001); this has some small implications on how both virtio-net and virtio-blk should be implemented.

@MatiasVara

This comment has been minimized.

Copy link
Owner

MatiasVara commented Jan 18, 2019

I would suggest to do Firecracker support in the following (largely independent) steps:

  • Add support for modern virtio interfaces. Linux has a split driver base model here, which we could also follow. I.e., have separate VirtioModern and VirtioLegacy base classes that encapsulate the differences.
  • Add support for VirtIO MMIO. Most of this is restructuring the drivers to use transport-independent interfaces and then have VirtioPCI and VirtioMMIO classes that implement transport-specific logic.
  • Add Intel mptable (used by Firecracker) support as an alternative to ACPI hardware discovery.
  • Add any misc Firecraker related things that are still needed to make OSv boot and run.
    Btw, this can be also tested with QEMU by passing the "disable-modern=off" and "disable-legacy=on" options to the "-device" option that defines the virtio devices.
@MatiasVara

This comment has been minimized.

Copy link
Owner

MatiasVara commented Jan 18, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment