GPU Support

[amrragab8080]

Public facing api doesnt seem to have support yet for passthrough pci devices, namely gpu is this technically feasible?

[goswamig]

That would be super helpful to have support for GPU.

[acatangiu]

GPU support in Firecracker is very hard/tricky at the moment. With current GPU hardware, there's two major problems:

1. To do device pass-through implies pinning physical memory which would remove our memory oversubscription capabilities.
2. We can only run 1 customer workload securely per physical GPU, and switching between customer workloads takes a long enough time to make it impractical.

As a result there is no known path to supporting GPUs in Firecracker.

[raduweiss]

@amrragab8080, we'll be looking at this as part of #1179.

[normtown]

To do device pass-through implies pinning physical memory which would remove our memory oversubscription capabilities.

Why do you want to maintain the ability to oversubscribe memory?

[raduweiss]

Oversubscription is a core part of makes Firecracker a great way to isolate serverless workloads; that's why we took on a tenet around it [1].

[1] https://github.com/firecracker-microvm/firecracker/blob/master/CHARTER.md

[normtown]

Why is over-subscription a great way to isolate serverless workloads? I genuinely don't know, so the reasoning that led to the existence of the tenet is not self-evident to me.

[raduweiss]

Like all services, serverless compute providers want to keep their servers busy and to improve their overall utilization. Ideally, every CPU cycle on the service provider's servers is running user code, and every byte of RAM is filled with user data. If servers are sitting idle, that’s inefficient.

A part of solving this optimization problem is having the ability to oversubscribe a given server's hardware capacity with workloads who's hardware resource usage is statistically uncorrelated, or, even better, with workloads selected specifically to pack well together.