VMDK support on Cloud Hypervisor

[fidencio]

EROFS could take advantage of an image format that supports split files, which is currently not the case for QCOW2.

VMDK specs: https://web.archive.org/web/20240527050638if_/https://www.vmware.com/app/vmdk/?src=vmdk
EROFS context: https://lore.kernel.org/linux-erofs/20250612073103.2538455-2-hsiangkao@linux.alibaba.com/

[hsiangkao]

Yeah, basically EROFS supports a merged block device feature, which can generate a metadata index to reference multiple layer blobs and concatenate them into a single block device.
For example:

The generated VMDK descriptor file (used to merge split raw files):

The QEMU option:
-drive file=~/erofs-utils/disk.vmdk,format=vmdk,if=virtio \
QEMU guest mount result:

[hsiangkao]

Libkrun already has limited VMDK support using imago:
containers/libkrun@3d74d23

I think cloud hypervisor can adapt imago too at least for VMDK support, or even switch Qcow2 support to imago too.

[ChengyuZhu6]

I'd like to take it.

[ChengyuZhu6]

I've been looking into this. Here's what I'm working for an initial implementation — would appreciate feedback.

For a first pass, I'd target read-only support for Flat VMDK only (monolithicFlat and twoGbMaxExtentFlat). These are the simplest sub-formats: a text descriptor file that points to one or more linear raw data extents. No grain tables, no compression, no delta chains. Sparse and streamOptimized would be future work.

For the VMDK parsing itself, I'm considering two options:

1. Use imago, a Rust disk image library. It already handles Flat VMDK descriptor parsing and extent mapping. The mapping lookup is pure in-memory arithmetic, and the underlying I/O is plain preadv/pwritev.
2. Write a self-contained Flat VMDK descriptor parser directly in the block crate without any new dependency.

I'm leaning toward option 1 so we don't have to deal with VMDK descriptor details ourselves, and if imago adds Sparse or streamOptimized support later, we'd get that mostly for free. I'm also willing to contribute the missing sub-format support upstream to imago, or implement it on the CLH side if needed. But happy to go with option 2 if you'd prefer to keep external dependencies minimal.

[rbradford]

We already have our own block image layer in block/ that handles QCOW2, VDX and raw files. I won't accept an external dependency to add support for just one type. If imago can replace our entire existing block subsystem that might be acceptable.

[ChengyuZhu6]

We already have our own block image layer in block/ that handles QCOW2, VDX and raw files. I won't accept an external dependency to add support for just one type. If imago can replace our entire existing block subsystem that might be acceptable.

Makes sense. I'll go with Option 2 then to keep things dependency-free. I'll implement Flat VMDK parser directly in block/, following the existing patterns. I'll stick to read-only support for the first PR to keep it manageable.

[weltling]

I took a look at imago v0.2.2 to answer @rbradford's question about whether it could replace the existing block subsystem.

It can't. imago only supports RAW, QCOW2, and VMDK, no VHD, no VHDX. The I/O architecture is also incompatible - AsyncIo trait in CH works with libc::iovec and supports io_uring and Linux AIO, while imago is built on tokio async with its own Storage abstraction. Adopting it would mean losing io_uring/AIO for any imago backed format, adding a tokio dependency in the block path, and writing a substantial adapter layer between the two trait systems. It seems not worth it and too complex.

VMDK in imago seems also readonly (and would need more insight into the fromat to understand more). Option 2 - a native flat VMDK parser in block seems to make more sense. No new dependencies, actual data I/O just passes through to existing raw backends.

Thanks

[russell-islam]

We already have our own block image layer in block/ that handles QCOW2, VDX and raw files. I won't accept an external dependency to add support for just one type. If imago can replace our entire existing block subsystem that might be acceptable.

To replace our entire block system with a new external crate we need to have some strong use cases and a reason like performance/security improvement.