vhost-device-console: Add initial implementation

[TimosAmpel]

Summary of the PR

This PR introduces an initial implementation of vhost-device-console device.
The development of this device was performed by Virtual Open Systems in the context
of the Automotive Grade Linux SDV-EG (Software Defined Vehicle expert group)
AGL Native VIRTIO project. Part of this development has already been upstreamed
(see AGL Gerrit console patch).

The new device creates a console device which communicates with vhost-user-console
frontend driver and eventually with virtio-console. Since there was no upstream
implementation of any vhost-user-console frontend in QEMU, the current version has
only been tested with virtio-loopback design.

Virtio-loopback architecture aims to create a new HAL (hardware abstraction layer),
designed for non-Hypervisor environments based on virtio. The main objective
is to enable virtio drivers to communicate with vhost-user devices without emulation
or virtualization being involved. This technology has also been upstreamed
into AGL gerrit repositories.

More information about virtio-loopback can be found in the links below:

• Virtio-loopback design document
• How to test the technology

Since we believe that this device might be interesting to the community,
this PR targets to share this work, ask for comments and potentially
merge the implementation under vhost-device repository.

Requirements

• All commits in this PR are signed (with git commit -s), and the commit
  message has max 60 characters for the summary and max 75 characters for each
  description line.
• All added/changed functionality has a corresponding unit/integration
  test.
• All added/changed public-facing functionality has entries in the "Upcoming
  Release" section of CHANGELOG.md (if no such section exists, please create one).
• Any newly added unsafe code is properly documented.

[stefano-garzarella]

Hi @TimosAmpel , thanks for this PR!

I'll review next week, but in the meantime I'd like to understand better virtio-loopback.
Is it similar to VDUSE?
Is there any RFC or patch posted upstream in the Linux kernel?

That said, QEMU 8.2 should support a generic vhost-user device, so maybe you give it a try to test this device using something like this: (@stsquad and @epilys knows better than me)

-device vhost-user-device-pci,virtio-id=41,num_vqs=2,config_size=8

[TimosAmpel]

Hi @TimosAmpel , thanks for this PR!

I'll review next week, but in the meantime I'd like to understand better virtio-loopback. Is it similar to VDUSE? Is there any RFC or patch posted upstream in the Linux kernel?

That said, QEMU 8.2 should support a generic vhost-user device, so maybe you give it a try to test this device using something like this: (@stsquad and @epilys knows better than me)

-device vhost-user-device-pci,virtio-id=41,num_vqs=2,config_size=8

Hi @stefano-garzarella, thanks for your response.

Virtio-loopback is indeed similar to VDUSE and the main differences between
them are mainly:

1. the data sharing mechanism: virtio-loopback implements a zero-copy memory
   mapping mechanism, the data are directly accessible by the user-space.
   VDUSE is based on a bouncing buffer mechanism which does not implement
   the zero-copy principle.
2. virtio-loopback is not implemented on top of vDPA, aiming for a simpler
   solution.

In addition, virtio-loopback supports the following rust-vmm devices:
vhost-device-gpio, input, rng, sound.

An RFC presenting the architecture (no patches included) had been submitted to
the upstream virtio-dev mailing list, please check the following link.
The upstream implementation of the driver and adapter (user-space component)
can be found in the AGL repositories link1 and link2 correspondingly.

Thank you a lot for the "vhost-user-device-pci" suggestion, I plan to test both
vhost-device-console and vhost-device-can devices with it and apply any changes
if needed.

[TimosAmpel]

Hello @epilys and @stefano-garzarella,

I have pushed a new simplified version of the device
with all your comments integrated.

Vhost-device-console is being tested with QEMU's "vhost-user-device" as suggested:

qemu-system-x86_64  \
        -m 4096 \
        -numa node,memdev=mem \
        -object memory-backend-file,id=mem,size=4G,mem-path=/dev/shm,share=on \
        -chardev socket,id=char1,path=/tmp/console.sock \
        -device vhost-user-device-pci,chardev=char1,virtio-id=3,num_vqs=4,config_size=12 \

Also, I have created a new QEMU vhost-user-console front-end based on
vhost-user-device which can be found in this repository.

More about, known limination, test examples and future work can be found in
the README file.

[TimosAmpel]

Hello @epilys,

I have pushed a newer version which covers the scenario of the device working with multiple QEMU's guest
and two different modes (backends: network console, nested console).

As before, the readme file has been updated and registers any known limitation, new features, testing examples etc.

[TimosAmpel]

Hello @epilys,

The latest version addresses all your comments, also adds more tests which goes through most of the cases,
and passes all the CI tests, expect the test coverage one. That should be the last open point which I plan to
eliminate next days if needed. Please let me know your opinion when you have time, thanks in advance!

[stefano-garzarella]

@TimosAmpel if this PR is not a Draft, I suggest cleaning a bit the commits and squash them. We usually prefer to merge a PR with commits that do not modify code added in the same PR.

[TimosAmpel]

@TimosAmpel if this PR is not a Draft, I suggest cleaning a bit the commits and squash them. We usually prefer to merge a PR with commits that do not modify code added in the same PR.

Thanks @stefano-garzarella! I have squashed them all in one new presenation commit, I hope that's more suitable.