vhost-user devices not available on Z

[ghost]

Description of problem

Qemu on s390x does not support vhost-user devices. We need to find a workaround to bypass this limitation.

any thoughts?

/cc @markdryan @rbradford @mcastelino @WeiZhang555 @jcvenegas @amshinde @devimc @jodh-intel

[jodh-intel]

First question: why doesn't it support this feature?

[borntraeger]

Several reasons:
the s390 machine uses the ccw bus instead of the pci bus to drive virtio. So we would need
vhost-user-ccw backends, which are not wired up. This would be a smaller change, but there is another issue: vhost-user requires shared memory between qemu and another process. Right now shared memory (such as -object memory-backend-file or hugetlbfs) is not supported either with QEMU/kvm on s390x.

[jodh-intel]

Hi @borntraeger - thanks for the information. Is adding support for memory-backend-file planned?

[borntraeger]

I had memory-backend-file not on my radar. The object itself can be created, but it cannot be used to back the guest as of today. FWIW, hugetlbfs support is also not available today (but in the works)
We would need to implement the numa support first to be able to attach a memory-backend-file object to a memory node.
So I do not expect this support to be ready soon, it needs someone working on that.

[jodh-intel]

/cc @markdryan, @rbradford.

Related: https://github.com/intel/govmm.

[alicefr]

@jodh-intel would it be possible to avoid VhostUserEndpointType? We can avoid to use huge page by avoiding to set the #enable_hugepages = true in the configuration.toml. It isn't complete clear to me in which case the vhost-user devices are required. Do they need explicitly to be configured in the devices.json ? Thanks for your help

[jodh-intel]

/cc @amshinde, @egernst, @mcastelino.

[jcvenegas]

@ydjainopensource @alicefr the VhostUserEndpointType endpoint today I think is only used for dpdk use cases. If that is the case the VhostUserEndpointType should not be used at all. @egernst is that correct?

[amshinde]

@alicefr You will hit VhostUserEndpointType code path, only if you are using a CNM/CNI network plugin that explicitly passes a vhost-user endpoint. You do not need to do any other configuration. So, we can simply list this as a limitation on s390x.

[egernst]

Yes, @jcvenegas.

[jcvenegas]

agree with @amshinde it should be documented as a limitation.

[ghost]

Sounds sane to me. We will document this as a limitation.

[alicefr]

thanks everybody! Yes, we can introduce this limitation on s390x and well document it. @ydjainopensource you could also introduce an error message in the function appendVhostUserDevice that explicitly uses vhost-user devices and it can be reimplemented for each architectures.

[grahamwhaley]

Just for ref then, Kata has a limitations doc over at https://github.com/kata-containers/documentation/blob/master/Limitations.md
Right now I don't believe there is any arch specific info in there - but, that doesn't mean you can't be the first to add some ;-) I don't know if it makes sense to then add architecture specific sections to that doc (I suspect maybe), or just blend the limitations into the rest of the text.

[alicefr]

@jodh-intel I guess we can close this issue. An error message is thrown if the vhost-user devices are used on s390x (in govmm also). See #990 and kata-containers/govmm#68

[jodh-intel]

@alicefr - ack. closing...