RX interrupt support in the DPDK ENA PMD

[mhawari]

Hi,

I noted that the functions rx_queue_intr_enable and rx_queue_intr_disable were not implemented in ena_dev_ops. This means that the ENA PMD can't be used in interrupt mode, contrary to other PMDs. Are there any plan to implement RX interrupt support in the near future?

Thanks and best regards
Mohammed

[I-gor-C]

Hi Mohammed,

Let me check this and get back to you

[I-gor-C]

@mhawari Interrupts are usually not used with PMDs because of DPDK's general concept to use polling mode. Could you please provide some details about your usecase?

[mhawari]

Thanks for the quick answer @I-gor-C ! Here are some details. Interrupt mode for ENA is extremely useful to the Calico-VPP project [1]. It relies on VPP [2] to provide a user-space, fast data plane to Calico. When deployed in AWS, VPP have access to the underlying ENA either by using DPDK or AF_XDP, in polling or interrupt mode. DPDK provides better performance than AF_XDP for a lot of use cases, because it supports TSO, while AF_XDP does not. Interrupt mode support with DPDK would considerably improve resource-efficiency in a lot of scenarios, as we wouldn't need to dedicate CPU cores to busy polling anymore. This topic is covered with more details here [3]

[1] https://github.com/projectcalico/vpp-dataplane
[2] https://fd.io/vppproject/vpptech/
[3] https://fosdem.org/2021/schedule/event/sdn_calicovpp/

[mhawari]

CCing @mdr78

[jtollet]

Hello, Is there any update on this one ?

[I-gor-C]

Hi @jtollet
We're still checking the feasibility of this. I'll update in a few days.

[I-gor-C]

We're planning to deliver this feature to the DPDK 21.05 release.
I'll update if the timeline shifts.

[jtollet]

Hello @I-gor-C, I'd be interested to test it ahead of time if you have a pre-release. Feel free to ask if that makes sense to you.

[jtollet]

We're planning to deliver this feature to the DPDK 21.05 release.
I'll update if the timeline shifts.

Hello @I-gor-C, can you confirm it will be part of DPDK 21.05 ?

[cdtomkins]

@I-gor-C Can we get an update on whether this will be in DPDK 21.05, please?

[I-gor-C]

Hi @cdtomkins
Unfortunately, it won't make it to 21.05 and is pushed to 21.08. We can share the prerelease version if you would like to try it in the meanwhile.

[cdtomkins]

Thanks @I-gor-C, @jtollet will probably want that.

[I-gor-C]

Please contact me directly at igorch@amazon.com

[jtollet]

Definitely helpful. We’ll give it a try and will let you know ! De : Chris Tomkins ***@***.***> Répondre à : amzn/amzn-drivers ***@***.***> Date : jeudi 6 mai 2021 à 15:07 À : amzn/amzn-drivers ***@***.***> Cc : Jerome Tollet ***@***.***>, Mention ***@***.***> Objet : Re: [amzn/amzn-drivers] RX interrupt support in the DPDK ENA PMD (#168) Thanks @I-gor-C<https://github.com/I-gor-C>, @jtollet<https://github.com/jtollet> will probably want that. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#168 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AGBK27WDSEAPQBWFBY3H553TMKIAZANCNFSM4YQ2YCQQ>.

[jtollet]

Hello, I can confirm that according to our first test, Interrupt mode now works fine. Thanks a lot!

[AloysAugustin]

Hi @I-gor-C , thanks for sharing the patch privately with us. We've been using it for a bit now without issues in EKS.

We'd like to include it in the next Calico VPP dataplane release, but to do so, we'd need to make it public in the VPP gerrit (so that we can patch the DPDK version we use, while the patch is not upstreamed in DPDK). Would you have any objection to that?

[ndagan]

Hi @AloysAugustin,
Yes, you are good to go to post it in VPP gerrit. We plan to release it for DPDK v21.08.

[AloysAugustin]

Hello, it looks like some changes in dpdk 21.05 cause the patch that was provided not to apply anymore, and I didn't see an equivalent patch merged in the dpdk main branch. Would it be possible for you to share an updated version of the patch that would apply to dpdk 21.05 or to the main branch ?
Thanks,
Aloys

[krawczyk-michal]

Hi @AloysAugustin,

patch with Rx interrupts was just pushed for the community review and if there won't be objections, it will be merged for the DPDK v21.08: https://patches.dpdk.org/project/dpdk/patch/20210713154118.32111-5-mk@semihalf.com/.

The patch was modified a bit since the last version you got (especially the error handling path for the admin interrupt), but the core functionality should remain the same.

You can try to cherry-pick it from there. It should be applicable to the DPDK v21.05 as well.

[onong]

Hi @semihalf-krawczyk-michal,

Just wanted to let you know that we have tested the patch with dpdk 21.05 in calico/vpp and it seems to be working fine. Look forward to the patch being merged in 21.08.

Thanks,
Onong

[krawczyk-michal]

Hi @onong,

Thanks a lot for your feedback!

The another good news is that this feature was already merged last week and can be found in tag v21.08-rc2 and newer, so it will be part of the DPDK 21.08 release.

Thanks,
Michal

[onong]

Thats great news, thanks much! @semihalf-krawczyk-michal

[shaibran]

@onong @mhawari, fix was merged to DPDK 21.08