nvmf_tgt still uses 99% of cpu when start with —interrupted-mode

[a13547]

Sighting report

Follwing documentation, compiling and then runs nvmf_tgt —interrput-mode, set scheduler to “dynamic”, still observing 99% cpu usage via top.

Expected Behavior

Much lower cpu usage in interrupt mode

Current Behavior

99% cpu usage

Possible Solution

Have no idea, maybe more detailed documentation on interrupt mode support.

Steps to Reproduce

nvmf_tgt —interrput-mode
rpc.py framework_set_scheduler dynamic

Context (Environment including OS version, SPDK version, etc.)

OS: RHEL 9.3
SPDK Version: github master branch

[changpe1]

@a13547 SPDK nvmf_tgt doesn't support interrupt mode, only a few applications in SPDK can support interrupt mode, such as vhost-blk target and vfio-user target.

[a13547]

@a13547 SPDK nvmf_tgt doesn't support interrupt mode, only a few applications in SPDK can support interrupt mode, such as vhost-blk target and vfio-user target.

Oh I see. Thanks! Btw is there any roadmap on adding interrupt mode support to nvme-of target?

[changpe1]

@a13547 SPDK nvmf_tgt doesn't support interrupt mode, only a few applications in SPDK can support interrupt mode, such as vhost-blk target and vfio-user target.

Oh I see. Thanks! Btw is there any roadmap on adding interrupt mode support to nvme-of target?

There are patches on the gerrit to add interrupt support to NVMeoF TCP transport, but still under review and problem solving.

[tomzawadzki]

@a13547 As @changpe1 has mentioned, the support for interrupt mode is limited to specific components. If an application uses one that does not support it, the full usage of CPU cores is expected. There is an example application that only links with such components examples/interrupt_tgt/.

More importantly for NVMe-oF TCP there is currently outstanding series https://review.spdk.io/gerrit/c/spdk/spdk/+/22746/. Please note that this does not cover all NVMe-oF transports.

Since what you've observed is expected behaviour, this GH issue is more of a question. Would you be okay with closing it?

[a13547]

@tomzawadzki Thanks a lot! It's okay to close this issue. Btw, is there any plan to add interrupt mode support to NVMe-of RDMA transport in the near future?

[tomzawadzki]

@a13547 I have not seen any patches with such contributions, but they are always welcome.