FIO compiled with xNVMe doesn't work with the SPDK FIO plugin

[karlowich]

When compiling xNVMe we also export all of the SPDK symbols as can be seen with:
objdump --syms /usr/local/lib/x86_64-linux-gnu/libxnvme.so.0 | grep spdk

Thus when using the SPDK FIO plugin with FIO compiled with xNVMe all of the symbols are there twice.

Errors look something like:

[2024-04-30 17:41:25.056817] rpc.c: 223:spdk_rpc_register_method: *ERROR*: duplicate RPC spdk_get_version registered...
[2024-04-30 17:41:25.056919] rpc.c: 223:spdk_rpc_register_method: *ERROR*: duplicate RPC rpc_get_methods registered...
[2024-04-30 17:41:25.056933] rpc.c: 223:spdk_rpc_register_method: *ERROR*: duplicate RPC trace_get_info registered...
[2024-04-30 17:41:25.056944] rpc.c: 223:spdk_rpc_register_method: *ERROR*: duplicate RPC trace_get_tpoint_group_mask registered...
[2024-04-30 17:41:25.056953] rpc.c: 223:spdk_rpc_register_method: *ERROR*: duplicate RPC trace_disable_tpoint_group registered...
[2024-04-30 17:41:25.056963] rpc.c: 223:spdk_rpc_register_method: *ERROR*: duplicate RPC trace_enable_tpoint_group registered...
[2024-04-30 17:41:25.056973] rpc.c: 223:spdk_rpc_register_method: *ERROR*: duplicate RPC trace_clear_tpoint_mask registered...
[2024-04-30 17:41:25.056982] rpc.c: 223:spdk_rpc_register_method: *ERROR*: duplicate RPC trace_set_tpoint_mask registered...
[2024-04-30 17:41:25.056991] rpc.c: 223:spdk_rpc_register_method: *ERROR*: duplicate RPC sock_set_default_impl registered...
[2024-04-30 17:41:25.057001] rpc.c: 223:spdk_rpc_register_method: *ERROR*: duplicate RPC sock_impl_set_options registered...
[2024-04-30 17:41:25.057011] rpc.c: 223:spdk_rpc_register_method: *ERROR*: duplicate RPC sock_impl_get_options registered...
EAL: RTE_RING tailq is already registered
PANIC in tailqinitfn_rte_ring_tailq():
Cannot initialize tailq: RTE_RING```

[karlowich]

@NaddiNadja This is our problem on FreeBSD right now

[karlowich]

The solution seems to be either using LD version scripts or perhaps something like -export-symbols-regex '^xnvme_.*' Read more...

[karlowich]

The meson FAQ has a section about this, they recommend using version scripts.

[karlowich]

Since #474 has been merged, this is now resolved