SPDK v21.04: ZNS NVMe bdev, PMR, ADQ initiator, RPM

accel

Two new functions for computing crc32 have been added to the accel library. The spdk_accel_submit_crc32cv and spdk_accel_batch_prep_crc32cv describe a series of buffers as a vector, allowing for chained CRC32 computation. As with all accel interfaces, if no hardware accelerator is available the computation will automatically be performed on the CPU (ISA-L).

bdev

For the bdev_ocssd_create RPC, the optional parameter range was removed. Only one OCSSD bdev can now be created for an OCSSD namespace.

Removed the spdk_bdev_open from bdev library API. Use spdk_bdev_open_ext instead.
Removed the spdk_vbdev_register and spdk_bdev_part_base_construct from bdev module API.
Removed the config_text function for bdev modules to report legacy config file sections. Everything now uses JSON configuration.

Added the spdk_bdev_get_max_active_zones API to display maximum number active zones of a given bdev.

Added the spdk_bdev_get_max_zone_append_size API to display maximum zone append data transfer size.

bdev_nvme

Added support for zoned namespaces.

blobstore

Removed the spdk_bdev_create_bs_dev_from_desc and the spdk_bdev_create_bs_dev API.

env

Added the spdk_pci_device_allow API to allow applications to add PCI addresses to the allowed list after the application has started.

Removed the pci_whitelist, pci_blacklist and master_core members of struct spdk_env_opts.

Added hotplug support based on uevent in pci_event.c. This functionality was present inside the nvme driver directly before, but the infrastructure is now in the env layer and can be used by multiple libraries. Currently it has been enabled in the nvme, virtio-scsi and virtio_blk libraries. Please see the new API spdk_pci_event_listen, spdk_pci_get_event, spdk_pci_register_error_handler and spdk_pci_unregister_error_handler.

event

Removed the config_file, max_delay_us, pci_whitelist and pci_blacklist members of struct spdk_app_opts.

idxd

A new API spdk_idxd_device_needs_rebalance was added so that users of the library can know whether they need to rebalance the flow control for the channel that was just added/removed. This is based on how the low level library shares devices amongst channels.

The API spdk_idxd_reconfigure_chan had the num_channels removed as this is now tracked in the library. The app makes use the new API above to determine whether to rebalance or not. This applies to spdk_idxd_configure_chan as well.

The API spdk_idxd_put_channel now returns the rebalance state for the underlying device.

iscsi

A security vulnerability has been identified and fixed in the SPDK iSCSI target. A TEXT PDU with no data, but CONTINUE flag set, would result in a NULL pointer dereference and crash the SPDK iSCSI target process. All users of the SPDK iSCSI target are recommended to update. All SPDK versions <= v21.01 are affected.

net

The net library is deprecated and will be removed in the 21.07 release.

nvme

Support for persistent memory region (PMR) has been added. This includes the new functions spdk_nvme_ctrlr_get_regs_pmrcap to get the persistent memory region capabilities, as well as spdk_nvme_ctrlr_get_pmrsz, spdk_nvme_ctrlr_enable_pmr, spdk_nvme_ctrlr_disable_pmr, spdk_nvme_ctrlr_map_pmr and spdk_nvme_ctrlr_unmap_pmr. Further, added transport operations to enable, disable, map and unmap the PMR for transports that support it (currently, only PCIe).

Added support for registering outside hardware accelerators with an spdk_nvme_poll_group. This can be used to provide hardware acceleration operations for the NVMe driver to consume. This currently supports crc32c offload for computing header and data digests in NVMe-oF/TCP.

Added the spdk_nvme_qpair_get_optimal_poll_group API and qpair_get_optimal_poll_group
function pointer in spdk_nvmf_transport_ops structure in order to allow users to query which poll group the qpair should be optimally placed into.

Added OPTPERF and namespace optimal performance fields to nvme_spec.h.

Added spdk_nvme_set_hotplug_filter API to allow applications to choose which hot-inserted SSDs should be probed. This is useful for use cases where multiple independent SPDK processes are running on one node. The filter function can then be implemented in these processes to decide which SSDs to probe based on the new SSD's PCI address.

New functions spdk_nvme_poll_group_get_stats and spdk_nvme_poll_group_free_stats
were added. These functions allow to get transport statistics per NVME poll group.

Added spdk_nvme_map_cmd API to map the NVMe command with SGL cases.

Added support for vector variants of ZNS zone append commands with new APIs spdk_nvme_zns_zone_appendv and spdk_nvme_zns_zone_appendv_with_md.

Added spdk_nvme_zns_ns_get_max_open_zones and spdk_nvme_zns_ns_get_max_active_zones API, to display maximum number of open and active zones of the given namespace.

Added spdk_nvme_zns_ns_get_zone_size_sectors API to provide size of zone in number of sectors.

Added spdk_nvme_qpair_get_id API to display the ID of the specified qpair.

nvmf

Removed the spdk_nvmf_tgt_listen and spdk_nvmf_subsystem_add_ns APIs.

Added new APIs:

• spdk_nvmf_poll_group_dump_stat (function in nvmf.h).
• poll_group_dump_stat (transport op in nvmf_transport.h).

The following APIs have been deprecated and will be removed in SPDK 21.07:

• spdk_nvmf_poll_group_get_stat (function in nvmf.h),
• spdk_nvmf_transport_poll_group_get_stat (function in nvmf.h),
• spdk_nvmf_transport_poll_group_free_stat(function in nvmf.h),
• spdk_nvmf_rdma_device_stat (struct in nvmf.h),
• spdk_nvmf_transport_poll_group_stat (struct in nvmf.h),
• poll_group_get_stat (transport op in nvmf_transport.h),
• poll_group_free_stat (transport op in nvmf_transport.h).

See header files for details.

The trtype field in JSON returned by nvmf_get_stats RPC contains now the name of the transport,
which is the same as the type for defined transports and more informative for a custom transport.

Added hdgst and ddgst parameters to bdev_nvme_attach_controller RPC in order change
state of TCP header and data digest.

Added num_cqe parameter to nvmf_create_transport RPC to set number of completion queues (CQ)
for RDMA transport. Useful when CQ resize operation is not supported, for example iWARP.

ocf

Updated OCF submodule to v20.12.2

Added bdev_ocf_set_cache_mode RPC to dynamically switch cache mode of OCF bdev.

opal

Removed the spdk_opal_supported API.

raid

For bdev_raid_create RPC, the deprecated parameter strip_size was removed.

rpc

New RPC bdev_nvme_get_transport_statistics was added, it allows to get transport statistics
of nvme poll groups.

Parameter enable-zerocopy-send of RPC sock_impl_set_options is deprecated and will be removed in SPDK 21.07,
use enable-zerocopy-send-server or enable-zerocopy-send-client instead.
Parameter disable-zerocopy-send of RPC sock_impl_set_options is deprecated and will be removed in SPDK 21.07,
use disable-zerocopy-send-server or disable-zerocopy-send-client instead.

rpm

Added support for a new RPM spec, rpmbuild/spdk.spec, which can be used for packaging the
SPDK. The pkg/spdk.spec is considered to be deprecated and scheduled for removal in SPDK 21.07.
See RPM documentation for more details.

sock

The option enable_placement_id in struct spdk_sock_impl_opts has been changed from a bool to an enum and now holds the placement mode. We can use an RPC to configure different modes for placing connections into groups based on what the underlying hardware and software features. There are currently four placement modes supported - NONE, NAPI, INCOMING_CPU, and MARK.

The NONE placement mode means that any connection can be placed into any group. NAPI mode means that connections will be grouped by their NAPI id, which corresponds to their hardware queue. Similarly, INCOMING_CPU means connections will be grouped by the SO_INCOMING_CPU value reported by the socket. Finally, MARK mode allows connections to be placed into any group, but will mark the connection using the socket option SO_MARK which, on some NICs, will force all sockets with the same marked value to use the same NIC hardware queue. These are all strategies to avoid interrupt, locking, and message passing overhead within the OS TCP/IP stacks. Not all modes are supported by all OS versions or NICs.

New parameters enable_zerocopy_send_server and enable_zerocopy_send_client were added to struct spdk_sock_impl_opts, these parameters enable or disable zero copy send for server and client sockets which are created using spdk_sock_listen and spdk_sock_listen_ext (server); spdk_sock_connect and spdk_sock_connect_ext (client) functions. Existing parameter enable_zerocopy_send enables or disables zero copy send for both server and client sockets.

thread

A new API spdk_io_channel_get_io_device was added to get the io_device for the specified I/O channel.

Added spdk_thread_set_interrupt_mode function in order to set present spdk_thread into interrupt mode or back to poll mode. It is valid only when thread interrupt facility is enabled by spdk_interrupt_mode_enable.

Added spdk_poller_register_interrupt function to mark that the poller is capable of entering interrupt mode. Callback function will be called when the poller must transition into or out of interrupt mode.

virtio

Added the bdev_virtio_blk_set_hotplug RPC for the virtio blk pci device.