Skip to content

Releases: spdk/spdk

SPDK v20.01.1: Maintenance LTS Release

20 Mar 19:34
Compare
Choose a tag to compare

SPDK 20.01.1 is a bug fix and maintenance LTS release.

GitHub issues

  • #960 #1170: lib/blob: queue up blob persists when one already is ongoing
  • #1184: rdma: Correct handling of RDMA_CM_EVENT_DEVICE_REMOVAL
  • #1192: nvmf/rpc: Destroy subsystem if spdk_rpc_nvmf_create_subsystem fails
  • #1206: env_dpdk: dont treat NULL as error in spdk_map_bar_rte()
  • #1223: test/nvme: fix correct controllers name in nvme-cli cuse test
  • #1228: nvme: set transport string before the probe based on transport type
  • #1232: module/crypto: increase the number of queue pairs for AESNI_MB
  • #1262: dpdk: move submodule to commit 3fcb1dd
  • #1273: bdev/nvme: do not destruct ctrlr if reset is in progress
  • #1278: lib/iscsi: Return when connection state is already exited at login completion

SPDK v20.01: Optimized thin provisioning, FTL bdev, VMD hot plug, FUSED support

31 Jan 09:55
Compare
Choose a tag to compare

bdev

A new function, spdk_bdev_set_timeout, has been added to set per descriptor I/O timeouts.

A new class of functions spdk_bdev_compare*, have been added to allow native bdev support
of block comparisons and compare-and-write.

A new class of bdev events, SPDK_BDEV_EVENT_MEDIA_MANAGEMENT, has been added to allow bdevs
which expose raw media to alert all I/O channels of pending media management events.

A new API was added spdk_bdev_io_get_aux_buf allowing the caller to request
an auxiliary buffer for its own private use. The API is used in the same manner that
spdk_bdev_io_get_buf is used and the length of the buffer is always the same as the
bdev_io primary buffer. 'spdk_bdev_io_put_aux_buf' frees the allocated auxiliary
buffer.

blobfs

Added boolean return value for function spdk_fs_set_cache_size to indicate its operation result.

Added blobfs_set_cache_size RPC method to set cache size for blobstore filesystem.

blobstore

Added new use_extent_table option to spdk_blob_opts for creating blobs with Extent Table descriptor.
Using this metadata format, dramatically decreases number of writes required to persist each cluster allocation
for thin provisioned blobs. Extent Table descriptor is enabled by default.
See the Blobstore Programmer's Guide for more details.

dpdk

Updated DPDK submodule to DPDK 19.11.

env_dpdk

spdk_env_dpdk_post_init now takes a boolean, legacy_mem, as an argument.

A new function, spdk_env_dpdk_dump_mem_stats, prints information about the memory consumed by DPDK to a file specified by
the user. A new utility, scripts/dpdk_mem_info.py, wraps this function and prints the output in an easy to read way.

event

The functions spdk_reactor_enable_framework_monitor_context_switch() and
spdk_reactor_framework_monitor_context_switch_enabled() have been changed to
spdk_framework_enable_context_switch_monitor() and
spdk_framework_context_switch_monitor_enabled(), respectively.

ftl

All NVMe dependencies were removed from ftl library.
From now ftl library is using bdev_zone API.
bdev_ftl becomes virtual bdev.

ctrlr and trid fields in spdk_ftl_dev_init_opts structure were replaced by
base_bdev_desc.

bdev_ftl_create RPC method trtype and traddr parameters were replaced by base_bdev
parameter.

spdk_ftl_module_init and spdk_ftl_module_fini functions were removed.

spdk_ftl_punit_range and ftl_module_init_opts structures were removed.

isa-l

Updated ISA-L submodule to commit f3993f5c0b6911 which includes implementation and
optimization for aarch64.

Enabled ISA-L on aarch64 by default in addition to x86.

nvme

delayed_pcie_doorbell parameter in spdk_nvme_io_qpair_opts was renamed to delay_cmd_submit
to allow reuse in other transports.

Added RDMA WR batching to NVMf RDMA initiator. Send and receive WRs are chained together
and posted with a single call to ibv_post_send(receive) in the next call to qpair completion
processing function. Batching is controlled by 'delay_cmd_submit' qpair option.

The NVMe-oF initiator now supports plugging out of tree NVMe-oF transports. In order
to facilitate this feature, several small API changes have been made:

The spdk_nvme_transport_id struct now contains a trstring member used to identify the transport.
A new function, spdk_nvme_transport_available_by_name, has been added.
A function table, spdk_nvme_transport_ops, and macro, SPDK_NVME_TRANSPORT_REGISTER, have been added which
enable registering out of tree transports.

A new function, spdk_nvme_ns_supports_compare, allows a user to check whether a given namespace supports the compare
operation.

A new family of functions, spdk_nvme_ns_compare*, give the user access to submitting compare commands to NVMe namespaces.

A new function, spdk_nvme_ctrlr_cmd_get_log_page_ext, gives users more granular control over the command dwords sent in
log page requests.

nvmf

Add SockPriority option in [Transport] section, this can be used for NVMe-oF target
on TCP transport to set sock priority for the incomming TCP connections.

The NVMe-oF target now supports plugging out of tree NVMe-oF transports. In order
to facilitate this feature, several small API changes have been made:

The spdk_nvme_transport_id struct now contains a trstring member used to identify the transport.
spdk_nvmf_tgt_get_transport, spdk_nvmf_transport_opts_init, and spdk_nvmf_transport_create all have been
modified to take a string.
A function table, spdk_nvmf_transport_ops, and macro, SPDK_NVMF_TRANSPORT_REGISTER, have been added which
enable registering out of tree transports.

Add spdk_nvmf_tgt_stop_listen() that can be used to stop listening for
incoming connections for specified target and trid. Listener is not stopped
implicitly upon destruction of a subsystem any more.

A custom NVMe admin command handler has been added which allows the user to use the real drive
attributes from one of the target NVMe drives when reporting drive attributes to the initiator.
This handler can be enabled via the nvmf_set_config RPC.
Note: In a future version of SPDK, this handler will be enabled by default.

The SPDK target and initiator both now include compare-and-write functionality with one caveat. If using the RDMA transport,
the target expects the initiator to send both the compare command and write command either with, or without inline data. The
SPDK initiator currently respects this requirement, but this note is included as a flag for other initiators attempting
compatibility with this version of SPDK.

rpc

A new RPC, bdev_zone_block_create, enables creating an emulated zoned bdev on top of a standard block device.

A new RPC, bdev_ocssd_create, enables creating an emulated zoned bdev on top of an Open Channel SSD.

A new RPC, blobfs_set_cache_size, enables managing blobfs cache size.

A new RPC, env_dpdk_get_mem_stats, has been added to facilitate reading DPDK related memory
consumption stats. Please see the env_dpdk section above for more details.

A new RPC, framework_get_reactors, has been added to retrieve a list of all reactors.

bdev_ftl_create now takes a base_bdev argument in lieu of trtype, traddr, and punits.

bdev_nvme_set_options now allows users to disable I/O submission batching with the -d flag

bdev_nvme_cuse_register now accepts a name parameter.

bdev_uring_create now takes arguments for bdev_name and block_size

nvmf_set_config now takes an argument to enable passthru of identify commands to base NVMe devices.
Please see the nvmf section above for more details.

scsi

spdk_scsi_lun_get_dif_ctx now takes an additional argument of type spdk_scsi_task.

sock

Added spdk_sock_writev_async for performing asynchronous writes to sockets. This call will
never return EAGAIN, instead queueing internally until the data has all been sent. This can
simplify many code flows that create pollers to continue attempting to flush writes
on sockets.

Added impl_name parameter in spdk_sock_listen and spdk_sock_connect functions. Users may now
specify the sock layer implementation they'd prefer to use. Valid implementations are currently
"vpp" and "posix" and NULL, where NULL results in the previous behavior of the functions.

thread

spdk_thread_send_msg now returns int indicating if the message was successfully
sent.

A new function spdk_thread_send_critical_msg, has been added to support sending a single message from
a context that may be interrupted, e.g. a signal handler.

Two new functions, spdk_poller_pause, and spdk_poller_resume, have been added to give greater control
of pollers to the application owner.

util

spdk_pipe, a new utility for buffering data from sockets or files for parsing
has been added. The public API is available at include/spdk/pipe.h.

bdev

Added spdk_bdev_io_get_nvme_fused_status function for translating bdev_io status to NVMe status
code for fused compare-and-write operation.

Added spdk_bdev_get_acwu function for getting block device atomic compare and write unit size.

SPDK v19.10.1: Maintenance Release

02 Jan 15:27
Compare
Choose a tag to compare

SPDK 19.10.1 is a bug fix and maintenance release.

thread

spdk_thread_send_msg now returns int indicating if the message was successfully
sent.

dpdk

DPDK submodule updated to include fix for vhost CVE-2019-14818.

GitHub issues

  • #811: thread: return int from spdk_thread_seng_msg
  • #1029: nvme: don't enable adminq until we know discovery_ctrlr exists
  • #1032: lib/nvme: cuse device avoid using signals
  • #1033: lib/nvme: stop all NVMe io producers on detach
  • #1063: nvme: don't monitor hotplug events in secondary process
  • #1075: rdma: Add synchronization for LAST_WQE_REACHED event
  • #1086: rdma: Fix incoming_queue cleanup when RDMA qpair is destroyed
  • #1089: lib/nvmf: Accept KATO for discovery controller
  • #1110: nvmf/rdma: make disconnect qp from cm event safe.

SPDK v19.10: Zoned bdev API, Opal bdev, NVMe character devices

31 Oct 21:34
Compare
Choose a tag to compare

rpc

Many of SPDK's RPCs were renamed to be more consistent and intuitive in this release.
The old names will continue to function, but will display a deprecation warning.

Added optional parameters '--arbitration-burst' and '--low/medium/high-priority-weight' to
'bdev_nvme_set_options' RPC method.

Added optional parameter '--md-size' to 'construct_null_bdev' RPC method.

Added optional parameters '--dif-type' and '--dif-is-head-of-md' to 'construct_null_bdev'
RPC method.

Added blobfs_detect RPC method to detect whether a blobfs exists on given bdev.

Added blobfs_create RPC method to build blobfs on given bdev.

Added blobfs_mount RPC method to mount blobfs on given bdev to a host path by FUSE.
Then on the host path, user can directly do some file operations which will be mapped
to blobfs.

bdev

Added new parameter cdw0 to spdk_bdev_io_complete_nvme_status() and
spdk_bdev_io_get_nvme_status() that allows setting/getting
the NVMe completion queue DW0 entry. This allows vendor specific IO commands
to return commmand specific completion info back to the initiator.

Added spdk_bdev_get_write_unit_size() function for retrieving required number
of logical blocks for write operation.

New zone-related fields were added to the result of the get_bdevs RPC call:

  • zoned: indicates whether the device is zoned or a regular
    block device
  • zone_size: number of blocks in a single zone
  • max_open_zones: maximum number of open zones
  • optimal_open_zones: optimal number of open zones
    The zoned field is a boolean and is always present, while the rest is only available for zoned
    bdevs.

A new spdk_bdev_open_ext function has been added and spdk_bdev_open function has been deprecated.
The new open function introduces requirement to provide callback function that will be called by
asynchronous event such as bdev removal. spdk_bdev_open_ext function takes bdev name as
an argument instead of bdev structure to avoid a race condition that can happen when the bdev
is being removed between a call to get its structure based on a name and actually openning it.

New 'resize' event has been added to notify about change of block count property of block device.
Event is delivered only if block device was opened with spdk_bdev_open_ext function.

bdev zone

Added new public header for zoned bdev. Zoned bdev is an extension
of the bdev interface.

spdk_bdev_get_zone_size(), spdk_bdev_get_max_open_zones(), spdk_bdev_get_optimal_open_zones()
APIs were added for retrieving zoned device information.
spdk_bdev_get_zone_info() API was added for retrieving information about zones in zoned
device.
Added spdk_bdev_zone_management() API for changing zone state.
spdk_bdev_zone_append() and spdk_bdev_zone_append_with_md() APIs were added for
appending data to a zone.
Added spdk_bdev_io_get_append location() function for retrieving append location for I/O.
Added spdk_bdev_is_zoned() function for checking if bdev supports zoned namespace semantics.

bdev opal

EXPERIMENTAL: A new opal bdev has been added to support management of
NVMe self-encrypting drives through the Opal specification. Users can
create opal bdevs from an NVMe namespace bdev, if the controller
containing that namespace supports Opal. Currently this is only
supported for namespace ID=1. The following RPCs have been added to
support Opal: bdev_nvme_opal_init, bdev_nvme_opal_revert,
bdev_opal_create, bdev_opal_delete, bdev_opal_get_info,
bdev_opal_new_user, bdev_opal_set_lock_state.
It does not yet support recreating the opal bdevs after application restart.
This bdev module should be considered very experimental, and the RPCs may
change significantly in future releases.

delay bdev

The bdev_delay_update_latency has been added to allow users to update
a latency value for a given delay bdev.

compress bdev

A new RPC bdev_compress_get_orphans has been added to list compress bdevs
that were not loaded due to a missing pm metadata file. In this state they
can only be deleted.

null bdev

Metadata support has been added to Null bdev module.

Protection information support has been added to Null bdev module.

nvme

Added no_shn_notification to NVMe controller initialization options, users can enable
it for NVMe controllers. When the option is enabled, the controller will not do the
shutdown process and just disable the controller, users can start their application
later again to initialize the controller to the ready state.

A controller flag SPDK_NVME_CTRLR_WRR_SUPPORTED was added to indicate the controller
can support weighted round robin arbitration feature with submission queue.

Added arbitration_burst option for arbitration feature, and added three
low/medium/high_priority_weight options for weighted round robin arbitration.

Added spdk_nvme_ns_cmd_write_uncorrectable.

Added new error handling and reporting functionality. This includes several
new API functions to facilitate applications recovering when a qpair or
controller fails.

spdk_nvme_ctrlr_reconnect_io_qpair - Reconnects a failed I/O qpair.
spdk_nvme_ctrlr_set_trid - Sets the trid of an existing controller. Can be used to
change the trid for failover cases.
spdk_nvme_ctrlr_is_failed - Returns the failed state of a controller.
spdk_nvme_ctrlr_fail - Forces a controller into a failed state.

Modified the return behavior of several API functions to better indicate to
applications when a qpair is failed. This list of functions includes:

spdk_nvme_qpair_process_completions
spdk_nvme_ns_cmd_*
spdk_nvme_ctrlr_process_admin_completions
spdk_nvme_ctrlr_cmd_*

These functions now return -ENXIO when the qpair or controller on which they
operate is failed.

EXPERIMENTAL: Added NVMe character device support to allow to create NVMe device nodes in Linux
kernel for controller as well as for namespace and process ioctl requests as usual
from linux environment.

nvmf

The spdk_nvmf_tgt_create function now accepts an object of type spdk_nvmf_target_opts
as its only parameter. This new structure contains the max_subsystems parameter previously
passed into that function.

A new public API function spdk_nvmf_get_tgt has been added which allows users to
retrieve a pointer to an spdk_nvmf_tgt object by supplying its name. In the special
case where an RPC or application only creates a single target, this function can accept
a null name parameter and will return the only available target.

The majority of the NVMe-oF RPCs now accept an optional tgt_name parameter. This will
allow those RPCs to work with applications that create more than one target.

Three new NVMe-oF RPCs have been added nvmf_create_target, nvmf_delete_target, and
nvmf_get_targets. These new RPCs provide a basic interface for managing multiple target
objects. In SPDK the target object defines a unique discovery service. As of this release,
these RPCs are not intended to be used with the in-tree SPDK target applications, spdk_tgt and
nvmf_tgt, which use a single, global target structure. As such, they are not included in scripts/rpc.py

Three new header functions have also been added to help deal with multiple targets.
spdk_nvmf_tgt_get_name takes a target pointer as an argument and returns its human readable name.
spdk_nvmf_get_first_target takes no arguments and returns the first target in the global list.
spdk_nvmf_get_next_tgt takes a target pointer as an argument and returns the next one in the global list.

The spdk_nvmf_tgt_accept takes additional argument allowing to pass arbitrary context
information to the new_qpair callback. This will simplify the code when having multiple
nvmf targets or when retrieving the context information from globals is not suitable.

blobstore

A new spdk_bdev_create_bs_dev_from_desc function has been added and spdk_bdev_create_bs_dev
function has been deprecated.
The new create function can cowork with spdk_bdev_open_ext function, which provides callback
function that will be called by asynchronous event such as bdev removal.

blobfs_bdev

A new blobfs module bdev has been added to simplify the operations of blobfs on bdev.

Function spdk_blobfs_bdev_detect is added to detect whether blobfs exists on the given block device.

Function spdk_blobfs_bdev_create is added to create a blobfs on the given block device.

Function spdk_blobfs_bdev_mount is added to mount a blobfs on the given block device to
a host path by FUSE. Then, a new thread is created dedicatedly for one mountpoint to handle
FUSE request by blobfs API.

build

Option to build FUSE components into blobfs_bdev module for mounting a blobfs filesystem.
It requires the installation of libfuse3. By default, it is disabled. And it will be
enabled if run ./configure with --with-fuse option.

iSCSI

Portals may no longer be associated with a cpumask. The scheduling of
connections is moving to a more dynamic model.

An new RPC iscsi_portal_group_set_auth has been added to set CHAP authentication
for discovery sessions specific for the existing iSCSI portal group. This RPC overwrites
the setting by the global parameters for the iSCSI portal group.

socket

Added spdk_sock_is_connected to check whether the socket is currently connected.
spdk_sock_group_poll now returns number of events on success.

env

Added spdk_pci_device_unclaim() function to cleanup pci claim file.

event

framework_start_init RPC no longer stops the application on error during
initialization.

DPDK

Updated DPDK submodule to DPDK 19.08.

ocf

Updated OCF submodule to OCF v19.06

Along with update, new cache mode 'write only' was added.

New cache modes added to use via RPC, wi - write invalidate and wa - write aroun...

SPDK v19.07.1: Maintenance Release

30 Sep 06:01
Compare
Choose a tag to compare

SPDK 19.07.1 is a bug fix and maintenance release.

build

Improved inter-lib dependencies to allow linking against only a subset of SPDK libraries when building SPDK applications.

GitHub issues

  • #892: rpc: ensure RPCs are registered before aliases
  • #898: opal: Fix get string for bigger length
  • #903: nvmf/rdma: Handle completions for destroyed QP associated with SRQ
  • #915: env_dpdk/memory: aggregate adjacent vfio mappings
  • #884: bdev/raid: Fix race issue among multiple threads to free RAID bdev
  • #897: bdev/compress: support host buffers that cross a 2MB boundary
  • #928: module/compress: Clear vol element in comp_bdev struct on vol unload

SPDK v19.07: NVMe-oF FC Transport, VMD, NVMe-oF Persistent reservations, Bdev I/O with separate metadata

31 Jul 08:10
Compare
Choose a tag to compare

ftl

EXPERIMENTAL: Added ability to mirror writes to persistent write buffer cache
to allow for recovery from dirty shutdown event.

Added handling of Asynchronous Nand Management Events (ANM).

vmd

EXPERIMENTAL: Added Intel Volume Management Device (VMD) driver. VMD is an integrated
controller inside the CPU PCIe root complex. It enables virtual HBAs for the connected
NVMe SSDs. spdk_vmd_init() enumerates NVMe SSDs behind VMD device and hook them into
SPDK PCI subsystem. spdk_nvme_probe() or spdk_nvme_connect() can be used to connect
NVMe driver to the device located at the given transport ID.

To obtain transport ID of NVMe SSD behind VMD spdk_lspci can be used.

Current implementation does not support hotplug.

blobfs

Blobfs file asynchronous operations were added to public APIs.

util

A new file API spdk_posix_file_load was added to load file content into a data buffer.

New APIs spdk_dif_ctx_set_data_offset, spdk_dif_verify_stream,
spdk_dif_get_range_with_md, spdk_dif_get_length_with_md have been added,
and existing APIs spdk_dif_ctx_init, spdk_dif_set_md_interleave_iovs, and
spdk_dif_generate_stream have been refined to insert or strip DIF by stream
fasion with any alignment.

New APIs spdk_dif_ctx_set_remapped_init_ref_tag, spdk_dif_remap_ref_tag,
and spdk_dix_remap_ref_tag have been added to remap DIF reference tag.

New APIs spdk_dif_update_crc32c and spdk_dif_update_crc32c_stream have been
added to compute CRC-32C checksum for extended LBA payload.

Bdevperf and bdevio applications now support starting tests with application specific
RPCs. Please see helper Python scripts in their respective directories.
This is a move towards simpler RPC-only configuration for all main
and auxiliary applications.

Legacy INI style configuration for SPDK applications will become deprecated in SPDK 19.10,
and removed in SPDK 20.01. Please consider moving to JSON-RPC configuration files and/or
RPC driven run-time configuration.

nvmf

EXPERIMENTAL: A Fibre Channel transport that supports Broadcom HBAs has been
added. This depends on the FC HBA driver at
https://github.com/ecdufcdrvr/bcmufctdrvr. See the documentation
for more information.

Persistent reservation emulation has been added to the NVMe-oF target. Persistent reservation
state is stored in a JSON file on the local filesystem between target restart. To support this,
an optional parameter to the RPC method nvmf_subsystem_add_ns called --ptpl-file was added.
This allows the user to specify which file to store the persistent reservation state in. Note
that this is done per namespace.

The c2h success optimization under which a command capsule response is not sent
for reads is turned on by default. A config knob was added to allow disabling
the optimization. This will mostly be used for integration testing with 5.0.x kernels
while some compatibility fixes make their way down the pipeline for 5.1.x kernels.

The sock priority setting of the TCP connection owned by the tcp transport is added. It is
used to optimize the TCP connection performance under designated traffic classes. And the
priority is used to differeniate the sock priority between SPDK NVMe-oF TCP target application
and other TCP based applications.

Shared receive queue can now be disabled even for NICs that support it using the
nvmf_create_transport RPC method parameter no_srq. The actual use of a shared
receive queue is predicated on hardware support when this flag is not used.

spdk_nvmf_get_optimal_poll_group was added, which is used to return the optimal
poll group for the qpair. And ConnectionScheduler configuration is added into the
[Nvmf] section in etc/spdk/nvmf.conf.in to demonstrate how to configure the connection
scheduling strategy among different spdk threads.

Added infrastructure to retrieve global and per poll group NVMf statistics.

DIF strip and insert is now supported for TCP transport. When it is enabled, DIF
setting is not exposed to the NVMe-oF initiator, and DIF is attached into data
for write I/O and stripped from data for read I/O.

Added a field dif_insert_or_strip to struct spdk_nvmf_transport_opts, and
updated the related rpc function nvmf_create_transport to make this
configurable parameter available to users. The dif_insert_or_strip is relevant
for TCP transport for now and used to configure the DIF strip and insert.

Added infrastructure to retrieve NVMf transport statistics.

notify

The function spdk_notify_get_types() and spdk_notify_get_events() were
renamed to spdk_notify_foreach_type() and spdk_notify_foreach_event(),
respectively. And update type name of callback accordingly.

bdev

The format of the data returned by the get_bdevs_iostat RPC has changed to
make it easier to parse. It now returns an object with a "ticks" object
and "bdevs" array with the per-bdev statistics.

A new bdev module delay has been added which simulates a drive latency when placed
on top of a Null bdev. This module is intended only for testing and can be created using
the new RPC bdev_delay_create. That RPC takes the name of the underlying bdev as well
as average and p99 latency arguments for both read and write operations. Average latency is
defined as a value close to what you would expect a perf tool such as FIO to report back as
the mean latency of all I/O submitted to the drive. p99 latency is defined as the value one
would expect the drive to see the slowest 1% of I/O report. For underlying drives with already
significant latency, the latency values provided to the drive will be additive. This should be
taken into account if trying to achieve an artificial latency on top of an nvme drive or aio device.

DIF reference tag remapping is now supported for partition type virtual bdev
modules. When using partition type virtual bdevs, block address space is
remapped during I/O processing and DIF reference tag is remapped accordingly.

Added spdk_bdev_*_with_md() functions allowing for IO with metadata being transferred in
separate buffer. To check support for separatate metadata, use spdk_bdev_is_md_separate().

All bdevs now have a UUID. For devices whose backing hardware does not provide a UUID,
one is automatically generated. Across runs of SPDK, bdevs whose UUID is automatically
generated may change.

A new virtual bdev module compress has been added to provide compression services on top of
a thinly provisioned logical volume. See documentation for complete details.

nvme

Added an optional parameter --io-queue-requests to RPC set_bdev_nvme_options, which
can be used to change the number of requests allocated for one NVMe I/O queue. For
very big I/O size, e.g. 128MiB, with this option user will not get an error due to
limited requests in NVMe driver layer.

Added spdk_nvme_ctrlr_get_transport_id() to get the transport ID from a
previously attached controller.

Nvme Opal library spdk_opal_cmd deprecated. Adding seperate command APIs.
NVMe Opal library add support for activating locking SP which will make the transaction
from "Manufactured-Inactive" state to "Manufactured" state. Upon successfully invoking
of this method, lock and unlock features will be enabled.
NVMe Opal library add support for locking/unlocking range and list locking range info.
NVMe opal library add support for multiuser. Admin can enable user and add user to specific
locking range and the user can lock/unlock his range.

Added spdk_nvme_ctrlr_io_cmd_raw_no_payload_build() allowing a caller to pass
a completely formed command to an NVMe submission queue (buffer addresses and all).
This is supported on the PCIe transport only.

Added spdk_nvme_get_ctrlr_registers() to return a pointer to the virtual address
of the NVMe controller registers. This is supported on the PCIe transport only.

Added additional options to the spdk_nvme_ctrlr_alloc_qpair() option parameter
structure to allow caller to override the virtual and optionally physical address
of the submission and completion queue pair to be created. This is supported on
the PCIe transport only.

Added disable_error_logging to struct spdk_nvme_ctrlr_opts, which disables
logging of failed requests. By default logging is enabled.

Added spdk_nvme_qpair_print_command(), spdk_nvme_qpair_print_completion() and
spdk_nvme_cpl_get_status_string(). Allowing for easier display of error messages.

Added support for NVMe Sanitize command.

env

The parameter free_space has been added to spdk_ring_enqueue() to wait when
the ring is almost full and resume when there is enough space available in
the ring.

A new API spdk_mempool_lookup has been added to lookup the memory pool created
by the primary process.

Added spdk_pci_get_first_device() and spdk_pci_get_next_device() to allow
iterating over PCI devices detected by SPDK. Because of this, all SPDK APIs
to attach/detach PCI devices are no longer thread safe. They are now meant to
be called from only a single thread only, the same only that called spdk_env_init().
This applies to the newly added APIs as well.

vpp

SPDK now supports VPP version 19.04.2, up from VPP 18.01.

VPP socket abstraction now uses VPP Session API, instead of VLC (VPP Communications Library).
This allows for better control over sessions and queues.
Please see VPP documentation for more details:
VPP Host Stack

sock

Add spdk_sock_get_optimal_sock_group(), which returns the optimal sock group for
this socket. When a socket is created, it is often assigned to a sock group using
spdk_sock_group_add_sock so that a set of sockets can be polled more efficiently.
For some network devices, it is optimal to assign particular sockets to specific
sock groups. This API is inten...

SPDK v19.04.1: Maintenance Release

19 Jun 10:48
Compare
Choose a tag to compare

SPDK 19.04.1 is a bug fix and maintenance release.

NVMe-oF

Increased default maximum number of queue pairs to 128 in order to match Linux kernel target. Users can still decrease this default when creating the transport (i.e. -p option for nvmf_create_transport in rpc.py).

Shared receive queue can now be disabled even for NICs that support it using the nvmf_create_transport RPC method parameter no_srq. The actual use of a shared receive queue is predicated on hardware support when this flag is not used.

iSCSI

Fixed hang on high QD large reads.

DPDK

Added DPDK 19.05 support.

raid bdev

Fixed race when issuing multiple destroy RPC for the same raid bdevs.

GitHub issues

  • #798: rpc: fix segfault on get_spdk_version request with extra params
  • #789: test/json_config: wait for clean config
  • #279: blobfs: ensure length xattr is written even if all data is flushed
  • #777: test/raid: switch from config file to RPC
  • #781: build: add full version to SONAME for shared libraries
  • #801: test/lvol: wait for bdevs to be examined

v19.04: Compression bdev, Notification library, NVMe Opal support

30 Apr 20:47
Compare
Choose a tag to compare

nvme

Added asynchronous probe support. New APIs spdk_nvme_probe_async(),
spdk_nvme_connect_async() and spdk_nvme_probe_poll_async() were added to enable
this feature. spdk_nvme_probe_async() and spdk_nvme_connect_async() return a
context associated with the specified controllers. Users then call
spdk_nvme_probe_poll_async() until it returns 0, indicating that the operation
completed.

A new qpair creation option, delay_pcie_doorbell, was added. This can be passed
to spdk_nvme_alloc_io_qpair(). This makes the I/O submission functions, such as
spdk_nvme_ns_writev(), skip ringing the submission queue doorbell. Instead the
doorbell will be rung as necessary inside spdk_nvme_qpair_process_completions().
This can result in significantly fewer MMIO writes to the doorbell register
under heavy load, greatly improving performance.

spdk_nvme_ctrlr_get_regs_cmbsz() was added to report the size of the controller
memory buffer, if available.

spdk_nvme_ctrlr_get_flags() was added to return controller feature
flags. Two flags are currently tracked:
SPDK_NVME_CTRLR_SGL_SUPPORTED
SPDK_NVME_CTRLR_SECURITY_SEND_RECV_SUPPORTED

The NVMe hotplug poller is now able to detach devices hot-removed from the system
via /sys/bus/pci/devices/<bdf>/remove and /sys/bus/pci/devices/<bdf>/driver/unbind.

Opal support was added for scan, take ownership, revert TPer, and dumping device
info. The nvme_manage tool can be used to perform these operations. The public
API functions are spdk_nvme_ctrlr_security_receive() and
spdk_nvme_ctrlr_security_send(). This module should be considered experimental
pending additional features and tests.

raid

Added new strip_size_kb rpc param on create to replace the more ambiguous
strip_size. The strip_size rpc param is deprecated.

thread

Added spdk_thread_has_pollers() function to verify if there are any registered
pollers to be run on the thread. Added spdk_thread_is_idle() function to check
if there are any scheduled operations to be performed on the thread at given
time.

spdk_thread_create() now takes an optional CPU affinity mask that is passed to
the scheduler.

spdk_thread_lib_init() now takes an optional context size. For each thread
created, an additional region of memory of this size will be allocated. A
pointer to this region of memory can be obtained by calling
spdk_thread_get_ctx(). The inverse operation is also available via
spdk_thread_get_from_ctx().

spdk_thread_poll() now optionally accepts the current time, in ticks. This can
be used to avoid some calls to spdk_get_ticks() internally.

spdk_app_start() now only accepts a single context argument.

bdev

An new API spdk_bdev_get_data_block_size has been added to get size of data
block except for metadata.

spdk_vbdev_register() has been deprecated. spdk_bdev_register() should be used
instead.

A mechanism for acquiring and releasing data buffers from bdev modules, used
to perform zero copy operations, was added.

New APIs spdk_bdev_get_md_size(), spdk_bdev_is_md_interleaved(), spdk_bdev_get_dif_type(),
spdk_bdev_is_dif_head_of_md(), and spdk_bdev_is_dif_check_enabled() have been
added to get metadata and DIF settings.

Bdevs claimed by the examine_config callback will be now further examined in the
examine_disk callback.

spdk_bdev_io_get_io_channel() was added as a convenient way to get an io_channel
from a bdev_io.

NVMe-oF Target

Support for per-device shared receive queues in the RDMA transport has been added.
It is enabled by default for any device that supports it.

The size of a shared receive queue is defined by transport configuration file parameter
MaxSRQDepth and nvmf_create_transport RPC method parameter max_srq_depth.
Default size is 4096.

Add model number as parameter to construct_nvmf_subsystem (-d option),
rather than using hardcoded define.

DIF passthrough feature has been added. DIF setting of the allocated bdevs is
exposed to the NVMe-oF initiator and data with DIF from the NVMe-oF initiator is
passed through to the allocated bdevs.

env

The phys_addr parameter in spdk_malloc() and spdk_zmalloc() has been deprecated.
For retrieving physical addresses, spdk_vtophys() should be used instead.

spdk_realloc() has been added to reallocate DMA/shared memory.

spdk_pci_device_is_removed() has been added to let the upper-layer SPDK drivers know
that device has a pending external hotremove request.

spdk_env_fini() and spdk_env_dpdk_post_fini() were added to release any resources
allocated by spdk_env_init() or spdk_env_dpdk_post_init() respectively. It is expected
that common usage of those functions is to call them just before terminating the process.

Added spdk_iommu_is_enabled() to report if SPDK application is using IOMMU for DMA.

DPDK

Dropped support for DPDK 17.07 and earlier, which SPDK won't even compile with right now.

Updated DPDK submodule to DPDK 19.02.

rpc

New get_spdk_version RPC method is introduced to get version info of the running SPDK application.

The start_nbd_disk RPC method now take nbd_device as an optional parameter. If nbd_device
is specified, use that specified nbd device. If it's not specified, pick available one.

iSCSI target

DIF strip and insert is now supported. DIF settings are not exposed to the iSCSI initiator.
DIF is attached into data for write I/O and stripped from data for read I/O.

vhost

Added experimental support for running with the external, upstream rte_vhost library.
This can be enabled by configuring SPDK with an --without-internal-vhost-lib flag.
The minimum supported rte_vhost version (DPDK version) is 19.05-rc1.

As a result of fuzz testing, a lot of data races in vhost-scsi LUN hotplug path were identified and
fixed. Those data races could have potentially resulted in SPDK crashes, RPC hangs, or memory leaks
if Vhost-SCSI LUN hotplug RPCs were executed while connected VMs were in the middle of restarting.

The SCSI target id in add_vhost_scsi_lun RPC is now optional. If -1 is passed, the first
unoccupied target id will be used.

AIO

AIO bdev module can now reap I/O completions directly from userspace, significantly improving
the overall performance.

blobfs

Synchronous IO operations no longer use spdk_io_channel, but instead use
spdk_fs_thread_ctx. The behavior is otherwise identical.

OCF

Added support for caching multiple bdevs using a single bdev as a cache.

notify

Added the notify library that provides a high performance local event bus
between libraries. Example usage was added to bdev module, which reports
notifications for added and removed bdevs via RPC.

sock

Added new API spdk_sock_readv() to the sock library for performing vectored
reads.

event

The function spdk_subsystem_init() no longer requires spdk_event as an argument.

Changed API of spdk_subsystem_config_json() to no longer be asynchronous.

io_uring

A bdev module that issues I/O to kernel block devices using the new io_uring Linux kernel
API was added. This module requires liburing.

build

Options to easily compile with profile guided optimization have been added to
configure. To use profile guided optimization with SPDK, run
./configure --with-pgo-capture, build SPDK, then run a workload of your
choosing. Then, simply run ./configure --with-pgo-enable and recompile to
build using the generated profile data. Profile guided optimization can yield
very large performance improvements, especially on GCC 8 and clang 7. This may
be combined with link time optimization which has been available under the
--enable-lto configure option for several releases.

compression bdev/reduce library

Added "reduce" block compression scheme based on using SSDs for storing
compressed blocks of storage and presistent memory for metadata. Please see
compression for more details.

v18.10.2: Maintenance Release

29 Mar 21:42
Compare
Choose a tag to compare

The SPDK 18.10.x line has been updated to use DPDK 18.11, which is a long term support release for DPDK.

SPDK v19.01.1: Maintenance Release

15 Mar 09:36
Compare
Choose a tag to compare

SPDK 19.01.1 is a bug fix and maintenance release.

logical volumes

Added option to change method for erasing data region on lvol store creation.
Default of unmapping can now be changed to writing zeroes or no operation.

autotest

Introduce SPDK_RUN_FUNCTIONAL_TEST variable enabled by default, that can be
manually disabled on systems where e.g. only unit tests are run.

FTL

Add detection for OpenChannel devices, so that NVMe tests and FTL bdev tests
are run on appropriate devices.

QoS

Enabled rate limit when opening the bdev, as there were some cases where
previously closed bdev would have QoS disabled.

GitHub issues

  • #523: (fio_plugin) Fixed hang on pthread_exit(NULL).
  • #633: (crypto bdev) Fixed memory leak in driver init path.
  • #602: (nvme bdev) Will not attach user deleted controllers automatically.
  • #676: (nvme_bdev) Added SHST_COMPLETE quirk for VMWare emulated SSDs.
  • #638: (nvmf) Fixed the lvol creation failure issue for NVMe-oF TCP.
  • #699: (nvmf) Updated default number of shared buffers for RDMA.
  • #673: (vhost) Will use mmap_size to check for 2MB hugepage multiple.
  • #663: (building) Fixed duplicated clean target in shared_lib/Makefile
  • #593: (spdkcli) Will exit with 1 when rpc throws JSONRPCException.