Releases: spdk/spdk
SPDK v20.01.1: Maintenance LTS Release
SPDK 20.01.1 is a bug fix and maintenance LTS release.
GitHub issues
- #960 #1170: lib/blob: queue up blob persists when one already is ongoing
- #1184: rdma: Correct handling of RDMA_CM_EVENT_DEVICE_REMOVAL
- #1192: nvmf/rpc: Destroy subsystem if spdk_rpc_nvmf_create_subsystem fails
- #1206: env_dpdk: dont treat NULL as error in spdk_map_bar_rte()
- #1223: test/nvme: fix correct controllers name in nvme-cli cuse test
- #1228: nvme: set transport string before the probe based on transport type
- #1232: module/crypto: increase the number of queue pairs for AESNI_MB
- #1262: dpdk: move submodule to commit 3fcb1dd
- #1273: bdev/nvme: do not destruct ctrlr if reset is in progress
- #1278: lib/iscsi: Return when connection state is already exited at login completion
SPDK v20.01: Optimized thin provisioning, FTL bdev, VMD hot plug, FUSED support
bdev
A new function, spdk_bdev_set_timeout
, has been added to set per descriptor I/O timeouts.
A new class of functions spdk_bdev_compare*
, have been added to allow native bdev support
of block comparisons and compare-and-write.
A new class of bdev events, SPDK_BDEV_EVENT_MEDIA_MANAGEMENT
, has been added to allow bdevs
which expose raw media to alert all I/O channels of pending media management events.
A new API was added spdk_bdev_io_get_aux_buf
allowing the caller to request
an auxiliary buffer for its own private use. The API is used in the same manner that
spdk_bdev_io_get_buf
is used and the length of the buffer is always the same as the
bdev_io primary buffer. 'spdk_bdev_io_put_aux_buf' frees the allocated auxiliary
buffer.
blobfs
Added boolean return value for function spdk_fs_set_cache_size to indicate its operation result.
Added blobfs_set_cache_size
RPC method to set cache size for blobstore filesystem.
blobstore
Added new use_extent_table
option to spdk_blob_opts
for creating blobs with Extent Table descriptor.
Using this metadata format, dramatically decreases number of writes required to persist each cluster allocation
for thin provisioned blobs. Extent Table descriptor is enabled by default.
See the Blobstore Programmer's Guide for more details.
dpdk
Updated DPDK submodule to DPDK 19.11.
env_dpdk
spdk_env_dpdk_post_init
now takes a boolean, legacy_mem
, as an argument.
A new function, spdk_env_dpdk_dump_mem_stats
, prints information about the memory consumed by DPDK to a file specified by
the user. A new utility, scripts/dpdk_mem_info.py
, wraps this function and prints the output in an easy to read way.
event
The functions spdk_reactor_enable_framework_monitor_context_switch()
and
spdk_reactor_framework_monitor_context_switch_enabled()
have been changed to
spdk_framework_enable_context_switch_monitor()
and
spdk_framework_context_switch_monitor_enabled()
, respectively.
ftl
All NVMe dependencies were removed from ftl library.
From now ftl library is using bdev_zone API.
bdev_ftl becomes virtual bdev.
ctrlr
and trid
fields in spdk_ftl_dev_init_opts
structure were replaced by
base_bdev_desc
.
bdev_ftl_create
RPC method trtype
and traddr
parameters were replaced by base_bdev
parameter.
spdk_ftl_module_init
and spdk_ftl_module_fini
functions were removed.
spdk_ftl_punit_range
and ftl_module_init_opts
structures were removed.
isa-l
Updated ISA-L submodule to commit f3993f5c0b6911 which includes implementation and
optimization for aarch64.
Enabled ISA-L on aarch64 by default in addition to x86.
nvme
delayed_pcie_doorbell
parameter in spdk_nvme_io_qpair_opts
was renamed to delay_cmd_submit
to allow reuse in other transports.
Added RDMA WR batching to NVMf RDMA initiator. Send and receive WRs are chained together
and posted with a single call to ibv_post_send(receive) in the next call to qpair completion
processing function. Batching is controlled by 'delay_cmd_submit' qpair option.
The NVMe-oF initiator now supports plugging out of tree NVMe-oF transports. In order
to facilitate this feature, several small API changes have been made:
The spdk_nvme_transport_id
struct now contains a trstring member used to identify the transport.
A new function, spdk_nvme_transport_available_by_name
, has been added.
A function table, spdk_nvme_transport_ops
, and macro, SPDK_NVME_TRANSPORT_REGISTER
, have been added which
enable registering out of tree transports.
A new function, spdk_nvme_ns_supports_compare
, allows a user to check whether a given namespace supports the compare
operation.
A new family of functions, spdk_nvme_ns_compare*
, give the user access to submitting compare commands to NVMe namespaces.
A new function, spdk_nvme_ctrlr_cmd_get_log_page_ext
, gives users more granular control over the command dwords sent in
log page requests.
nvmf
Add SockPriority option in [Transport] section, this can be used for NVMe-oF target
on TCP transport to set sock priority for the incomming TCP connections.
The NVMe-oF target now supports plugging out of tree NVMe-oF transports. In order
to facilitate this feature, several small API changes have been made:
The spdk_nvme_transport_id
struct now contains a trstring member used to identify the transport.
spdk_nvmf_tgt_get_transport
, spdk_nvmf_transport_opts_init
, and spdk_nvmf_transport_create
all have been
modified to take a string.
A function table, spdk_nvmf_transport_ops
, and macro, SPDK_NVMF_TRANSPORT_REGISTER
, have been added which
enable registering out of tree transports.
Add spdk_nvmf_tgt_stop_listen()
that can be used to stop listening for
incoming connections for specified target and trid. Listener is not stopped
implicitly upon destruction of a subsystem any more.
A custom NVMe admin command handler has been added which allows the user to use the real drive
attributes from one of the target NVMe drives when reporting drive attributes to the initiator.
This handler can be enabled via the nvmf_set_config
RPC.
Note: In a future version of SPDK, this handler will be enabled by default.
The SPDK target and initiator both now include compare-and-write functionality with one caveat. If using the RDMA transport,
the target expects the initiator to send both the compare command and write command either with, or without inline data. The
SPDK initiator currently respects this requirement, but this note is included as a flag for other initiators attempting
compatibility with this version of SPDK.
rpc
A new RPC, bdev_zone_block_create
, enables creating an emulated zoned bdev on top of a standard block device.
A new RPC, bdev_ocssd_create
, enables creating an emulated zoned bdev on top of an Open Channel SSD.
A new RPC, blobfs_set_cache_size
, enables managing blobfs cache size.
A new RPC, env_dpdk_get_mem_stats
, has been added to facilitate reading DPDK related memory
consumption stats. Please see the env_dpdk section above for more details.
A new RPC, framework_get_reactors
, has been added to retrieve a list of all reactors.
bdev_ftl_create
now takes a base_bdev
argument in lieu of trtype
, traddr
, and punits
.
bdev_nvme_set_options
now allows users to disable I/O submission batching with the -d
flag
bdev_nvme_cuse_register
now accepts a name
parameter.
bdev_uring_create
now takes arguments for bdev_name
and block_size
nvmf_set_config
now takes an argument to enable passthru of identify commands to base NVMe devices.
Please see the nvmf section above for more details.
scsi
spdk_scsi_lun_get_dif_ctx
now takes an additional argument of type spdk_scsi_task
.
sock
Added spdk_sock_writev_async for performing asynchronous writes to sockets. This call will
never return EAGAIN, instead queueing internally until the data has all been sent. This can
simplify many code flows that create pollers to continue attempting to flush writes
on sockets.
Added impl_name
parameter in spdk_sock_listen and spdk_sock_connect functions. Users may now
specify the sock layer implementation they'd prefer to use. Valid implementations are currently
"vpp" and "posix" and NULL, where NULL results in the previous behavior of the functions.
thread
spdk_thread_send_msg
now returns int indicating if the message was successfully
sent.
A new function spdk_thread_send_critical_msg
, has been added to support sending a single message from
a context that may be interrupted, e.g. a signal handler.
Two new functions, spdk_poller_pause
, and spdk_poller_resume
, have been added to give greater control
of pollers to the application owner.
util
spdk_pipe
, a new utility for buffering data from sockets or files for parsing
has been added. The public API is available at include/spdk/pipe.h
.
bdev
Added spdk_bdev_io_get_nvme_fused_status function for translating bdev_io status to NVMe status
code for fused compare-and-write operation.
Added spdk_bdev_get_acwu function for getting block device atomic compare and write unit size.
SPDK v19.10.1: Maintenance Release
SPDK 19.10.1 is a bug fix and maintenance release.
thread
spdk_thread_send_msg
now returns int indicating if the message was successfully
sent.
dpdk
DPDK submodule updated to include fix for vhost CVE-2019-14818.
GitHub issues
- #811: thread: return int from spdk_thread_seng_msg
- #1029: nvme: don't enable adminq until we know discovery_ctrlr exists
- #1032: lib/nvme: cuse device avoid using signals
- #1033: lib/nvme: stop all NVMe io producers on detach
- #1063: nvme: don't monitor hotplug events in secondary process
- #1075: rdma: Add synchronization for LAST_WQE_REACHED event
- #1086: rdma: Fix incoming_queue cleanup when RDMA qpair is destroyed
- #1089: lib/nvmf: Accept KATO for discovery controller
- #1110: nvmf/rdma: make disconnect qp from cm event safe.
SPDK v19.10: Zoned bdev API, Opal bdev, NVMe character devices
rpc
Many of SPDK's RPCs were renamed to be more consistent and intuitive in this release.
The old names will continue to function, but will display a deprecation warning.
Added optional parameters '--arbitration-burst' and '--low/medium/high-priority-weight' to
'bdev_nvme_set_options' RPC method.
Added optional parameter '--md-size' to 'construct_null_bdev' RPC method.
Added optional parameters '--dif-type' and '--dif-is-head-of-md' to 'construct_null_bdev'
RPC method.
Added blobfs_detect
RPC method to detect whether a blobfs exists on given bdev.
Added blobfs_create
RPC method to build blobfs on given bdev.
Added blobfs_mount
RPC method to mount blobfs on given bdev to a host path by FUSE.
Then on the host path, user can directly do some file operations which will be mapped
to blobfs.
bdev
Added new parameter cdw0
to spdk_bdev_io_complete_nvme_status()
and
spdk_bdev_io_get_nvme_status()
that allows setting/getting
the NVMe completion queue DW0 entry. This allows vendor specific IO commands
to return commmand specific completion info back to the initiator.
Added spdk_bdev_get_write_unit_size()
function for retrieving required number
of logical blocks for write operation.
New zone-related fields were added to the result of the get_bdevs
RPC call:
zoned
: indicates whether the device is zoned or a regular
block devicezone_size
: number of blocks in a single zonemax_open_zones
: maximum number of open zonesoptimal_open_zones
: optimal number of open zones
Thezoned
field is a boolean and is always present, while the rest is only available for zoned
bdevs.
A new spdk_bdev_open_ext
function has been added and spdk_bdev_open
function has been deprecated.
The new open function introduces requirement to provide callback function that will be called by
asynchronous event such as bdev removal. spdk_bdev_open_ext
function takes bdev name as
an argument instead of bdev structure to avoid a race condition that can happen when the bdev
is being removed between a call to get its structure based on a name and actually openning it.
New 'resize' event has been added to notify about change of block count property of block device.
Event is delivered only if block device was opened with spdk_bdev_open_ext
function.
bdev zone
Added new public header for zoned bdev. Zoned bdev is an extension
of the bdev interface.
spdk_bdev_get_zone_size()
, spdk_bdev_get_max_open_zones()
, spdk_bdev_get_optimal_open_zones()
APIs were added for retrieving zoned device information.
spdk_bdev_get_zone_info()
API was added for retrieving information about zones in zoned
device.
Added spdk_bdev_zone_management()
API for changing zone state.
spdk_bdev_zone_append()
and spdk_bdev_zone_append_with_md()
APIs were added for
appending data to a zone.
Added spdk_bdev_io_get_append location()
function for retrieving append location for I/O.
Added spdk_bdev_is_zoned()
function for checking if bdev supports zoned namespace semantics.
bdev opal
EXPERIMENTAL: A new opal bdev has been added to support management of
NVMe self-encrypting drives through the Opal specification. Users can
create opal bdevs from an NVMe namespace bdev, if the controller
containing that namespace supports Opal. Currently this is only
supported for namespace ID=1. The following RPCs have been added to
support Opal: bdev_nvme_opal_init
, bdev_nvme_opal_revert
,
bdev_opal_create
, bdev_opal_delete
, bdev_opal_get_info
,
bdev_opal_new_user
, bdev_opal_set_lock_state
.
It does not yet support recreating the opal bdevs after application restart.
This bdev module should be considered very experimental, and the RPCs may
change significantly in future releases.
delay bdev
The bdev_delay_update_latency
has been added to allow users to update
a latency value for a given delay bdev.
compress bdev
A new RPC bdev_compress_get_orphans
has been added to list compress bdevs
that were not loaded due to a missing pm metadata file. In this state they
can only be deleted.
null bdev
Metadata support has been added to Null bdev module.
Protection information support has been added to Null bdev module.
nvme
Added no_shn_notification
to NVMe controller initialization options, users can enable
it for NVMe controllers. When the option is enabled, the controller will not do the
shutdown process and just disable the controller, users can start their application
later again to initialize the controller to the ready state.
A controller flag SPDK_NVME_CTRLR_WRR_SUPPORTED
was added to indicate the controller
can support weighted round robin arbitration feature with submission queue.
Added arbitration_burst
option for arbitration feature, and added three
low/medium/high_priority_weight
options for weighted round robin arbitration.
Added spdk_nvme_ns_cmd_write_uncorrectable
.
Added new error handling and reporting functionality. This includes several
new API functions to facilitate applications recovering when a qpair or
controller fails.
spdk_nvme_ctrlr_reconnect_io_qpair
- Reconnects a failed I/O qpair.
spdk_nvme_ctrlr_set_trid
- Sets the trid of an existing controller. Can be used to
change the trid for failover cases.
spdk_nvme_ctrlr_is_failed
- Returns the failed state of a controller.
spdk_nvme_ctrlr_fail
- Forces a controller into a failed state.
Modified the return behavior of several API functions to better indicate to
applications when a qpair is failed. This list of functions includes:
spdk_nvme_qpair_process_completions
spdk_nvme_ns_cmd_*
spdk_nvme_ctrlr_process_admin_completions
spdk_nvme_ctrlr_cmd_*
These functions now return -ENXIO when the qpair or controller on which they
operate is failed.
EXPERIMENTAL: Added NVMe character device support to allow to create NVMe device nodes in Linux
kernel for controller as well as for namespace and process ioctl requests as usual
from linux environment.
nvmf
The spdk_nvmf_tgt_create
function now accepts an object of type spdk_nvmf_target_opts
as its only parameter. This new structure contains the max_subsystems parameter previously
passed into that function.
A new public API function spdk_nvmf_get_tgt
has been added which allows users to
retrieve a pointer to an spdk_nvmf_tgt
object by supplying its name. In the special
case where an RPC or application only creates a single target, this function can accept
a null name parameter and will return the only available target.
The majority of the NVMe-oF RPCs now accept an optional tgt_name parameter. This will
allow those RPCs to work with applications that create more than one target.
Three new NVMe-oF RPCs have been added nvmf_create_target
, nvmf_delete_target
, and
nvmf_get_targets
. These new RPCs provide a basic interface for managing multiple target
objects. In SPDK the target object defines a unique discovery service. As of this release,
these RPCs are not intended to be used with the in-tree SPDK target applications, spdk_tgt and
nvmf_tgt, which use a single, global target structure. As such, they are not included in scripts/rpc.py
Three new header functions have also been added to help deal with multiple targets.
spdk_nvmf_tgt_get_name
takes a target pointer as an argument and returns its human readable name.
spdk_nvmf_get_first_target
takes no arguments and returns the first target in the global list.
spdk_nvmf_get_next_tgt
takes a target pointer as an argument and returns the next one in the global list.
The spdk_nvmf_tgt_accept
takes additional argument allowing to pass arbitrary context
information to the new_qpair
callback. This will simplify the code when having multiple
nvmf targets or when retrieving the context information from globals is not suitable.
blobstore
A new spdk_bdev_create_bs_dev_from_desc
function has been added and spdk_bdev_create_bs_dev
function has been deprecated.
The new create function can cowork with spdk_bdev_open_ext
function, which provides callback
function that will be called by asynchronous event such as bdev removal.
blobfs_bdev
A new blobfs module bdev
has been added to simplify the operations of blobfs on bdev.
Function spdk_blobfs_bdev_detect
is added to detect whether blobfs exists on the given block device.
Function spdk_blobfs_bdev_create
is added to create a blobfs on the given block device.
Function spdk_blobfs_bdev_mount
is added to mount a blobfs on the given block device to
a host path by FUSE. Then, a new thread is created dedicatedly for one mountpoint to handle
FUSE request by blobfs API.
build
Option to build FUSE components into blobfs_bdev module for mounting a blobfs filesystem.
It requires the installation of libfuse3. By default, it is disabled. And it will be
enabled if run ./configure
with --with-fuse
option.
iSCSI
Portals may no longer be associated with a cpumask. The scheduling of
connections is moving to a more dynamic model.
An new RPC iscsi_portal_group_set_auth
has been added to set CHAP authentication
for discovery sessions specific for the existing iSCSI portal group. This RPC overwrites
the setting by the global parameters for the iSCSI portal group.
socket
Added spdk_sock_is_connected
to check whether the socket is currently connected.
spdk_sock_group_poll
now returns number of events on success.
env
Added spdk_pci_device_unclaim()
function to cleanup pci claim file.
event
framework_start_init
RPC no longer stops the application on error during
initialization.
DPDK
Updated DPDK submodule to DPDK 19.08.
ocf
Updated OCF submodule to OCF v19.06
Along with update, new cache mode 'write only' was added.
New cache modes added to use via RPC, wi - write invalidate and wa - write aroun...
SPDK v19.07.1: Maintenance Release
SPDK 19.07.1 is a bug fix and maintenance release.
build
Improved inter-lib dependencies to allow linking against only a subset of SPDK libraries when building SPDK applications.
GitHub issues
- #892: rpc: ensure RPCs are registered before aliases
- #898: opal: Fix get string for bigger length
- #903: nvmf/rdma: Handle completions for destroyed QP associated with SRQ
- #915: env_dpdk/memory: aggregate adjacent vfio mappings
- #884: bdev/raid: Fix race issue among multiple threads to free RAID bdev
- #897: bdev/compress: support host buffers that cross a 2MB boundary
- #928: module/compress: Clear vol element in comp_bdev struct on vol unload
SPDK v19.07: NVMe-oF FC Transport, VMD, NVMe-oF Persistent reservations, Bdev I/O with separate metadata
ftl
EXPERIMENTAL: Added ability to mirror writes to persistent write buffer cache
to allow for recovery from dirty shutdown event.
Added handling of Asynchronous Nand Management Events (ANM).
vmd
EXPERIMENTAL: Added Intel Volume Management Device (VMD) driver. VMD is an integrated
controller inside the CPU PCIe root complex. It enables virtual HBAs for the connected
NVMe SSDs. spdk_vmd_init()
enumerates NVMe SSDs behind VMD device and hook them into
SPDK PCI subsystem. spdk_nvme_probe()
or spdk_nvme_connect()
can be used to connect
NVMe driver to the device located at the given transport ID.
To obtain transport ID of NVMe SSD behind VMD spdk_lspci
can be used.
Current implementation does not support hotplug.
blobfs
Blobfs file asynchronous operations were added to public APIs.
util
A new file API spdk_posix_file_load
was added to load file content into a data buffer.
New APIs spdk_dif_ctx_set_data_offset
, spdk_dif_verify_stream
,
spdk_dif_get_range_with_md
, spdk_dif_get_length_with_md
have been added,
and existing APIs spdk_dif_ctx_init
, spdk_dif_set_md_interleave_iovs
, and
spdk_dif_generate_stream
have been refined to insert or strip DIF by stream
fasion with any alignment.
New APIs spdk_dif_ctx_set_remapped_init_ref_tag
, spdk_dif_remap_ref_tag
,
and spdk_dix_remap_ref_tag
have been added to remap DIF reference tag.
New APIs spdk_dif_update_crc32c
and spdk_dif_update_crc32c_stream
have been
added to compute CRC-32C checksum for extended LBA payload.
Bdevperf and bdevio applications now support starting tests with application specific
RPCs. Please see helper Python scripts in their respective directories.
This is a move towards simpler RPC-only configuration for all main
and auxiliary applications.
Legacy INI style configuration for SPDK applications will become deprecated in SPDK 19.10,
and removed in SPDK 20.01. Please consider moving to JSON-RPC configuration files and/or
RPC driven run-time configuration.
nvmf
EXPERIMENTAL: A Fibre Channel transport that supports Broadcom HBAs has been
added. This depends on the FC HBA driver at
https://github.com/ecdufcdrvr/bcmufctdrvr. See the documentation
for more information.
Persistent reservation emulation has been added to the NVMe-oF target. Persistent reservation
state is stored in a JSON file on the local filesystem between target restart. To support this,
an optional parameter to the RPC method nvmf_subsystem_add_ns
called --ptpl-file
was added.
This allows the user to specify which file to store the persistent reservation state in. Note
that this is done per namespace.
The c2h success optimization under which a command capsule response is not sent
for reads is turned on by default. A config knob was added to allow disabling
the optimization. This will mostly be used for integration testing with 5.0.x kernels
while some compatibility fixes make their way down the pipeline for 5.1.x kernels.
The sock priority setting of the TCP connection owned by the tcp transport is added. It is
used to optimize the TCP connection performance under designated traffic classes. And the
priority is used to differeniate the sock priority between SPDK NVMe-oF TCP target application
and other TCP based applications.
Shared receive queue can now be disabled even for NICs that support it using the
nvmf_create_transport
RPC method parameter no_srq
. The actual use of a shared
receive queue is predicated on hardware support when this flag is not used.
spdk_nvmf_get_optimal_poll_group was added, which is used to return the optimal
poll group for the qpair. And ConnectionScheduler
configuration is added into the
[Nvmf] section in etc/spdk/nvmf.conf.in to demonstrate how to configure the connection
scheduling strategy among different spdk threads.
Added infrastructure to retrieve global and per poll group NVMf statistics.
DIF strip and insert is now supported for TCP transport. When it is enabled, DIF
setting is not exposed to the NVMe-oF initiator, and DIF is attached into data
for write I/O and stripped from data for read I/O.
Added a field dif_insert_or_strip
to struct spdk_nvmf_transport_opts, and
updated the related rpc function nvmf_create_transport to make this
configurable parameter available to users. The dif_insert_or_strip
is relevant
for TCP transport for now and used to configure the DIF strip and insert.
Added infrastructure to retrieve NVMf transport statistics.
notify
The function spdk_notify_get_types()
and spdk_notify_get_events()
were
renamed to spdk_notify_foreach_type()
and spdk_notify_foreach_event()
,
respectively. And update type name of callback accordingly.
bdev
The format of the data returned by the get_bdevs_iostat RPC has changed to
make it easier to parse. It now returns an object with a "ticks" object
and "bdevs" array with the per-bdev statistics.
A new bdev module delay
has been added which simulates a drive latency when placed
on top of a Null bdev. This module is intended only for testing and can be created using
the new RPC bdev_delay_create
. That RPC takes the name of the underlying bdev as well
as average and p99 latency arguments for both read and write operations. Average latency is
defined as a value close to what you would expect a perf tool such as FIO to report back as
the mean latency of all I/O submitted to the drive. p99 latency is defined as the value one
would expect the drive to see the slowest 1% of I/O report. For underlying drives with already
significant latency, the latency values provided to the drive will be additive. This should be
taken into account if trying to achieve an artificial latency on top of an nvme drive or aio device.
DIF reference tag remapping is now supported for partition type virtual bdev
modules. When using partition type virtual bdevs, block address space is
remapped during I/O processing and DIF reference tag is remapped accordingly.
Added spdk_bdev_*_with_md() functions allowing for IO with metadata being transferred in
separate buffer. To check support for separatate metadata, use spdk_bdev_is_md_separate().
All bdevs now have a UUID. For devices whose backing hardware does not provide a UUID,
one is automatically generated. Across runs of SPDK, bdevs whose UUID is automatically
generated may change.
A new virtual bdev module compress
has been added to provide compression services on top of
a thinly provisioned logical volume. See documentation for complete details.
nvme
Added an optional parameter --io-queue-requests
to RPC set_bdev_nvme_options
, which
can be used to change the number of requests allocated for one NVMe I/O queue. For
very big I/O size, e.g. 128MiB, with this option user will not get an error due to
limited requests in NVMe driver layer.
Added spdk_nvme_ctrlr_get_transport_id() to get the transport ID from a
previously attached controller.
Nvme Opal library spdk_opal_cmd deprecated. Adding seperate command APIs.
NVMe Opal library add support for activating locking SP which will make the transaction
from "Manufactured-Inactive" state to "Manufactured" state. Upon successfully invoking
of this method, lock and unlock features will be enabled.
NVMe Opal library add support for locking/unlocking range and list locking range info.
NVMe opal library add support for multiuser. Admin can enable user and add user to specific
locking range and the user can lock/unlock his range.
Added spdk_nvme_ctrlr_io_cmd_raw_no_payload_build() allowing a caller to pass
a completely formed command to an NVMe submission queue (buffer addresses and all).
This is supported on the PCIe transport only.
Added spdk_nvme_get_ctrlr_registers() to return a pointer to the virtual address
of the NVMe controller registers. This is supported on the PCIe transport only.
Added additional options to the spdk_nvme_ctrlr_alloc_qpair() option parameter
structure to allow caller to override the virtual and optionally physical address
of the submission and completion queue pair to be created. This is supported on
the PCIe transport only.
Added disable_error_logging
to struct spdk_nvme_ctrlr_opts, which disables
logging of failed requests. By default logging is enabled.
Added spdk_nvme_qpair_print_command(), spdk_nvme_qpair_print_completion() and
spdk_nvme_cpl_get_status_string(). Allowing for easier display of error messages.
Added support for NVMe Sanitize command.
env
The parameter free_space
has been added to spdk_ring_enqueue() to wait when
the ring is almost full and resume when there is enough space available in
the ring.
A new API spdk_mempool_lookup
has been added to lookup the memory pool created
by the primary process.
Added spdk_pci_get_first_device() and spdk_pci_get_next_device() to allow
iterating over PCI devices detected by SPDK. Because of this, all SPDK APIs
to attach/detach PCI devices are no longer thread safe. They are now meant to
be called from only a single thread only, the same only that called spdk_env_init().
This applies to the newly added APIs as well.
vpp
SPDK now supports VPP version 19.04.2, up from VPP 18.01.
VPP socket abstraction now uses VPP Session API, instead of VLC (VPP Communications Library).
This allows for better control over sessions and queues.
Please see VPP documentation for more details:
VPP Host Stack
sock
Add spdk_sock_get_optimal_sock_group(), which returns the optimal sock group for
this socket. When a socket is created, it is often assigned to a sock group using
spdk_sock_group_add_sock so that a set of sockets can be polled more efficiently.
For some network devices, it is optimal to assign particular sockets to specific
sock groups. This API is inten...
SPDK v19.04.1: Maintenance Release
SPDK 19.04.1 is a bug fix and maintenance release.
NVMe-oF
Increased default maximum number of queue pairs to 128 in order to match Linux kernel target. Users can still decrease this default when creating the transport (i.e. -p option for nvmf_create_transport in rpc.py).
Shared receive queue can now be disabled even for NICs that support it using the nvmf_create_transport
RPC method parameter no_srq
. The actual use of a shared receive queue is predicated on hardware support when this flag is not used.
iSCSI
Fixed hang on high QD large reads.
DPDK
Added DPDK 19.05 support.
raid bdev
Fixed race when issuing multiple destroy RPC for the same raid bdevs.
GitHub issues
- #798: rpc: fix segfault on get_spdk_version request with extra params
- #789: test/json_config: wait for clean config
- #279: blobfs: ensure length xattr is written even if all data is flushed
- #777: test/raid: switch from config file to RPC
- #781: build: add full version to SONAME for shared libraries
- #801: test/lvol: wait for bdevs to be examined
v19.04: Compression bdev, Notification library, NVMe Opal support
nvme
Added asynchronous probe support. New APIs spdk_nvme_probe_async(),
spdk_nvme_connect_async() and spdk_nvme_probe_poll_async() were added to enable
this feature. spdk_nvme_probe_async() and spdk_nvme_connect_async() return a
context associated with the specified controllers. Users then call
spdk_nvme_probe_poll_async() until it returns 0, indicating that the operation
completed.
A new qpair creation option, delay_pcie_doorbell, was added. This can be passed
to spdk_nvme_alloc_io_qpair(). This makes the I/O submission functions, such as
spdk_nvme_ns_writev(), skip ringing the submission queue doorbell. Instead the
doorbell will be rung as necessary inside spdk_nvme_qpair_process_completions().
This can result in significantly fewer MMIO writes to the doorbell register
under heavy load, greatly improving performance.
spdk_nvme_ctrlr_get_regs_cmbsz() was added to report the size of the controller
memory buffer, if available.
spdk_nvme_ctrlr_get_flags() was added to return controller feature
flags. Two flags are currently tracked:
SPDK_NVME_CTRLR_SGL_SUPPORTED
SPDK_NVME_CTRLR_SECURITY_SEND_RECV_SUPPORTED
The NVMe hotplug poller is now able to detach devices hot-removed from the system
via /sys/bus/pci/devices/<bdf>/remove
and /sys/bus/pci/devices/<bdf>/driver/unbind
.
Opal support was added for scan, take ownership, revert TPer, and dumping device
info. The nvme_manage tool can be used to perform these operations. The public
API functions are spdk_nvme_ctrlr_security_receive() and
spdk_nvme_ctrlr_security_send(). This module should be considered experimental
pending additional features and tests.
raid
Added new strip_size_kb rpc param on create to replace the more ambiguous
strip_size. The strip_size rpc param is deprecated.
thread
Added spdk_thread_has_pollers() function to verify if there are any registered
pollers to be run on the thread. Added spdk_thread_is_idle() function to check
if there are any scheduled operations to be performed on the thread at given
time.
spdk_thread_create() now takes an optional CPU affinity mask that is passed to
the scheduler.
spdk_thread_lib_init() now takes an optional context size. For each thread
created, an additional region of memory of this size will be allocated. A
pointer to this region of memory can be obtained by calling
spdk_thread_get_ctx(). The inverse operation is also available via
spdk_thread_get_from_ctx().
spdk_thread_poll() now optionally accepts the current time, in ticks. This can
be used to avoid some calls to spdk_get_ticks() internally.
spdk_app_start() now only accepts a single context argument.
bdev
An new API spdk_bdev_get_data_block_size
has been added to get size of data
block except for metadata.
spdk_vbdev_register() has been deprecated. spdk_bdev_register() should be used
instead.
A mechanism for acquiring and releasing data buffers from bdev modules, used
to perform zero copy operations, was added.
New APIs spdk_bdev_get_md_size(), spdk_bdev_is_md_interleaved(), spdk_bdev_get_dif_type(),
spdk_bdev_is_dif_head_of_md(), and spdk_bdev_is_dif_check_enabled() have been
added to get metadata and DIF settings.
Bdevs claimed by the examine_config
callback will be now further examined in the
examine_disk
callback.
spdk_bdev_io_get_io_channel() was added as a convenient way to get an io_channel
from a bdev_io.
NVMe-oF Target
Support for per-device shared receive queues in the RDMA transport has been added.
It is enabled by default for any device that supports it.
The size of a shared receive queue is defined by transport configuration file parameter
MaxSRQDepth
and nvmf_create_transport
RPC method parameter max_srq_depth
.
Default size is 4096.
Add model number as parameter to construct_nvmf_subsystem (-d option),
rather than using hardcoded define.
DIF passthrough feature has been added. DIF setting of the allocated bdevs is
exposed to the NVMe-oF initiator and data with DIF from the NVMe-oF initiator is
passed through to the allocated bdevs.
env
The phys_addr
parameter in spdk_malloc() and spdk_zmalloc() has been deprecated.
For retrieving physical addresses, spdk_vtophys() should be used instead.
spdk_realloc() has been added to reallocate DMA/shared memory.
spdk_pci_device_is_removed() has been added to let the upper-layer SPDK drivers know
that device has a pending external hotremove request.
spdk_env_fini() and spdk_env_dpdk_post_fini() were added to release any resources
allocated by spdk_env_init() or spdk_env_dpdk_post_init() respectively. It is expected
that common usage of those functions is to call them just before terminating the process.
Added spdk_iommu_is_enabled() to report if SPDK application is using IOMMU for DMA.
DPDK
Dropped support for DPDK 17.07 and earlier, which SPDK won't even compile with right now.
Updated DPDK submodule to DPDK 19.02.
rpc
New get_spdk_version
RPC method is introduced to get version info of the running SPDK application.
The start_nbd_disk
RPC method now take nbd_device as an optional parameter. If nbd_device
is specified, use that specified nbd device. If it's not specified, pick available one.
iSCSI target
DIF strip and insert is now supported. DIF settings are not exposed to the iSCSI initiator.
DIF is attached into data for write I/O and stripped from data for read I/O.
vhost
Added experimental support for running with the external, upstream rte_vhost library.
This can be enabled by configuring SPDK with an --without-internal-vhost-lib
flag.
The minimum supported rte_vhost version (DPDK version) is 19.05-rc1.
As a result of fuzz testing, a lot of data races in vhost-scsi LUN hotplug path were identified and
fixed. Those data races could have potentially resulted in SPDK crashes, RPC hangs, or memory leaks
if Vhost-SCSI LUN hotplug RPCs were executed while connected VMs were in the middle of restarting.
The SCSI target id in add_vhost_scsi_lun
RPC is now optional. If -1
is passed, the first
unoccupied target id will be used.
AIO
AIO bdev module can now reap I/O completions directly from userspace, significantly improving
the overall performance.
blobfs
Synchronous IO operations no longer use spdk_io_channel, but instead use
spdk_fs_thread_ctx. The behavior is otherwise identical.
OCF
Added support for caching multiple bdevs using a single bdev as a cache.
notify
Added the notify library that provides a high performance local event bus
between libraries. Example usage was added to bdev module, which reports
notifications for added and removed bdevs via RPC.
sock
Added new API spdk_sock_readv() to the sock library for performing vectored
reads.
event
The function spdk_subsystem_init() no longer requires spdk_event as an argument.
Changed API of spdk_subsystem_config_json() to no longer be asynchronous.
io_uring
A bdev module that issues I/O to kernel block devices using the new io_uring Linux kernel
API was added. This module requires liburing.
build
Options to easily compile with profile guided optimization have been added to
configure
. To use profile guided optimization with SPDK, run
./configure --with-pgo-capture
, build SPDK, then run a workload of your
choosing. Then, simply run ./configure --with-pgo-enable
and recompile to
build using the generated profile data. Profile guided optimization can yield
very large performance improvements, especially on GCC 8 and clang 7. This may
be combined with link time optimization which has been available under the
--enable-lto
configure option for several releases.
compression bdev/reduce library
Added "reduce" block compression scheme based on using SSDs for storing
compressed blocks of storage and presistent memory for metadata. Please see
compression for more details.
v18.10.2: Maintenance Release
The SPDK 18.10.x line has been updated to use DPDK 18.11, which is a long term support release for DPDK.
SPDK v19.01.1: Maintenance Release
SPDK 19.01.1 is a bug fix and maintenance release.
logical volumes
Added option to change method for erasing data region on lvol store creation.
Default of unmapping can now be changed to writing zeroes or no operation.
autotest
Introduce SPDK_RUN_FUNCTIONAL_TEST variable enabled by default, that can be
manually disabled on systems where e.g. only unit tests are run.
FTL
Add detection for OpenChannel devices, so that NVMe tests and FTL bdev tests
are run on appropriate devices.
QoS
Enabled rate limit when opening the bdev, as there were some cases where
previously closed bdev would have QoS disabled.
GitHub issues
- #523: (fio_plugin) Fixed hang on pthread_exit(NULL).
- #633: (crypto bdev) Fixed memory leak in driver init path.
- #602: (nvme bdev) Will not attach user deleted controllers automatically.
- #676: (nvme_bdev) Added SHST_COMPLETE quirk for VMWare emulated SSDs.
- #638: (nvmf) Fixed the lvol creation failure issue for NVMe-oF TCP.
- #699: (nvmf) Updated default number of shared buffers for RDMA.
- #673: (vhost) Will use mmap_size to check for 2MB hugepage multiple.
- #663: (building) Fixed duplicated clean target in shared_lib/Makefile
- #593: (spdkcli) Will exit with 1 when rpc throws JSONRPCException.