Skip to content

@tomzawadzki tomzawadzki released this Oct 30, 2020

accel

New APIs spdk_accel_batch_cancel and spdk_idxd_batch_cancel were added to allow
cancelling a batch sequence.

bdev

A new spdk_bdev_part_base_construct_ext function has been added and the
spdk_bdev_part_base_construct has been deprecated. The
spdk_bdev_part_base_construct_ext function takes bdev name as an argument instead
of bdev structure to avoid a race condition that can happen when the bdev is being
removed between a call to get its structure based on a name and actually opening it.

Removed spdk_bdev_config_text function for bdev modules to report legacy config.

blobstore

A new spdk_bdev_create_bs_dev_ext function has been added and spdk_bdev_create_bs_dev_from_desc
function has been deprecated.

dpdk

Updated DPDK submodule to DPDK 20.08.

env

A new API spdk_mem_get_fd_and_offset was added to get file descriptor and offset for
a given virtual address.

event

Removed spdk_subsystem_config callback for submodules as part of legacy config removal.

Removed spdk_app_get_running_config function that printed configuration in legacy format,
and removed usr1_handler from struct spdk_app_opts callback that was used to call it.

Added SPDK thread scheduler framework which is used to rebalance load among CPU cores.
The scheduler implementation is pluggable and two schedulers are added first, static scheduler
and gscheduler scheduler.

static scheduler is used by default and does not reschedule threads and keeps the initial
assignments.

CPU frequency governor framework and implementation are also added.
dpdk_governor is the first implementation and is based on rte_power library of DPDK.
gscheduler scheduler utilizes the CPU frequency governor.

Scheduler and governor frameworks are experimental features.

fio

Bdev fio_plugin no longer supports legacy configuration files. Options spdk_conf and
spdk_json_conf have to point to valid JSON configuration.

gpt

Removed option to disable GPT probing via legacy configuration. Please use generic
bdev functionality to replicate the behavior. See bdev_set_options --disable-auto-examine
and bdev_examine RPC.

intel-ipsec-mb

Updated intel-ipsec-mb submodule to v0.54

isa-l

Updated ISA-L submodule to v2.29.0.

iscsi

The iSCSI target supports iSCSI login redirection feature to achieve fault tolerance
and load balancing particularly in scale out iSCSI target system, which runs multiple
SPDK iSCSI targets.

json

A new API spdk_json_decode_object_relaxed was added to allow parsing json values
even if decoder for a given key is not found.

A new API spdk_json_free_object was added to free memory allocated by spdk_json_decode_object.

log

A log flag structure, spdk_log_register_flag, spdk_log_get_first_flag,
spdk_log_get_next_flag and macro SPDK_LOG_REGISTER_COMPONENT, have been added which
enable registering and interacting with log flags. Added SPDK_INFOLOG, SPDK_DEBUGLOG
and SPDK_LOGDUMP macros to display logs for specified log flag.

The log flags in the event framework -L and several SPDK applications has been enabled on
release builds of SPDK. On debug builds this option will additionally set
log print level to SPDK_LOG_DEBUG.

A new API spdk_log_enable_timestamps and new RPC log_enable_timestamps was added to
toggle log timestamps.

miscellaneous

The contents of the log_rpc library have been moved to the event library. The log_rpc
library now no longer exists.

The contents of the app_rpc library have been moved to the event library. The app_rpc
library now no longer exists.

The contents of the bdev_rpc library have been moved to the bdev library. The app_rpc
library now no longer exists.

The bdevperf application now disables the zcopy API by default. Prior to this change,
bdevperf enabled using the zcopy API by default which caused a performance impact of
up to 25% on bdevs that don't natively support zcopy because the API emulates zero-copy
by allocating a buffer. The bdevperf -x param was renamed to -Z and the default
value changed to false. For bdevs that support zcopy, use the -Z flag to enable
using zcopy API.

net

Net framework intended for use by user-space TCP stacks was removed.

nvme

Zoned Namespace Command Set support has been added to NVMe driver, see nvme_zns.h for details.

New APIs, spdk_nvme_detach_async and spdk_nvme_detach_poll_async, have been added to
detach multiple controllers in parallel to mitigate lengthy shutdown notification of
a few NVMe SSDs.

New APIs, spdk_nvme_ns_get_ana_group_id and spdk_nvme_ns_get_ana_state, have been
added to get ANA group ID and ANA state of the given namespace, respectively.

If no specific command set is requested (by setting the command_set member in the
spdk_nvme_ctrlr_opts structure), SPDK will automatically select the most appropriate
command set based on what the controller supports.

nvmf

A new API spdk_nvmf_subsystem_add_ns_ext has been added and the API spdk_nvmf_subsystem_add_ns
has been deprecated.

The NVMe-oF target now supports Asymmetric Namespace Access (ANA) Reporting to provide
multipath to NVMe-oF initiator.

Add no_wr_batching parameter to spdk_nvmf_transport_opts struct to disable
Work Requests batching in RDMA transport.

NVMf Target transports can now parse any additional JSON params in the nvmf_create_transport RPC
through the JSON context provided via spdk_nvmf_target_opts->transport_specific.

A new RPC nvmf_subsystem_set_options was added to allow transport specific options within subsystem.

A new API spdk_nvmf_transport_stop_listen_async was added to stop accepting new connections
at the provided address. nvmf_subsystem_remove_listener RPC now uses this function.

A new API spdk_nvmf_subsystem_disconnect_host was added to disconnect all connections
originating from the provided hostnqn. nvmf_subsystem_remove_host RPC now uses this function.

ocf

Updated OCF submodule to v20.03.1

rpc

New optional parameters, enable_placement_id and enable_quickack were added to the
sock_impl_set_options RPC.

A new RPC bdev_examine_bdev was added to allow users to examine a bdev explicitly.
It can be used only if bdev_auto_examine is set to false by the RPC bdev_set_options.

Add optional 'no_wr_batching' parameter to 'nvmf_create_transport' RPC method.

New RPCs, iscsi_target_node_set_redirect and iscsi_target_node_request_logout, have
been added, and a new optional parameter private has been added to the RPC
iscsi_create_portal_group to support iSCSI login redirection feature.

A new optional parameter ana_reporting was added to the RPC nvmf_create_subsystem, and
a new RPC nvmf_subsystem_listner_set_ana_state was added for ANA reporting.

New RPCs, nvmf_subsystem_get_listeners and nvmf_subsystem_get_qpairs, were added to
retrieve configuration of the NVMe-oF subsystem.

scsi

Two new APIs have been added spdk_scsi_dev_construct_ext and
spdk_scsi_dev_add_lun_ext that allow the upper layer(e.g. vhost-scsi) to
receive the notification when the scsi bdev has been resized.

The spdk_scsi_dev_construct and spdk_scsi_dev_add_lun eventually may be
deprecated and removed.

sock

The enable_placement_id field was added in the struct spdk_sock_impl_opts to
make the placement_id feature configurable by users. The default setting is
not enabled.

The enable_quick_ack field was added in the struct spdk_sock_impl_opts to enable
or disable quick ack for the POSIX sock module. The default setting is not enabled.

thread

fd_group is applied to support interrupt mode.
New APIs were added to support an experimental interrupt mode. This allows modules or
libraries to selectively register file descriptors that the spdk_thread can wait on,
as an alternative to polling. In v20.10, this functionality is enabled in a very small
subset of SPDK libraries and modules.

util

A new utility named fd_group was add. It is now
implemented by epoll on Linux platform. It can be used by
spdk_thread and reactor to implement interrupt mode.

Added spdk_bit_pool utility, see bit_pool.h for details.

vhost

SPDK has switched to DPDK's rte_vhost library since 19.07 release, removed the internal
rte_vhost library which is used for DPDK older than 19.05, removed the experimental vhost
nvme target which depends on the internal rte_vhost library.

vpp

Removed socket abstraction for VPP.

Assets 2
Oct 23, 2020
event: add scheduler_set RPC
Add RPC that allows to change scheduler at runtime.

Change-Id: I008670f5e936bc25a0fbc923b826277d15343273
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3958
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

@tomzawadzki tomzawadzki released this Jul 31, 2020 · 1 commit to v20.07.x since this release

accel

A new API was added spdk_accel_get_capabilities that allows applications to
query the capabilities of the currently enabled accel engine back-end.

A new capability, CRC-32C, was added via spdk_accel_submit_crc32c.

The software accel engine implemenation has added support for CRC-32C.

A new capability, compare, was added via spdk_accel_submit_compare.

The software accel engine implemenation has added support for compare.

Several APIs were added to accel_engine.h to support batched submission
of operations.

Several APIs were added to accel_engine.h to support dualcast operations.

accel_fw

The accel_fw was updated to no longer require the app to allocate an
accel_task on its behalf. All public APIs now take a callback arg as
the parameter that used to be the accel_task.

The accel_fw API spdk_accel_task_size was removed as it is no longer
required.

The accel_fw was updated to support compare, dualcast, crc32c.

The accel_fw introduced batching support for all commands in all plug-ins.
See docs for detailed information.

bdev

A new API spdk_bdev_abort has been added to submit abort requests to abort all I/Os
whose callback context match to the bdev on the given channel.

build

The fio plugins now compile to build/fio and are named spdk_bdev and spdk_nvme.
Existing fio configuration files will need to be updated.

dpdk

Updated DPDK submodule to DPDK 20.05.

env

A new public API spdk_env_get_primary_core returns the index of the primary core of
the SPDK process.

Several new APIs have been added to provide greater flexibility in registering and
accessing polled mode PCI drivers. See env.h for more details.

idxd

The idxd library and plug-in module for the accel_fw were updated to support
all accel_fw commands as well as batching. Batching is supported both
through the library and the plug-in module.

IDXD engine support for CRC-32C has been added.

ioat

A new API spdk_ioat_get_max_descriptors was added.

nvme

An opts_sizeelement was added in the spdk_nvme_ctrlr_opts structure
to solve the ABI compatiblity issue between different SPDK version.

A new API spdk_nvme_ctrlr_cmd_abort_ext has been added to abort previously submitted
commands whose callback argument match.

Convenience functions, spdk_nvme_print_command and spdk_nvme-print_completion were added
to the public API.

A new function, spdk_nvmf_cuse_update_namespaces, updates the cuse representation of an NVMe
controller.

A new function qpair_iterate_requests has been added to the nvme transport interface. ALl
implementations of the transport interface will have to implement that function.

nvmf

The NVMe-oF target no longer supports connecting scheduling configuration and instead
always uses what was previously called "transport" scheduling.

spdk_nvmf_tgt_accept no longer takes a function pointer as an argument. New connections
are automatically assigned to poll groups by the underlying transport. Further,
spdk_nvmf_transport_ops has changed such that the accept function pointer no longer
takes a function pointer as an argument. Instead, transports should call
spdk_nvmf_tgt_new_qpair whenever they previously would have called that callback.

The NVMe-oF target now supports aborting any submitted NVM or Admin command. Previously,
the NVMe-oF target could abort only Asynchronous Event Request commands.

rdma

A new rdma library has been added. It is an abstraction layer over different RDMA providers.
Two providers are available - verbs (used by default when RDMA is enabled or enabled explicitly
using --with-rdma=verbs) and mlx5 Direct Verbs aka DV (enabled by --with-rdma=mlx5_dv).
Using mlx5_dv requires libmlx5 installed on the system.

rpc

Parameter -p or --max-qpairs-per-ctrlr of nvmf_create_transport RPC command accepted by the
rpc.py script is deprecated, new parameter -m or --max-io-qpairs-per-ctrlr was added.

Added sock_impl_get_options and sock_impl_set_options RPC methods.

Command line parameters -r and --rpc-socket will longer accept TCP ports. RPC server
must now be started on a Unix domain socket. Exposing RPC on the network, as well as providing
proper authentication (if needed) is now a responsibility of the user.

The bdev_set_options RPC has a new option, bdev_auto_examine to control the auto examine function
of bdev modules.

New RPCs sock_impl_get_options and sock_impl_set_options been added to expose new socket features.
See sock section for more details.

sock

Added spdk_sock_impl_get_opts and spdk_sock_impl_set_opts functions to set/get socket layer configuration
options. Options can be set independently for each implementation.

Added recv_buf_size and 'send_buf_size' socket layer options. They are used only in posix implementation.

Added uring based socket implementation, the code is located in module/sock/uring. This feature is only
available in Linux which requires kernel version is greater than 5.4.3. Currently, our CI pool added the uring
based socket tests for iSCSI target and also the tests for SPDK NVMe-oF tcp transport.

Added enable_recv_pipe socket layer option to allow disabling of double buffering on receive.
New option is used only in posix implementation.

Added enable_zerocopy_send socket layer option to allow disabling of zero copy flow on send.
New option is used only in posix implementation.

util

Some previously exposed CRC32 functions have been removed from the public API -
spdk_crc32_update, spdk_crc32_table_init, and the spdk_crc32_table struct.

vhost

The function spdk_vhost_blk_get_dev has been removed.

Assets 2
Jul 24, 2020
dpdkbuild: build dpdk with meson+ninja
Makefile support in DPDK was deprecated and will be removed soon,
so switch to the officially supported way of building DPDK -
with meson and ninja. Two new tools. Basically, our Makefiles
will invoke meson+ninja for DPDK, no other SPDK components are
affected.

Apparently DPDK wanted to move away from an octopus-like config
system and the ideology behind meson configuration is simple now:
build everything by default. Some PMDs can be explicitly disabled
with meson command line, but all libraries (both static and shared
versions) and test apps are built unconditionally.

How long does it take to build minimal DPDK with meson? Too much.
On my machine half of the total build time is spent on libraries
we don't need at all. (I have some hacks up my sleeve to disable
building those libraries - see the subsequent patch.) As for the
official way of building a minimal DPDK, there was a patch [1]
on dpdk mailing list to introduce more specific configuration,
but it was rejected:

> We talked about this a few times in the past, and it was actually one
> of the design goals to _avoid_ replicating the octopus-like config
> system of the makefiles. That's because it makes the test matrix
> insanely complicated, not to mention the harm to user friendliness,
> among other things.
>
> If someone doesn't want to use a PMD, they can just avoid installing it
> - it's simple enough.
>
> Sorry, but from me it's a very strong NACK.

Let's not follow that direction, hack the DPDK build system instead.

As for advantages of meson+ninja over Makefiles? I can't find any.
It's another build system that does a lot for you with some functions,
magic options, and a built-in dependency system. It seems nice if you know
the syntax, but it's another component that you need to learn, debug,
and possibly find bugs in (there's a lot of github issues open for meson).
I would compare it to CMake.

As for changes in this patch: rather that explicitly disabling
PMDs we don't need, specify a list of PMDs we do need and disable
everything else found in ./dpdk/drivers/*. This way we won't have
to disable the new PMDs as they're added to DPDK.

Meson configuration also sets RTE_EAL_PMD_PATH #define to a valid directory
with built PMD shared libs. When it's set, DPDK dynamically loads all shared
libraries inside. The drivers there depend on DPDK shared libs and fail to
load in static SPDK builds, so we disable them altogether by unsetting
RTE_EAL_PMD_PATH in the meson-generated config file - just like
DPDK Makefiles did. EAL checks for RTE_EAL_PMD_PATH being empty and skips
loading any external PMDs then. We do it for both static and shared libs.
We specify all PMDs at build time for now, so there's just no need to load
them dynamically.

We have three more hacks in our submodule:
 * disable building dpdk apps by commenting-out a line in dpdk/meson.build
 * disable building unnecessary libs (build everything that spdk *may*
   need)
 * build isa-l compress pmd with `-L[...] -lisal`. DPDK expects to find
   libisal with pkg-config. We don't want to prepare a pkg-config file,
   so comment-out a failing check in another meson.build file and provide
   isa-l through CFLAGS and LDFLAGS.

We also need to make some changes to our test/external_code. First of
all, -ldpdk is no more. Meson build generates a pkg-config file with all
libs, but we'll switch to it in a separate patch - for now just specify
all -lrte_ libs one by one. -Wl,--no-as-needed has to be added to some
test cases, otherwise rte_mempool_ring isn't loaded. We don't use any
APIs from this library, it only has a static constructor that provides
a few callbacks used by rte_mempool_create(). Also, since DPDK now builds
both static and shared libraries, we need to add -Wl,-Bstatic to force
using static libswhere required. It's only needed for DPDK libs, but we
use it for SPDK libs as well since there's no harm.

As for performance:
$ ./configure --enable-debug --with-crypto --with-reduce
$ time make -j40 -C dpdkbuild all
with meson:
real    0m8.287s
user    1m7.983s
sys     0m10.548s

before, with the old DPDK makefiles:
real    0m20.232s
user    0m55.921s
sys     0m16.491s

The subsequent builds are much faster too:
$ time make -j40 -C dpdkbuild all
meson:
real    0m0.876s
user    0m0.663s
sys     0m0.217s

makefiles:
real    0m10.150s
user    0m11.740s
sys     0m6.772s

[1] http://inbox.dpdk.org/dev/1a07d1cd59d84dce84e56c10fdabf5e5504560a6.camel@debian.org/

Change-Id: Ic65db563014100bafb12e61ee0530cc2ae64401d
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1440
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

@tomzawadzki tomzawadzki released this Jun 1, 2020 · 2 commits to v20.04.x since this release

SPDK 20.04.1 is a bug fix and maintenance release.

dpdk

Updated DPDK submodule to DPDK 19.11.2, which includes fixes for DPDK vulnerabilities:
CVE-2020-10722, CVE-2020-10723, CVE-2020-10724, CVE-2020-10725, CVE-2020-10724.

Assets 2

@tomzawadzki tomzawadzki released this Jun 1, 2020 · 2 commits to v20.01.x since this release

SPDK 20.01.2 is a bug fix and maintenance LTS release.

dpdk

Updated DPDK submodule to DPDK 19.11.2, which includes fixes for DPDK vulnerabilities:
CVE-2020-10722, CVE-2020-10723, CVE-2020-10724, CVE-2020-10725, CVE-2020-10724.

env_dpdk

A new function, spdk_mem_reserve, has been added to reserve a memory region in SPDK's
memory maps. It pre-allocates data structures to hold memory address translations
without populating the region.

rpc

A new RPC, bdev_rbd_resize has been added to resize the Ceph RBD bdev.

GitHub issues

  • #1291: lib/blob: merge EP of a clone when deleting a snapshot
  • #1292: nvme/rdma: Wait for completions of both RDMA RECV and SEND
Assets 2

@tomzawadzki tomzawadzki released this Apr 30, 2020 · 6 commits to v20.04.x since this release

configuration

Legacy INI style configuration for SPDK applications has been deprecated and will be
removed in future release.
Please switch to JSON-RPC configuration files and/or RPC driven run-time configuration.

copy

The copy engine library, modules and public APIs have been renamed. Use of the word copy
has been replaced with the word accel short for accelerator in preparation for adding new
capabilities in the future. Additionally, APIs for what was previously called the memcpy
engine have been renamed to identify the engine as a software accelerator.

crypto

Support for AES_XTS was added for the QAT polled mode driver (pmd). The create RPC
bdev_crypto_create has 2 new optional parameters: cipher and key2. Cipher can be either
AES_CBC (default) or AES_XTS. AES_XTS is only valid when using the QAT polled mode driver.
The key2 parameter is the second key required for AES_XTS.

event

Reactors now accumulate CPU stats and they are retrieved by the RPC framework_get_reactors.

idxd

IDXD support was added in the form of a low level library that can directly
utilize the IDXD offload engine as well as an accel plug-in module allowing for use
with the generic accel framework API. IDXD is the first in a family of offload
engines that share the same interface, specifically DSA is added here. More info
can be found here: https://01.org/blogs/2019/introducing-intel-data-streaming-accelerator

Much of the implementation models IOAT, however the low level interfaces are very
different. The RPC to enable IDXD requires a configuration number as well. The
code includes two pre-defined configurations of IDXD groups/work queues/engines. A future
version will provide an interface to allow for setting of individual configuration
parameters.

IDXD is not yet available so this feature should be considered experimental. It will
be built up with additional documentation as an ongoing activity.

iscsi

The iSCSI target now creates a lightweight thread per poll group instead of assuming a pool
of lightweight threads already exist at start up time. A poll group is a collection of
unrelated iSCSI connections. Each poll group is only accessed from the associated
lightweight thread.

ftl

Several changes have been made to the spdk_ftl_conf, spdk_ftl_dev_init_ops, and
spdk_ftl_attrs structs. Please see include/spdk/ftl.h for more details.

miscellaneous

The --json-ignore-init-errors command line parameter has been added to ignore
initialization errors on JSON config load.

The public header file io_channel.h has been removed. Please use thread.h which has the
exact same API.

nvme

Exported internal nvme_ctrlr_cmd_security_receive/send() APIs as public APIs with "the spdk_"
prefix.

Added priority field in spdk_nvme_transport_id, this field is used to specify the priority
of the NVMe-oF connection, and currently it is used for NVMe-oF tcp connection.

A new poll group API has been added to allow for pooling of nvme qpairs across a single
entity which can be polled for completions. This new API consists of the spdk_nvme_poll_group
family of functions. As a result of this new API, all NVMe transports are expected to implement
several poll group related functions.

A new flag, create_only, has been added to the spdk_nvme_io_qpair_opts structure. This flag
allows a user to call spdk_nvme_ctrlr_get_default_io_qpair without also connecting the qpair
within the context of that call.

As a result of the create_only flag, two new API functions, spdk_nvme_ctrlr_connect_io_qpair
and spdk_nvme_ctrlr_disconnect_io_qpair, have been added to facilitate connecting newly created
qpairs (for example, after they have been added to a poll group) and disconnecting qpairs without
destroying them (for example to disconnect a qpair before migrating it to a new poll group and
reconnecting it).

The functions spdk_nvme_ctrlr_alloc_cmb_io_buffer and spdk_nvme_ctrlr_free_cmb_io_buffer
have been changed to spdk_nvme_ctrlr_map_cmb and spdk_nvme_ctrlr_unmap_cmb respectively.

An additional function, spdk_nvme_ctrlr_reserve_cmb, has been added to facilitate reserving
the entire size of the controller memory buffer for data transfer.

nvme_cuse

spdk_nvme_cuse_get_ctrlr_name now takes two additional parameters, char *name which
stores the pointer to the controller name, and size_t *size which stores the length of
the name. The return type has also been changed from char * to int.

spdk_nvme_cuse_get_ns_name now takes two additional parameters, char *name which
stores the pointer to the namespace name, and size_t *size which stores the length of
the name. The return type has also been changed from char * to int.

nvme_opal

Several public OPAL structure definitions have been changed since the last release.

nvmf

spdk_nvmf_poll_group_destroy() is now asynchronous and accepts a completion callback.

The NVMe-oF target now creates a lightweight thread per poll group instead of assuming a pool
of lightweight threads already exist at start up time. A poll group is a collection of
unrelated NVMe-oF connections. Each poll group is only accessed from the associated
lightweight thread.

A new struct, spdk_nvmf_subsystem_listener, has been added to encapsulate the subsystem specific
nature of a listener object.

spdk_nvmf_tgt_listen no longer accepts a callback function or argument. It also returns an
int to indicate the status of the listen call.

The execution of spdk_nvme_poll_group_destroy is now asynchronous and the function accepts
a cb_fn and cb_arg to call upon completion.

The execution of spdk_nvmf_subsystem_add_listener is now asynchronous and the function accepts
a cb_fn and cb_arg to call upon completion.

The nvmf_transport.h header has been made public to allow custom NVMe-oF transports to integrate
with NVMe-oF libraries without using internal APIs.

ocf

Updated the OCF submodule to OCF v20.03

New version of OCF comes with API changes and bug fixes

rpc

A new RPC thread_set_cpumask has been added to set the cpumask of the thread
to the specified value.

A new RPC thread_get_pollers has been added to retrieve pollers from SPDK threads.

A new RPC thread_get_io_channels has been added to retrieve I/O channels from SPDK threads.

A new RPC bdev_rbd_resize has been added to resize Ceph RBD bdevs.

sock

The spdk_sock_set_priority function has been removed since the feature to set the sock priority
will be contained in two new functions, i.e., spdk_sock_listen_ext and spdk_sock_connect_ext.
Users may now specify the priority of the socket in the opts that they want to use.

spdk_top

A new application, spdk_top, has been added which allows users to monitor resource consumption
by a running SPDK application. More information on this application can be found in
app/spdk_top/README.

thread

A new function spdk_thread_lib_init_ext has been added, and the function
spdk_thread_lib_init has been deprecated. The user of spdk_thread_lib_init_ext is
expected to implement both functions spdk_thread_op_fn and spdk_thread_op_supported_fn.
spdk_thread_op_supported_fn is called to check whether the SPDK thread operation
is supported. spdk_thread_op_fn is called to execute the SPDK thread operation.
Current SPDK operation types are SPDK_THREAD_OP_NEW and SPDK_THREAD_OP_RESCHED.
The operation SPDK_THREAD_OP_NEW is called each time a new thread is created.
The operation SPDK_THREAD_OP_RESCHED is called when SPDK thread needs to be rescheduled.

A unique ID has been added for each created SPDK thread, it is retrieved by a new function
spdk_thread_get_id, and the SPDK thread which has the specific ID is got by
a new function spdk_thread_get_by_id.

A new function spdk_thread_cpumask has been added to set the current thread's cpumask
to the specified value. The function requires the operation SPDK_THREAD_OP_RESCHED
is supported.

A new function spdk_poller_register_named has been added to set arbitrary name to the
created poller. If NULL, the name is set to the pointer of the poller function.

The function spdk_thread_poll now measures run time per call correctly on multiple SPDK
threads configuration, and a new function spdk_thread_get_last_tsc has been added to use together.

Voluntary termination of SPDK thread has been supported by refining the functions spdk_thread_exit
and spdk_thread_poll.

util

New functions spdk_sn32_lt and spdk_sn32_gt have been added. They compare two sequence
numbers based on serial number arithmetic.

vhost

Poll groups per session have been replaced by SPDK threads per vhost controller.

vmd

A new function, spdk_vmd_fini, has been added. It releases all resources acquired by the VMD
library through the spdk_vmd_init call.

Assets 2
Apr 27, 2020
bdev/nvme: use g_nvme_bdev_ctrlrs as the poll group io device
IO device "bdev_nvme_poll_groups" is not unregistered in the finish
module, so we need to unregister this io device, since the "nvme_if"
is staic variable in bdev_nvme.c, it's can't be accessed in common.c
so we use "g_nvme_bdev_ctrlrs" instead.

Change-Id: Id4675f5acc8f386609903497da5ca84bf8af3a15
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2035
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>

@tomzawadzki tomzawadzki released this Mar 20, 2020 · 2651 commits to master since this release

SPDK 20.01.1 is a bug fix and maintenance LTS release.

GitHub issues

  • #960 #1170: lib/blob: queue up blob persists when one already is ongoing
  • #1184: rdma: Correct handling of RDMA_CM_EVENT_DEVICE_REMOVAL
  • #1192: nvmf/rpc: Destroy subsystem if spdk_rpc_nvmf_create_subsystem fails
  • #1206: env_dpdk: dont treat NULL as error in spdk_map_bar_rte()
  • #1223: test/nvme: fix correct controllers name in nvme-cli cuse test
  • #1228: nvme: set transport string before the probe based on transport type
  • #1232: module/crypto: increase the number of queue pairs for AESNI_MB
  • #1262: dpdk: move submodule to commit 3fcb1dd
  • #1273: bdev/nvme: do not destruct ctrlr if reset is in progress
  • #1278: lib/iscsi: Return when connection state is already exited at login completion
Assets 2

@tomzawadzki tomzawadzki released this Jan 31, 2020 · 56 commits to v20.01.x since this release

bdev

A new function, spdk_bdev_set_timeout, has been added to set per descriptor I/O timeouts.

A new class of functions spdk_bdev_compare*, have been added to allow native bdev support
of block comparisons and compare-and-write.

A new class of bdev events, SPDK_BDEV_EVENT_MEDIA_MANAGEMENT, has been added to allow bdevs
which expose raw media to alert all I/O channels of pending media management events.

A new API was added spdk_bdev_io_get_aux_buf allowing the caller to request
an auxiliary buffer for its own private use. The API is used in the same manner that
spdk_bdev_io_get_buf is used and the length of the buffer is always the same as the
bdev_io primary buffer. 'spdk_bdev_io_put_aux_buf' frees the allocated auxiliary
buffer.

blobfs

Added boolean return value for function spdk_fs_set_cache_size to indicate its operation result.

Added blobfs_set_cache_size RPC method to set cache size for blobstore filesystem.

blobstore

Added new use_extent_table option to spdk_blob_opts for creating blobs with Extent Table descriptor.
Using this metadata format, dramatically decreases number of writes required to persist each cluster allocation
for thin provisioned blobs. Extent Table descriptor is enabled by default.
See the Blobstore Programmer's Guide for more details.

dpdk

Updated DPDK submodule to DPDK 19.11.

env_dpdk

spdk_env_dpdk_post_init now takes a boolean, legacy_mem, as an argument.

A new function, spdk_env_dpdk_dump_mem_stats, prints information about the memory consumed by DPDK to a file specified by
the user. A new utility, scripts/dpdk_mem_info.py, wraps this function and prints the output in an easy to read way.

event

The functions spdk_reactor_enable_framework_monitor_context_switch() and
spdk_reactor_framework_monitor_context_switch_enabled() have been changed to
spdk_framework_enable_context_switch_monitor() and
spdk_framework_context_switch_monitor_enabled(), respectively.

ftl

All NVMe dependencies were removed from ftl library.
From now ftl library is using bdev_zone API.
bdev_ftl becomes virtual bdev.

ctrlr and trid fields in spdk_ftl_dev_init_opts structure were replaced by
base_bdev_desc.

bdev_ftl_create RPC method trtype and traddr parameters were replaced by base_bdev
parameter.

spdk_ftl_module_init and spdk_ftl_module_fini functions were removed.

spdk_ftl_punit_range and ftl_module_init_opts structures were removed.

isa-l

Updated ISA-L submodule to commit f3993f5c0b6911 which includes implementation and
optimization for aarch64.

Enabled ISA-L on aarch64 by default in addition to x86.

nvme

delayed_pcie_doorbell parameter in spdk_nvme_io_qpair_opts was renamed to delay_cmd_submit
to allow reuse in other transports.

Added RDMA WR batching to NVMf RDMA initiator. Send and receive WRs are chained together
and posted with a single call to ibv_post_send(receive) in the next call to qpair completion
processing function. Batching is controlled by 'delay_cmd_submit' qpair option.

The NVMe-oF initiator now supports plugging out of tree NVMe-oF transports. In order
to facilitate this feature, several small API changes have been made:

The spdk_nvme_transport_id struct now contains a trstring member used to identify the transport.
A new function, spdk_nvme_transport_available_by_name, has been added.
A function table, spdk_nvme_transport_ops, and macro, SPDK_NVME_TRANSPORT_REGISTER, have been added which
enable registering out of tree transports.

A new function, spdk_nvme_ns_supports_compare, allows a user to check whether a given namespace supports the compare
operation.

A new family of functions, spdk_nvme_ns_compare*, give the user access to submitting compare commands to NVMe namespaces.

A new function, spdk_nvme_ctrlr_cmd_get_log_page_ext, gives users more granular control over the command dwords sent in
log page requests.

nvmf

Add SockPriority option in [Transport] section, this can be used for NVMe-oF target
on TCP transport to set sock priority for the incomming TCP connections.

The NVMe-oF target now supports plugging out of tree NVMe-oF transports. In order
to facilitate this feature, several small API changes have been made:

The spdk_nvme_transport_id struct now contains a trstring member used to identify the transport.
spdk_nvmf_tgt_get_transport, spdk_nvmf_transport_opts_init, and spdk_nvmf_transport_create all have been
modified to take a string.
A function table, spdk_nvmf_transport_ops, and macro, SPDK_NVMF_TRANSPORT_REGISTER, have been added which
enable registering out of tree transports.

Add spdk_nvmf_tgt_stop_listen() that can be used to stop listening for
incoming connections for specified target and trid. Listener is not stopped
implicitly upon destruction of a subsystem any more.

A custom NVMe admin command handler has been added which allows the user to use the real drive
attributes from one of the target NVMe drives when reporting drive attributes to the initiator.
This handler can be enabled via the nvmf_set_config RPC.
Note: In a future version of SPDK, this handler will be enabled by default.

The SPDK target and initiator both now include compare-and-write functionality with one caveat. If using the RDMA transport,
the target expects the initiator to send both the compare command and write command either with, or without inline data. The
SPDK initiator currently respects this requirement, but this note is included as a flag for other initiators attempting
compatibility with this version of SPDK.

rpc

A new RPC, bdev_zone_block_create, enables creating an emulated zoned bdev on top of a standard block device.

A new RPC, bdev_ocssd_create, enables creating an emulated zoned bdev on top of an Open Channel SSD.

A new RPC, blobfs_set_cache_size, enables managing blobfs cache size.

A new RPC, env_dpdk_get_mem_stats, has been added to facilitate reading DPDK related memory
consumption stats. Please see the env_dpdk section above for more details.

A new RPC, framework_get_reactors, has been added to retrieve a list of all reactors.

bdev_ftl_create now takes a base_bdev argument in lieu of trtype, traddr, and punits.

bdev_nvme_set_options now allows users to disable I/O submission batching with the -d flag

bdev_nvme_cuse_register now accepts a name parameter.

bdev_uring_create now takes arguments for bdev_name and block_size

nvmf_set_config now takes an argument to enable passthru of identify commands to base NVMe devices.
Please see the nvmf section above for more details.

scsi

spdk_scsi_lun_get_dif_ctx now takes an additional argument of type spdk_scsi_task.

sock

Added spdk_sock_writev_async for performing asynchronous writes to sockets. This call will
never return EAGAIN, instead queueing internally until the data has all been sent. This can
simplify many code flows that create pollers to continue attempting to flush writes
on sockets.

Added impl_name parameter in spdk_sock_listen and spdk_sock_connect functions. Users may now
specify the sock layer implementation they'd prefer to use. Valid implementations are currently
"vpp" and "posix" and NULL, where NULL results in the previous behavior of the functions.

thread

spdk_thread_send_msg now returns int indicating if the message was successfully
sent.

A new function spdk_thread_send_critical_msg, has been added to support sending a single message from
a context that may be interrupted, e.g. a signal handler.

Two new functions, spdk_poller_pause, and spdk_poller_resume, have been added to give greater control
of pollers to the application owner.

util

spdk_pipe, a new utility for buffering data from sockets or files for parsing
has been added. The public API is available at include/spdk/pipe.h.

bdev

Added spdk_bdev_io_get_nvme_fused_status function for translating bdev_io status to NVMe status
code for fused compare-and-write operation.

Added spdk_bdev_get_acwu function for getting block device atomic compare and write unit size.

Assets 2
You can’t perform that action at this time.