Permalink
Fetching contributors…
Cannot retrieve contributors at this time
569 lines (347 sloc) 13.9 KB

NAME

nbdkit - toolkit for creating NBD servers

SYNOPSIS

__SYNOPSIS__

DESCRIPTION

Network Block Device (NBD) is a network protocol for accessing block devices over the network. Block devices are hard disks and things that behave like hard disks such as disk images and virtual machines.

nbdkit is both a toolkit for creating NBD servers from "unconventional" sources and the name of an NBD server.

nbdkit ships with many plugins and filters for performing common tasks like serving files. See the "EXAMPLES" and "SEE ALSO" sections below, and look in the directory $libdir/nbdkit.

To create a new Network Block Device source, all you need to do is write a few glue functions, possibly in C, or perhaps in a high level language like Perl or Python. The liberal licensing of nbdkit is meant to allow you to link nbdkit with proprietary libraries or to include nbdkit in proprietary code.

If you want to write an nbdkit plugin, you should read nbdkit-plugin(3).

You can give the full path to the plugin, like this:

nbdkit $libdir/nbdkit/plugins/nbdkit-file-plugin.so [...]

but it is usually more convenient to use this equivalent syntax:

nbdkit file [...]

$libdir is set at compile time. To print it out, do:

nbdkit --dump-config

EXAMPLES

Basic file serving

Serve file disk.img on port 10809 using nbdkit-file-plugin(1), and connect to it using guestfish(1):

nbdkit file file=disk.img
guestfish --rw --format=raw -a nbd://localhost

Serve file disk.img on port 10809, requiring clients to use encrypted (TLS) connections:

nbdkit --tls=require file file=disk.img

Other nbdkit plugins

Create a 1MB disk with one empty partition entirely on the command line using nbdkit-data-plugin(1):

nbdkit data data="@0x1b8 0xf8 0x21 0xdc 0xeb 0 0 0 0
                  2 0 0x83 0x20 0x20 0 1 0  0 0 0xff 0x7
                  @0x1fe 0x55 0xaa" \
                  size=1M

Forward an NBD connection over HTTP using nbdkit-curl-plugin(1):

nbdkit -r curl url=http://example.com/disk.img

Combining plugins and filters

Serve only the first partition from compressed disk image disk.img.xz, combining nbdkit-xz-plugin(1) and nbdkit-partition-filter(1):

nbdkit --filter=partition  xz disk.img.xz  partition=1

To understand this command line:

plugin name and plugin parameter
             │
       ┌─────┴────┐
       │          │
 nbdkit --filter=partition  xz disk.img.xz  partition=1
                  │                               │
                  └─────────────┬─────────────────┘
           │
                filter name and filter parameter

Create a scratch, empty nbdkit device and inject errors and delays, for testing clients, using nbdkit-memory-plugin(1), nbdkit-error-filter(1) and nbdkit-delay-filter(1):

nbdkit --filter=error --filter=delay memory size=100M \
       error-rate=10% rdelay=1 wdelay=1

Display information

Display information about nbdkit or a specific plugin:

nbdkit --help
nbdkit --version
nbdkit --dump-config
nbdkit example1 --help
nbdkit example1 --dump-plugin

GLOBAL OPTIONS

--help

Display brief command line usage information and exit.

-D PLUGIN.FLAG=N
-D FILTER.FLAG=N
--debug PLUGIN.FLAG=N
--debug FILTER.FLAG=N

Set the plugin or filter Debug Flag called FLAG to the integer value N. See "Debug Flags" in nbdkit-plugin(3).

--dump-config

Dump out the compile-time configuration values and exit. See nbdkit-probing(1).

--dump-plugin

Dump out information about the plugin and exit. See nbdkit-probing(1).

--exit-with-parent

If the parent process exits, we exit. This can be used to avoid complicated cleanup or orphaned nbdkit processes. There are some important caveats with this, see "EXIT WITH PARENT" in nbdkit-captive(1).

An alternative to this is "CAPTIVE NBDKIT" in nbdkit-captive(1).

This option implies --foreground.

-e EXPORTNAME
--export EXPORTNAME
--export-name EXPORTNAME
--exportname EXPORTNAME

Set the exportname.

If not set, exportname "" (empty string) is used. Exportnames are not allowed with the oldstyle protocol.

-f
--foreground
--no-fork

Don't fork into the background.

--filter FILTER

Add a filter before the plugin. This option may be given one or more times to stack filters in front of the plugin. They are processed in the order they appear on the command line. See "FILTERS" and nbdkit-filter(3).

-g GROUP
--group GROUP

Change group to GROUP after starting up. A group name or numeric group ID can be used.

The server needs sufficient permissions to be able to do this. Normally this would mean starting the server up as root.

See also -u.

-i IPADDR
--ip-addr IPADDR
--ipaddr IPADDR

Listen on the specified interface. The default is to listen on all interfaces. See also -p.

--log=stderr
--log=syslog

Send error messages to either standard error (--log=stderr) or to the system log (--log=syslog).

The default is to send error messages to stderr, unless nbdkit forks into the background in which case they are sent to syslog.

For more details see "LOGGING" in nbdkit-service(1).

-n
--new-style
--newstyle

Use the newstyle NBD protocol protocol. This is the default in nbdkit ≥ 1.3. In earlier versions the default was oldstyle. See nbdkit-protocol(1).

-o
--old-style
--oldstyle

Use the oldstyle NBD protocol. This was the default in nbdkit ≤ 1.2, but now the default is newstyle. Note this is incompatible with newer features such as export names and TLS. See nbdkit-protocol(1).

-P PIDFILE
--pid-file PIDFILE
--pidfile PIDFILE

Write PIDFILE (containing the process ID of the server) after nbdkit becomes ready to accept connections.

If the file already exists, it is overwritten. nbdkit does not delete the file when it exits.

-p PORT
--port PORT

Change the TCP/IP port number on which nbdkit serves requests. The default is 10809. See also -i.

-r
--read-only
--readonly

The export will be read-only. If a client writes, then it will get an error.

Note that some plugins inherently don't support writes. With those plugins the -r option is added implicitly.

nbdkit-cow-filter(1) can be placed over read-only plugins to provide copy-on-write (or "snapshot") functionality. If you are using qemu as a client then it also supports snapshots.

--run CMD

Run nbdkit as a captive subprocess of CMD. When CMD exits, nbdkit is killed. See "CAPTIVE NBDKIT" in nbdkit-captive(1).

This option implies --foreground.

-s
--single
--stdin

Don't fork. Handle a single NBD connection on stdin/stdout. After stdin closes, the server exits.

You can use this option to run nbdkit from inetd or similar superservers; or just for testing; or if you want to run nbdkit in a non-conventional way. Note that if you want to run nbdkit from systemd, then it may be better to use "SOCKET ACTIVATION" in nbdkit-service(1) instead of this option.

This option implies --foreground.

--selinux-label SOCKET-LABEL

Apply the SELinux label SOCKET-LABEL to the nbdkit listening socket.

The common — perhaps only — use of this option is to allow libvirt guests which are using SELinux and sVirt confinement to access nbdkit Unix domain sockets:

nbdkit --selinux-label system_u:object_r:svirt_t:s0 ...
-t THREADS
--threads THREADS

Set the number of threads to be used per connection, which in turn controls the number of outstanding requests that can be processed at once. Only matters for plugins with thread_model=parallel (where it defaults to 16). To force serialized behavior (useful if the client is not prepared for out-of-order responses), set this to 1.

--tls=off
--tls=on
--tls=require

Disable, enable or require TLS (authentication and encryption support). See nbdkit-tls(1).

--tls-certificates /path/to/certificates

Set the path to the TLS certificates directory. If not specified, some built-in paths are checked. See nbdkit-tls(1) for more details.

--tls-psk /path/to/pskfile

Set the path to the pre-shared keys (PSK) file. If used, this overrides certificate authentication. There is no built-in path. See nbdkit-tls(1) for more details.

--tls-verify-peer

Enables TLS client certificate verification. The default is not to check the client's certificate.

-U SOCKET
--unix SOCKET
-U -
--unix -

Accept connections on the Unix domain socket SOCKET (which is a path).

nbdkit creates this socket, but it will probably have incorrect permissions (too permissive). If it is a problem that some unauthorized user could connect to this socket between the time that nbdkit starts up and the authorized user connects, then put the socket into a directory that has restrictive permissions.

nbdkit does not delete the socket file when it exits. The caller should delete the socket file after use (else if you try to start nbdkit up again you will get an Address already in use error).

If the socket name is - then nbdkit generates a randomly named private socket. This is useful with "CAPTIVE NBDKIT" in nbdkit-captive(1).

-u USER
--user USER

Change user to USER after starting up. A user name or numeric user ID can be used.

The server needs sufficient permissions to be able to do this. Normally this would mean starting the server up as root.

See also -g.

-v
--verbose

Enable verbose messages.

It's a good idea to use -f as well so the process does not fork into the background (but not required).

-V
--version

Print the version number of nbdkit and exit.

PLUGIN CONFIGURATION

After specifying the plugin name you can (optionally, it depends on the plugin) give plugin configuration on the command line in the form of key=value. For example:

nbdkit file file=disk.img

To list all the options supported by a plugin, do:

nbdkit --help file

To dump information about a plugin, do:

nbdkit file --dump-plugin

Magic script parameter

For some plugins, especially those supporting scripting languages like Perl, if the first plugin argument does not contain an '=' character then it is assumed to be script=value.

That allows scripting language plugins to do:

nbdkit perl foo.pl [args...]

which has the same meaning as:

nbdkit perl script=foo.pl [args...]

Shebang scripts

You can use #! to run nbdkit plugins written in most scripting languages. The file should be executable. For example:

#!/usr/sbin/nbdkit perl
sub open {
  # etc
}

(see nbdkit-perl-plugin(3) for a full example).

FILTERS

One or more filters can be placed in front of an nbdkit plugin to modify the behaviour of the plugin, using the --filter parameter. Filters can be used for example to limit requests to an offset/limit, add copy-on-write support, or inject delays or errors (for testing).

NBD     ┌─────────┐    ┌─────────┐          ┌────────┐
  client ───▶│ filter1 │───▶│ filter2 │── ─ ─ ──▶│ plugin │
 request     └─────────┘    └─────────┘          └────────┘

Several existing filters are available in the $filterdir. Use nbdkit --dump-config to find the directory name.

How to write filters is described in nbdkit-filter(3).

SIGNALS

nbdkit responds to the following signals:

SIGINT
SIGQUIT
SIGTERM

The server exits cleanly.

SIGPIPE

This signal is ignored.

ENVIRONMENT VARIABLES

LISTEN_FDS
LISTEN_PID

If present in the environment when nbdkit starts up, these trigger "SOCKET ACTIVATION" in nbdkit-service(1).

SEE ALSO

Other topics

nbdkit-captive(1) — Run nbdkit under another process and have it reliably cleaned up.

nbdkit-probing(1) — How to probe for nbdkit configuration and plugins.

nbdkit-protocol(1) — Which parts of the NBD protocol nbdkit supports.

nbdkit-service(1) — Running nbdkit as a service, and systemd socket activation.

nbdkit-tls(1) — Authentication and encryption of NBD connections (sometimes incorrectly called "SSL").

Plugins

__PLUGIN_LINKS__.

Filters

__FILTER_LINKS__.

For developers

nbdkit-plugin(3), nbdkit-filter(3).

Writing plugins in other programming languages

nbdkit-lua-plugin(3), nbdkit-ocaml-plugin(3), nbdkit-perl-plugin(3), nbdkit-python-plugin(3), nbdkit-ruby-plugin(3), nbdkit-sh-plugin(3), nbdkit-tcl-plugin(3).

NBD clients

qemu(1), nbd-client(1), guestfish(1).

http://github.com/libguestfs/nbdkit — Source code.

Other NBD servers

qemu-nbd(1), nbd-server(1), https://bitbucket.org/hirofuchi/xnbd.

Documentation for the NBD protocol

https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md, https://nbd.sourceforge.io/.

Similar protocols

https://en.wikipedia.org/wiki/iSCSI, https://en.wikipedia.org/wiki/ATA_over_Ethernet, https://en.wikipedia.org/wiki/Fibre_Channel_over_Ethernet.

Other manual pages of interest

gnutls_priority_init(3), qemu-img(1), psktool(1), systemd.socket(5).

AUTHORS

Eric Blake

Richard W.M. Jones

Yann E. MORIN

Nir Soffer

Pino Toscano

COPYRIGHT

Copyright (C) 2013-2018 Red Hat Inc.

POD ERRORS

Hey! The above document had some coding errors, which are explained below:

Around line 83:

Non-ASCII character seen before =encoding in '│'. Assuming UTF-8