We want some testsuite coverage of handling non-1 .can_flush values.
The only in-tree plugin with this property historically was
nbd-standalone, but it didn't get frequently tested, and was recently
changed to no longer have that property (commit 15d4c4f).  Revive the
specific behaviors we want to guarantee into a new independent test.

Signed-off-by: Eric Blake <eblake@redhat.com>

Similarly to the sh plugin, we want to let nbdkit advertise fast zero
support when it will not be calling into our .zero.  And just like the
sh plugin, this requires that we cache a bit of information ourselves
(since it would be a layering violation to get what nbdkit has itself
cached).  Thankfully, our defaults for .can_fua are easier implement
correctly than what the sh plugin has to do.

To demonstrate this, try adding:

def can_zero(h):
    print("can_zero")
    return False

to example.py, before running

./nbdkit python plugins/python/example.py --run 'qemu-nbd --list'

Signed-off-by: Eric Blake <eblake@redhat.com>

The next patch needs to store more information per connection than
just the script's Python handle.  This patch is the mechanical
conversion that adds a struct wrapper, with no semantic change.

Signed-off-by: Eric Blake <eblake@redhat.com>

Repeatedly calling out to the plugin to determine the thread model is
not only potentially expensive (for a plugin that takes its time
answering, as is easy to do with 'nbdkit eval thread_model="sleep 1;
echo parallel" ...'), but risks confusion if the plugin changes its
answer over time.  Expose the cache variable in locks.c to the rest of
the server code so that we can learn the thread model exactly once,
with testsuite coverage to ensure we don't regress.

Signed-off-by: Eric Blake <eblake@redhat.com>

In commit c306fa9 and neighbors (v1.15.1), a concerted effort went
into caching the results of .can_FOO callbacks, with commit messages
demonstrating that a plugin with a slow callback should not have that
delay magnified multiple times.  But nothing was added to the
testsuite at the time, and with the sh and eval plugins, we still have
two scenarios where we cannot take advantage of the nbdkit cache
because we directly call things ourselves (.can_fua pre-existing since
v1.13.9, .can_fast_zero introduced in v1.15.1 right after the
cleanups) - because our language bindings unconditionally provide
.can_FOO, they have to duplicate logic that nbdkit would do for a C
plugin where .can_FOO is absent.  Fix the double call to .can_zero and
.can_flush by expanding the struct containing the handle we pass to
nbdkit, in order to cache what we need. Then enhance test-eflags.sh to
ensure we don't regress again.

Note that nbdkit does not yet cache .thread_model; that will be
addressed in the next patch.  Furthermore, we end up duplicating the
caching that nbdkit itself does (just like we are already duplicating
logic), because it would be a layering violation for us to have enough
information to call into backend_can_flush().

Fixes: 05c5eed
Fixes: 8490694
Signed-off-by: Eric Blake <eblake@redhat.com>

The next patch needs to store more information per connection than
just the script's return string.  This patch is the mechanical
conversion that adds a struct wrapper, with no semantic change.

Signed-off-by: Eric Blake <eblake@redhat.com>

This is already set in the test harness.

When renaming the global 'backend' to 'top', we missed a file, but
only when compiling for Haiku.

Fixes: 6a0dee1
Signed-off-by: Eric Blake <eblake@redhat.com>

This can be used for exporting temporary disks to thin clients, as a
kind of "remote tmpfs".

See also:
https://www.redhat.com/archives/libguestfs/2020-March/msg00134.html

If nbdkit has a bug (such as the nbd-standalone bug fixed in the
previous commit), qemu-io ends up waiting forever rather than
realizing that if the server disappears unexpectedly then qemu-io
should quit.  So add timeouts so the testsuite will flag the problem
instead of hang (tested by reordering this commit before the
previous).

It's trickier than I expected: from the command line, 'timeout 10s
qemu-io ...' works just fine, but from a script, it fails.  Why?
Because timeout defaults to creating a new process group when run
non-interactively, but qemu-io is generally an interactive program
that expects to be able to modify the terminal settings when stdin is
a terminal, even when it will not be reading from stdin because of the
use of -c.  When stdin is inherited but no longer the controlling
terminal because of timeout's new process group, qemu-io fails to make
progress. As solutions, we can either use 'timeout --foreground', or
redirect stdin to something that is not a terminal; this patch uses
the latter.  Since the timeout is a last-ditch effort that won't
trigger in normal successful usage, we can make it long: 30s for
probing qemu-io, and 60s for the actual tests (when io is slower due
to our use of --filter=delay).

Signed-off-by: Eric Blake <eblake@redhat.com>

Although nbdkit documents that any positive value should be treated as
success to the .can_* callbacks, we had a window of releases where
anything other than 1 could trigger an assertion failure, fixed in the
previous patch.  Update what we return to avoid tripping the bug in
broken nbdkit.

Our return value has been latent since 0e8e8eb (v1.1.17), but the
assertion failures did not occur until c306fa9 and neighboring
commits (v1.15.1).  As v1.13.6 was the point where we started
preferring builds against libnbd, where we always returned 1 on
success, the problem was not detected until now; but it is still in
the wild for any user mixing old plugins with new libnbd.

Reported-by: Svenne Krap (https://svenne.dk)
Signed-off-by: Eric Blake <eblake@redhat.com>

The documentation for .can_write and friends mentioned merely that the
callback must return a boolean value, or -1 on failure.  Historically,
we have treated any non-zero value other than -1 as true, and some
plugins have sloppily relied on this behavior, including the
standalone nbd plugin prior to the use of libnbd in v1.14.  But in
1.15.1, we added an assert in our backend code if an enabled feature
bit is not exactly 1.  To avoid tripping the assertion, we have to
normalize the plugin values before storing our internal value.

Fixes: c306fa9
Reported-by: Svenne Krap (https://svenne.dk)
Signed-off-by: Eric Blake <eblake@redhat.com>

Commit 39b4075 works if you rely on 'make clean' after a git
update, but not if you rely on incremental 'make'.  Git likes to
update files by creating new inodes, which strands hard links pointing
at their older contents.  Switch to symlinks, and expand the rules to
ensure updated timestamps any time git updates the original.

Fixes: 45b4877
Signed-off-by: Eric Blake <eblake@redhat.com>

The valgrind tests (‘make check-valgrind’) have always been a little
unstable.  This seems to be because the close callback isn't always
called in time before the server is killed, since everything runs
quite a lot more slowly under valgrind.  Adding a small sleep here
appears to help.

Created when using './configure -C'.

Signed-off-by: Eric Blake <eblake@redhat.com>

This broke during a bisect because the old copy of methods.c was left
around, which lacked sh_get_ready.  Make sure ‘make clean’ removes
these generated files.

Fixes commit 45b4877.

Rebuild the OCaml test plugin completely if the OCaml NBDKit module
changes at all.  Previously we only checked if the interface
(NBDKit.mli) changed, but we must also check if the implementation
(NBDKit.ml) changes.

Adds bindings for:
  nbdkit_realpath
  nbdkit_nanosleep
  nbdkit_export_name
  nbdkit_shutdown

When displaying OCaml sources it's most natural to show
  *.mli (interface)
  *.ml (implementation)
  *.c (native code).

Add extra lines to the git.orderfile to use this ordering.  It doesn't
change the ordering for non-OCaml code.

Plugins and filters may call this to initiate an asynchronous shutdown
of the server.  This would only be used in the connected phase —
plugins should still call exit(3) directly for configuration failure.

It is equivalent to sending a kill signal to self, but it's cleaner to
have an API for this and better for potential future non-POSIX platforms.

This is testing the delay filter, and we want to use the name
‘test-shutdown’ in a later commit to test the new nbdkit_shutdown()
API, so let’s rename it.

The code was very inconsistent about checking if top is NULL.  I
believe it's safe for us to only check it once with the connection
locked and return if it's NULL (ie. if the server is being
asynchronously shut down).

I checked this with valgrind, and although it doesn't fix the three
valgrind failures we have, it also doesn't cause any more problems.

Updates commit f492d96.