This always filled with zero rather than the requested byte.  It is
easy to demonstrate this using:

  $ nbdkit data '1*6' -D data.AST=1 \
           --run 'nbdcopy $uri - | hexdump -C' allocator=malloc
  00000000  00 00 00 00 00 00                                 |......|

This was broken since the malloc allocator was introduced in 1.21.9.

Fixes: commit 87ea585
(cherry picked from commit b81da0b)

../../src/libguestfs.org/nbdkit/nbdkit.go:532:22: cannot use struct_size (type _Ctype_ulong) as type _Ctype_ulonglong in assignment
../../src/libguestfs.org/nbdkit/nbdkit.go:566:41: cannot use struct_size (type _Ctype_ulong) as type _Ctype_uint in argument to _Cfunc__CMalloc

(cherry picked from commit 4457156)

Missed in 9d46a95.

Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit ccaf81f)

Fix everything that flake8 complains about (mostly whitespace changes
and deleting unused imports).

Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit 3aae7ba)

Using a py linter on a .sh test that invokes nbdsh is not practical,
so I probably missed some style changes.  However, it's pretty
mechanical to fix our testsuite to consistently prefer 'method()' over
'method ()' when it comes to python snippets; likewise, a consistent
pattern of handling expected exceptions is easier to follow.

Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit 9d46a95)

Using 'nbdsh -u $bad_uri' will cause nbdsh 1.4 to leak an nbd.Error
excpetion all the way out of python, which in turn triggers ABRT with
python3-abrt-addon to potentially flag a program crash.  Also, our
logic to test for expected failure is rather convoluted.  Better is to
avoid -u, and instead catch the expected exception ourselves, so that
we can rely on successful exit as our witness that the export did not
exist.

Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit 2053002)

Python prefers 'method()' over 'method ()'; our documentation was
inconsistent on which style we presented, so this scrubs all .pod
files for any embedded python snippets.

Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit 6d50b02)

(cherry picked from commit 13b41bd)

Fixes: commit fa3aabc
(cherry picked from commit 197e87d)

Make allocators a little bit more independent.  They now register
themselves with the allocator code at start up.

This is just refactoring and should change nothing about how it works.
Unfortunately it does require every call to go through an extra
indirection although I guess that doesn't really matter.  It would be
possible to optimize this by copying the whole function table into
‘struct allocator’.

(cherry picked from commit fa3aabc)

(cherry picked from commit a8fc5b8)

With the data plugin it is possible to create subpatterns which are
larger than a uint32_t.  This causes the plugin to silently overflow
when doing operations like '*'.  eg. This impossible operation worked
before:

$ ./nbdkit data '
           1 *2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2
             *2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2 ' \
           --run 'qemu-img info $nbd'
image: nbd://localhost:10809
file format: raw
virtual size: 1 TiB (1099511627776 bytes)

Simply changing all the functions to take uint64_t is sufficient.  The
data plugin is already passing 64 bit ints for counts, and the memory
plugin uses 32 bit ints which will be extended because the functions
are prototyped.

Thanks: Eric Blake
(cherry picked from commit 058be0a)

When blitting sparse/zero pages into the allocator, test if they are
zero (expected to be a common case) and free the page if so.

We could actually do even better here by using the extents information
from the other allocator to skip the copying entirely, but the
implementation of that would be much more complex.

(cherry picked from commit 0ff4b38)

When implementing the new parser this was reimplemented incorrectly so
it moved forwards rather than backwards, and also didn't test bounds
correctly.  Note there was no test before of either @+N or @-N so we
didn't catch this.  In this commit I have added a simple test.

Fixes: commit eb0062d
(cherry picked from commit 25f1e85)

[For the stable-1.22 branch this only adds the test from the original commit.]

(cherry picked from commit dc6b6b1)

Both the old and new parsers supported \x00 within strings, but we
never tested it before.

(cherry picked from commit 7e8a296)

(cherry picked from commit c0c60c4)

This should have no effect.

(cherry picked from commit ee8f1d0)

Improves readability.

(cherry picked from commit 0562fa1)

Fixes: commit dcba0f0
(cherry picked from commit e130ced)

I inadvertently copied too much text when adding support for '-' in
.name.

Thanks: Nikolay Ivanets
Fixes: e85b5ef
Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit 7db9268)

(cherry picked from commit 569182c)

On Windows the feature() function failed because of improper quoting.
This was only cosmetic.

    TLS .................................... ./configure: line 21827: test: too many arguments

(cherry picked from commit 3683e80)

Actually NULL works fine on both Linux and Windows, but RTLD_DEFAULT
is what is documented.

(cherry picked from commit 6fde747)

(cherry picked from commit b901f7d)

test-offset-truncate.sh intentionally provokes out-of-bound reads
under the shell function expect_fail; but does so in a way that causes
nbdsh to die with a python stack dump.  Abrt then picked this up and
tried to report it as an application crash.  Improve the test by
gracefully letting python exit with non-zero status without a crash.

Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit e13048f)

These were largely implemented already by Alan Somers in nbdkit 1.22.

Updates: commit 666ec24
(cherry picked from commit 01865a8)

Fixes: commit d9faa09
(cherry picked from commit dccf16e)

This test used libnbd before.  However it only tested pread and
nothing else so it was hardly a useful test.  This test didn't run on
Windows (since no libnbd).  Rewrite the test to use nbdsh, and at the
same time extend it to test a few useful calls.

(cherry picked from commit d9faa09)

Although unlikely, if pthread_create fails, we shouldn't attempt to
report an error from libnbd.

Fixes: ab7760f
Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit 1fbcae3)

nbdkit calls .preconnect prior to the client actually requesting any
export name, so it is always going to be NULL at the point of the
query.  As this is a pointless parameter that has always been empty,
we can drop it without breaking existing sh scripts.

Fixes: 01049ae
Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit f53e20f)

Due to a semantic merge conflict (competing introduction of fadvise=
vs. dir= configuration parameters), a failure in posix_fadvise while
in dir mode would pass NULL to a %s format.  While glibc handles this
fine, it is not portable.

Fixes: 5c1f4d6
Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit d5b32ec)

If the OCaml plugin was not built then this test cannot run.

Fixes: commit 7869004
(cherry picked from commit fbec2db)

Clear up wording about tls-fallback filter.

Signed-off-by: Eric Blake <eblake@redhat.com>
(cherry picked from commit 44903fe)

Fixes: commit 3a46eb1