borg test failure on Ubuntu yakkety armhf

[LocutusOfBorg]

Hi, not sure what is wrong, in the meanwhile I'm retrying the build and attaching the log:

arm-linux-gnueabihf-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-armv7l-3.5/borg/platform_linux.o -lacl -o /«PKGBUILDDIR»/.pybuild/pythonX.Y_3.5/build/borg/platform_linux.cpython-35m-arm-linux-gnueabihf.so
   debian/rules override_dh_auto_test
make[1]: Entering directory '/«PKGBUILDDIR»'
fakeroot dh_auto_test
I: pybuild base:184: cd /«PKGBUILDDIR»/.pybuild/pythonX.Y_3.5/build; python3.5 -m pytest -k 'not benchmark and not fuse_mount' --pyargs borg.testsuite
============================= test session starts ==============================
platform linux -- Python 3.5.2, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /«PKGBUILDDIR», inifile: setup.cfg
plugins: cov-2.2.1
collected 554 items

borg/testsuite/archive.py ...............................
borg/testsuite/archiver.py .s...............................X....F....sssssssssssssssssssssssssssssssssssssssss...............s.....................X.....F.....
borg/testsuite/chunker.py ...
borg/testsuite/compress.py ........
borg/testsuite/crypto.py .....
borg/testsuite/hashindex.py .....................
borg/testsuite/helpers.py ...............................................................................................................................................................
borg/testsuite/key.py ....
borg/testsuite/locking.py ..............
borg/testsuite/logger.py ....
borg/testsuite/lrucache.py ..
borg/testsuite/platform.py sssss
borg/testsuite/repository.py .........................................
borg/testsuite/shellpattern.py .............................................................
borg/testsuite/upgrader.py sssss..
borg/testsuite/xattr.py .

=================================== FAILURES ===================================
______________________ ArchiverTestCase.test_sparse_file _______________________

self = <borg.testsuite.archiver.ArchiverTestCase testMethod=test_sparse_file>

    def test_sparse_file(self):
        # no sparse file support on Mac OS X
        sparse_support = sys.platform != 'darwin'
        filename = os.path.join(self.input_path, 'sparse')
        content = b'foobar'
        hole_size = 5 * (1 << CHUNK_MAX_EXP)  # 5 full chunker buffers
        with open(filename, 'wb') as fd:
            # create a file that has a hole at the beginning and end (if the
            # OS and filesystem supports sparse files)
            fd.seek(hole_size, 1)
            fd.write(content)
            fd.seek(hole_size, 1)
            pos = fd.tell()
            fd.truncate(pos)
        total_len = hole_size + len(content) + hole_size
        st = os.stat(filename)
        self.assert_equal(st.st_size, total_len)
        if sparse_support and hasattr(st, 'st_blocks'):
            self.assert_true(st.st_blocks * 512 < total_len)  # is input sparse?
        self.cmd('init', self.repository_location)
        self.cmd('create', self.repository_location + '::test', 'input')
        with changedir('output'):
            self.cmd('extract', '--sparse', self.repository_location + '::test')
        self.assert_dirs_equal('input', 'output/input')
        filename = os.path.join(self.output_path, 'input', 'sparse')
        with open(filename, 'rb') as fd:
            # check if file contents are as expected
            self.assert_equal(fd.read(hole_size), b'\0' * hole_size)
            self.assert_equal(fd.read(len(content)), content)
            self.assert_equal(fd.read(hole_size), b'\0' * hole_size)
        st = os.stat(filename)
        self.assert_equal(st.st_size, total_len)
        if sparse_support:
            if hasattr(st, 'st_blocks'):
                # do only check if it is less, do NOT check if it is much less
                # as that causes troubles on xfs, zfs, ntfs:
                self.assert_true(st.st_blocks * 512 < total_len)
            if hasattr(os, 'SEEK_HOLE') and hasattr(os, 'SEEK_DATA'):
                with open(filename, 'rb') as fd:
                    # only check if the first hole is as expected, because the 2nd hole check
                    # is problematic on xfs due to its "dynamic speculative EOF preallocation
                    try:
>                       self.assert_equal(fd.seek(0, os.SEEK_HOLE), 0)
E                       AssertionError: 83886086 != 0

borg/testsuite/archiver.py:429: AssertionError
___________________ RemoteArchiverTestCase.test_sparse_file ____________________

self = <borg.testsuite.archiver.RemoteArchiverTestCase testMethod=test_sparse_file>

    def test_sparse_file(self):
        # no sparse file support on Mac OS X
        sparse_support = sys.platform != 'darwin'
        filename = os.path.join(self.input_path, 'sparse')
        content = b'foobar'
        hole_size = 5 * (1 << CHUNK_MAX_EXP)  # 5 full chunker buffers
        with open(filename, 'wb') as fd:
            # create a file that has a hole at the beginning and end (if the
            # OS and filesystem supports sparse files)
            fd.seek(hole_size, 1)
            fd.write(content)
            fd.seek(hole_size, 1)
            pos = fd.tell()
            fd.truncate(pos)
        total_len = hole_size + len(content) + hole_size
        st = os.stat(filename)
        self.assert_equal(st.st_size, total_len)
        if sparse_support and hasattr(st, 'st_blocks'):
            self.assert_true(st.st_blocks * 512 < total_len)  # is input sparse?
        self.cmd('init', self.repository_location)
        self.cmd('create', self.repository_location + '::test', 'input')
        with changedir('output'):
            self.cmd('extract', '--sparse', self.repository_location + '::test')
        self.assert_dirs_equal('input', 'output/input')
        filename = os.path.join(self.output_path, 'input', 'sparse')
        with open(filename, 'rb') as fd:
            # check if file contents are as expected
            self.assert_equal(fd.read(hole_size), b'\0' * hole_size)
            self.assert_equal(fd.read(len(content)), content)
            self.assert_equal(fd.read(hole_size), b'\0' * hole_size)
        st = os.stat(filename)
        self.assert_equal(st.st_size, total_len)
        if sparse_support:
            if hasattr(st, 'st_blocks'):
                # do only check if it is less, do NOT check if it is much less
                # as that causes troubles on xfs, zfs, ntfs:
                self.assert_true(st.st_blocks * 512 < total_len)
            if hasattr(os, 'SEEK_HOLE') and hasattr(os, 'SEEK_DATA'):
                with open(filename, 'rb') as fd:
                    # only check if the first hole is as expected, because the 2nd hole check
                    # is problematic on xfs due to its "dynamic speculative EOF preallocation
                    try:
>                       self.assert_equal(fd.seek(0, os.SEEK_HOLE), 0)
E                       AssertionError: 83886086 != 0

borg/testsuite/archiver.py:429: AssertionError
========= 55 tests deselected by '-knot benchmark and not fuse_mount' ==========
= 2 failed, 442 passed, 53 skipped, 55 deselected, 2 xpassed in 554.98 seconds =
E: pybuild pybuild:274: test: plugin distutils failed with: exit code=1: cd /«PKGBUILDDIR»/.pybuild/pythonX.Y_3.5/build; python3.5 -m pytest -k 'not benchmark and not fuse_mount' --pyargs borg.testsuite
dh_auto_test: pybuild --test -i python{version} -p 3.5 returned exit code 13
make[1]: *** [override_dh_auto_test] Error 25
debian/rules:63: recipe for target 'override_dh_auto_test' failed
make[1]: Leaving directory '/«PKGBUILDDIR»'
make: *** [build-arch] Error 2
debian/rules:46: recipe for target 'build-arch' failed
dpkg-buildpackage: error: debian/rules build-arch gave error exit status 2

[enkore]

What's the file system mounted at /tmp?

Principially this is not a critical test failure. We're aware that this test depends on completely unspecified filesystem behaviour, it was only recently adjusted to work on more FSes.

[LocutusOfBorg]

mount |grep tmp
tmpfs on /run/shm type tmpfs (rw,relatime)

this is on a qemu-based pbuilder environment, not sure if offical buildds have a different configuration

[enkore]

Hmm, strange. Normally tmpfs on Linux has no issues with this test, since it has sparse file support with page-granularity. The system doesn't have any odd page size (getconf PAGE_SIZE)?

[LocutusOfBorg]

I would wild guess 4096 but I can't be sure until I get official answers.

[LocutusOfBorg]

I have been told to check this log to see the armhf environment
https://launchpadlibrarian.net/228062603/buildlog_ubuntu-xenial-armhf.procenv_0.43-2_BUILDING.txt.gz

does this answer your questions?

[LocutusOfBorg]

so, 4096 but tmp is not under tmpfs

mounts:
  /:
    filesystem: 'rootfs'
    canonical: unknown
    type: 'rootfs'
    options: 'rw'
    pathconf:
      _PC_ALLOC_SIZE_MIN: 4096
      _PC_ASYNC_IO: n/a
      _PC_CHOWN_RESTRICTED: 1
      _PC_FILESIZEBITS: 64
      _PC_LINK_MAX: 65000
      _PC_MAX_CANON: 255
      _PC_MAX_INPUT: 255
      _PC_NAME_MAX: 255
      _PC_NO_TRUNC: 1
      _PC_PATH_MAX: 4096
      _PC_PIPE_BUF: 4096
      _PC_PRIO_IO: n/a
      _PC_REC_INCR_XFER_SIZE: n/a
      _PC_REC_MAX_XFER_SIZE: n/a
      _PC_REC_MIN_XFER_SIZE: 4096
      _PC_REC_XFER_ALIGN: 4096
      _PC_SOCK_MAXBUF: n/a
      _PC_SYMLINK_MAX: n/a
      _PC_2_SYMLINKS: 1
      _PC_SYNC_IO: n/a
      _PC_VDISABLE: 0
    device:
      major: 8
      minor: 1
    dump frequency: 0
    fsck pass number: 0
    fsid: 03aed76700000000
    optimal block size: 4096
    blocks:
      size: 1024 bytes
      total: 451883616
      used: 16832968
      free: 435050648
      available: 412096256
    files/inodes:
      total: 28696576
      used: 1010845
      free: 27685731

[enkore]

Strange... I thought rootfs == tmpfs, therefore there should be no problem. If rootfs != tmpfs then whatever rootfs is doesn't support sparse files as we expected (SEEK_HOLE => 83 MiB = EOF, no hole in the output file), but somehow the input file was still sparse. Huh? Or maybe the whatever-rootfs-really-is somehow doesn't expose/implement SEEK_HOLE/DATA correctly, since the check above that with the stat data succeeded.

[LocutusOfBorg]

BTW I'll upload in Ubuntu any patch you give me :)
I can't reproduce on my pla probably because of the qemu tmpfs mount...

[ThomasWaldmann]

@LocutusOfBorg can you try #1312?

[LocutusOfBorg]

it seems good so far! closing and I'll reopen if it doesn't migrate in a few hours.

thanks!