djwong/xfstests

…very

Add a couple of tests to check that we don't leak inodes or dquots
if CoW recovery fails and therefore the mount fails.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Several tests uses both _filter_test_dir and _filter_scratch
concatenated by pipe to filter $TEST_DIR and $SCRATCH_MNT. However,
this would fail if the shorter string is a substring of the other
(like "/mnt" and "/mnt2").

This patch introduces new common filter function to safely call both
_filter_test_dir and _filter_scratch, and update tests and functions
to use this new function.

I checked this with btrfs/029, generic/409,410,411, and
generic/381,383, xfs/106,108 (which calls _filter_quota). Thanks
Eryu for advice.

[eguan: folded 2nd patch into 1st patch and update commit log a bit]

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

Linux filesystems generally treat filenames and extended attribute
keys as a bag of bytes, which means that there can be unique
sequences of bytes that render the same on most modern GUIs.  So,
let's rig up a test to see if it's really true that we can create
filenames and xattrs that look the same but point to different
files.  xfs_scrub will warn about these kinds of situations, though
they're not technically fs "corruption".

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

Make sure that we update the rmapbt correctly when we collapse-range
a file and the extents on both sides of the hole can be merged.  We
can construct this pretty trivially with insert-range and write, so
test that too.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

The size of the structure used to retrieve per-AG reserved blocks
status has changed (it's not in a released upstream), so update
xfs/122.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

The name of the xfs_io scrub subcommand to test for the existence of
the ioctl has been changed to 'test' from 'dummy', so fix xfstests.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

In generic/173, we try to force a CoW to a mmap'd region to fail if
there's no space to actually stage the CoW operation.  That failure
comes in the form of a SIGBUS to xfs_io.  If the tester just happens
to have a nonzero coresize ulimit set, a core dump is generated and
the test is marked as having failed, even though the dump generation
is exactly the correct behavior.

Therefore, set the coresize ulimit to zero while calling
_mwrite_byte.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

XFS_IOC_DIOINFO is only used for XFS, but fsstress use it to get
DIO aligned size. If XFS_IOC_DIOINFO returns error, then stop
doing any DIO related test (dread/dwrite/aread/awrite etc). That
means we never do DIO related test on other filesystems by fsstress.

The real minimal dio size is really not so important for DIO test
in fsstress. The multiple of real min dio size is fine too. I think
the stat.st_blksize get from stat() system call can be used to be
a fake minimal dio size, if XFS_IOC_DIOINFO fails (not supported).

Note that the equation about d_maxiosz is copied from kernel
XFS_IOC_DIOINFO ioctl source code:

  case XFS_IOC_DIOINFO: {
    ...

    da.d_mem =  da.d_miniosz = target->bt_logical_sectorsize;
    da.d_maxiosz = INT_MAX & ~(da.d_miniosz - 1);

    ...
  }

[eguan: update commit log add d_maxiosz reference]

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

This adds a regression test for the following kernel patch:

commit 42d4a99b09cb ("ext4: fix fault handling when mounted with -o
dax,ro")

The above patch fixes an issue with ext4 where executables cannot be
run on read-only filesystems mounted with the DAX option.

This issue does not appear to be present in ext2 or XFS, as they
both pass the test.  I've also confirmed outside of the test that
they are both indeed able to execute binaries on read-only DAX
mounts.

Thanks to Randy Dodgen for the bug report and reproduction steps.

Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Randy Dodgen <rdodgen@gmail.com>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

[eguan: fix conflicts with patch "fsx: Fix -Wformat-security
warnings"]

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

In this test, the cleaner thread deletes the directory trees created
by fsstress in order to exercise the free inode btree code.
However, if fsstress dies, the cleaner can end up waiting forever
for a directory that will never be created, which hangs up the test
run. Therefore, abort if fsstress has ended.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eryu Guan <eguan@redhat.com>

These two tests simulate log failure during a reflink operation.
However, the contents of the target of the reflink operation depend
on the block size, so we cannot hardcode md5 hashes in this test.
Since the whole point of the test is to ensure that the the complex
chain of transactions actually finishes no matter where the
interruption, it is sufficient simply to run the usual end-of-test
fsck to look for corrupt metadata.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

xfs/095 fails on 4k hard sector size device, due to it runs:

  _mkfs_log "-l version=1 -m crc=0 -d sectsize=512"

So _notrun if SCRATCH_DEV's sector size is bigger than 512b.

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

Some distros (NixOS) have their build environment enable
-Werror=format-security by default for security/hardening reasons.
Currently fsx fails to build due to this:

fsx.c: In function 'prt':
fsx.c:215:18: error: format not a string literal and no format arguments [-Werror=format-security]
  fprintf(stdout, buffer);
                  ^
fsx.c:217:20: error: format not a string literal and no format arguments [-Werror=format-security]
   fprintf(fsxlogf, buffer);
                    ^
Indeed the compiler is correct here, if the message-to-be-printed were
to contain a '%', unpredictable things would happen. Fix this.

Signed-off-by: Tuomas Tynkkynen <tuomas.tynkkynen@iki.fi>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

I can't recall anymore what exactly I did to have tests using t_mtab
to fail, but nevertheless this commit adds the proper newlines.

Signed-off-by: Tuomas Tynkkynen <tuomas.tynkkynen@iki.fi>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

When command line arg -P <dirpath> is used, compose the
path for .fsxgood .fsxlog .fsxops files from dirpath and
work file basename.

This fix is ported from LTP.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

Not sure why, but with initstate()/setstate(), fsx generates
same events regadless of the input seed argument.

Change to use srandom() to fix the problem.

Add pid to auto random seed, so parallel fsx executions with auto
seed will use different seed values.

At this time there are 6 tests that use fsx, out of which:
2 use -S 0 as seed (gettime()) - generic/{075,112}
2 do not specify seed (default = 1) - generic/{091,263}
1 uses explicit constant seed - generic/127
1 uses explicit $RANDOM seed - generic/231

This change affects all those tests.
The tests that intended to randomize the seed will now really
randomize the seed.
The tests that intended to use a constant seed will still use
a constant seed, but resulting event sequence will be different
than before this change.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

When _require_xfs_io_command is passed command parameters,
the resulting error from invalid parameters may be ignored.

For example, the following bogus params would not abort the test:
_require_xfs_io_command "falloc" "-X"
_require_xfs_io_command "fiemap" "-X"

Fix this by looking for the relevant error message.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

When mixing buffered reads and asynchronous direct writes, it is
possible to end up with the situation where we have stale data in
the page cache while the new data is already written to disk.

This issue should be fixed by patch titled:

fs: Fix page cache inconsistency when mixing buffered and AIO DIO

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

If generic/437 is run before generic/420, the latter fails with:

  4c4
  < stat.size = 2048
  ---
  > stat.size = 2097152

because both use $TEST_DIR/testfile.  generic/437 leaves it at 2M,
while generic/420 assumes that it is empty (or at least smaller than
2048 bytes).

Use a private test file (testfile.$seq) and truncate it on open just in
case.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

lvm utility in Ubuntu 14.04 LTS treats -l 100%FREE as a hard number
and not as an approximate upper limit.  With ~5G scratch partition
and ~128M scsi_debug device, vg_108 is 1279+31=1310 extents long,
but only 31*2=62 can be allocated with -i 2:

  # lvm lvcreate -i 2 -I 4m -l 100%FREE -n lv_108 vg_108
  Insufficient suitable allocatable extents for logical volume lv_108: 1248 more required

lvm2 commit 4b6e3b5e5ea6 ("allocation: Allow approximate
allocation when specifying size in percent") made '-l 100%FREE'
possible when creating RAID LVs or setting number of stripes.

Fix it by setting the size to allocate to 100M, which is enough for
the test with 128M scsi_debug device.

[eguan: update commit log a bit to mention the lvm2 commit that
changed the lvcreate behavior]

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

This is obviously wrong and makes ./check -r skip over tests on ext4
with "ext4 on $DEV not configured with metadata journaling".

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

See what happens when we fuzz every field of a quota information structure.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

When we're creating a populated xfs image, turn on quotas so that we can
fuzz those fields too.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

Ensure that the fuzz command does what it says.

[eguan: fixed test failures on non-CRC XFS]

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

As part of upstreaming, the xfs_db fuzz command change the help output
which breaks the fuzzers' ability to detect fuzz verbs.  Fix that.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

XFS is susceptible to log recovery problems if the fs crashes under
certain circumstances. If the tail has been pinned for long enough
to the log to fill and the next batch of log buffer submissions
happen to fail, the filesystem shuts down having potentially
overwritten part of the range between the last good tail->head range
in the log. This causes log recovery to fail with crc mismatch or
invalid log record errors.

Add a test that uses XFS DEBUG mode error injection to force the
tail overwrite condition with a known bad (crc mismatch) log write
and tests that log recovery succeeds. Note that this problem is
currently only reproducible with larger (non-default) log buffer
sizes (i.e., '-o logbsize=256k') or smaller block sizes (1k).

Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

- Upper/lower mismatch
- Index/upper mismatch

With index=on, lowerdir and upperdir are verified using a file
handle stored in trusted.overlay.origin xattr in upperdir and
indexdir.

Failure to verify lowerdir/upperdir on mount results in ESTALE.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

I catch this following error from dmesg when this testcase fails.

[17446.661127] Buffer I/O error on dev sdb1, logical block 64, async page read

We expect to inject disk IO errors on the device when xfs_io reads
the specific file, but other processes may trigger IO error earlier.
So, we can use task-filter to solve this problem.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

Test that an incremental send/receive operation will not fail when the
destination filesystem has compression enabled and the source filesystem
has a 4K extent at a file offset 0 that is not compressed and that is
shared.

This currently fails on btrfs and is fixed by the following patch for the
linux kernel:

  "Btrfs: incremental send, fix emission of invalid clone operations"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

1) This test can check if setting types causes error regardless of
   supporting crc, so we can update existed comments about it.  We
   also add new comments about known issues triggered in this test.

2) When finobt is disabled, xfs_db fails to get current address of
   free_root, as below:
   xfs_db -c "agi" -c "addr free_root" -c "daddr" /dev/sda11
   Metadata CRC error detected at xfs_inobt block 0x0/0x1000
   ...
   Running related tests without finobt makes no sense, so we add
   check for finobt.

Signed-off-by: Xiao Yang <yangx.jy@cn.fujitsu.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

As posix standard, if the file offset is at or past the end of file,
no bytes are read, and read() returns zero. There was a bug, when
DIO read offset is just past the EOF a little, but in the same block
with EOF, read returns different negative values.

Kernel commit 74cedf9b6c60 ("direct-io: Fix negative return from dio
read beyond eof") and commit 2d4594acbf6d ("fix the regression from
"direct-io: Fix negative return from dio read beyond eof"") fixed
the bug.

This case reads from range within EOF, past EOF and at EOF, to make
sure the return value as expected, especially read from past/at EOF
returns 0.

[eguan: update commit log and comments about information of the
specific bug, adjust read_test param order (offset, count, ret) and
test description]

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

Signed-off-by: Ernesto A. Fernández <ernesto.mnd.fernandez@gmail.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>

Refactor the XFS error injection helpers to use the new errortag
interface to configure error injection.  If that isn't present, fall
back either to the xfs_io/ioctl based injection or the older sysfs
knobs.  Refactor existing testcases to use the new helpers.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eryu Guan <eguan@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>