djwong/xfsprogs

    
Update all the necessary files for a 4.13.1 release.
    
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

The new -d cowextsize option overran the subopts array; make it larger.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

    
Update all the necessary files for a 4.13.0 release.
    
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

    
Update all the necessary files for a 4.13.0-rc2 release.
    
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

Missed a couple of the new extent states in the bmbt processing, so add
them to avoid aborting xfs_repair.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
[sandeen: move XR_E_REFC case above fallthrough to emit both do_warn]
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

Create a -d option to mkfs.xfs that enables administrators to set
the CoW extent size hint on the created files.

[sandeen: Note, the switch to xfs_flags2diflags looks like
a bugfix, but it's not - the flags set by mkfs up to this
point just happened to line up without any translation.]

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

When I create a XFS with "rmapbt=1,reflink=1", "xfs_db -c version"
didn't show RMAPBT feature. But REFLINK can be found.

  # mkfs/mkfs.xfs -f -m rmapbt=1,reflink=1 /dev/sda3
  # db/xfs_db -c version /dev/sda3
  versionnum [0xb4a5+0x18a] = V5,NLINK,DIRV2,ALIGN,LOGV2,EXTFLG,MOREBITS,
                  ATTR2,LAZYSBCOUNT,PROJID32BIT,CRC,FTYPE,FINOBT,REFLINK

Signed-off-by: Zorro Lang <zlang@redhat.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

    
Update all the necessary files for a 4.13.0-rc1 release.
    
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

In some case, exit status of xfs_repair -n is different even for
the same file system when -v is specified or not. This patch fixes
this behavior.

If -v is specified, do_warn() is used in zero_log() for printing
a normal message. That makes the exit status to 1 though there
is no dirtiness in the file system.

Signed-off-by: Masatake YAMATO <yamato@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
[sandeen: edit changelog for brevity]
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

Add documentation for some undocumented mkfs -d and -l flags.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

Back in the old days of Irix, the library saved and restored chdir so
that (I guess?) check_open can change directory without screwing up our
ability to use relative paths.  However, there's nothing in Linux that
actually does this, so just rip out the getcwd/chdir stuff since we
absolute device paths work just fine without it.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

Fix more compiler warnings about pointless checks, unchecked return
values, brace problems, and missing parentheses.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

When the (threaded) inode scanner iterates the blocks of a bmbt tree and
wants to record the bmbt blocks in the in-core rmap database, we have to
take the ag_lock for the AG that the bmbt block is in, or else we can
accidentally corrupt the rmap slab by calling slab_add from two threads.

Reported-by: matorola@gmail.com
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Carlos Maiolino <cmaiolino@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

When sunit and swidth are used mkfs.xfs tries to avoid all allocation
groups aligning on the same stripe and will attempt to stagger them
across the stripes that make up swidth.  If there is only one stripe
then there is no benefit in this optimisation.

$ truncate -s10G xfs_10G_su256k_sw1.image
$ mkfs.xfs -d su=256k,sw=1 xfs_10G_su256k_sw1.image
meta-data=xfs_10G_su256k_sw1.image isize=512    agcount=16, agsize=163776 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=0, sparse=0
data     =                       bsize=4096   blocks=2620416, imaxpct=25
         =                       sunit=64     swidth=64 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal log           bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=64 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

A side effect of the optimisation is that the size adjustment used to stager
the allocation groups causes the last sunit of storage to be unused.

$ echo $((2620416*4096))
10733223936
$ ls -l xfs_10G_su256k_sw1.image
-rw-rw-r--. 1 test test 10737418240 Aug 30 10:54 xfs_10G_su256k_sw1.image

Skip this optimisation when sunit == swidth.

Signed-off-by: Donald Douwsma <ddouwsma@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

Source kernel commit: c44245b3d5435f533ca8346ece65918f84c057f9

When we try to allocate a free inode by searching the inobt, we try to
find the inode nearest the parent inode by searching chunks both left
and right of the chunk containing the parent. As an optimization, we
cache the leftmost and rightmost records that we previously searched; if
we do another allocation with the same parent inode, we'll pick up the
search where it last left off.

There's a bug in the case where we found a free inode to the left of the
parent's chunk: we need to update the cached left and right records, but
because we already reassigned the right record to point to the left, we
end up assigning the left record to both the cached left and right
records.

This isn't a correctness problem strictly, but it can result in the next
allocation rechecking chunks unnecessarily or allocating inodes further
away from the parent than it needs to. Fix it by swapping the record
pointer after we update the cached left and right records.

Fixes: bd169565993b ("xfs: speed up free inode search")
Signed-off-by: Omar Sandoval <osandov@fb.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

Explicitly declare that the 'start' and 'end' arguments to fsmap require
one of -d, -l, or -r to select the data, log, or realtime device, and fix
misspelled command name while we're at it.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

xfs_bmap's manpage mis-describes the behavior of the
-n option.  xfs_io's fiemap command references the xfs_bmap
manpage, and has the same problem:

-n does not change the query batch size, it limits the number
of extents displayed.

This has been true for 15+ years, so change the documentation
to match reality.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

Currently the semantics of the -n argument are a bit idiosyncratic. We want the
argument to be the limit of extents that are going to be output by the tool. This
is clearly broken now as evident from the following example on a fragmented file:

xfs_io -c "fiemap -v -n 5" test-dir/fragmented-file
test-dir/fragmented-file:
 EXT: FILE-OFFSET      BLOCK-RANGE          TOTAL FLAGS
   0: [0..15]:         hole                    16
   1: [16..23]:        897847296..897847303     8   0x0
   2: [24..31]:        hole                     8
   3: [32..39]:        897851392..897851399     8   0x0

So we want at most 5 extents printed, yet we get 4. So we always print n - 1
extents.

With this modification the output looks like:

xfs_io -c "fiemap -v -n 5" test-dir/fragmented-file
test-dir/fragmented-file:
 EXT: FILE-OFFSET      BLOCK-RANGE          TOTAL FLAGS
   0: [0..15]:         hole                    16
   1: [16..23]:        897847296..897847303     8   0x0
   2: [24..31]:        hole                     8
   3: [32..39]:        897851392..897851399     8   0x0
   4: [40..47]:        hole                     8

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
[sandeen: fix initialization of max_extents]
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

The code responsible for printing holes is scattered across 3 places:
plain print function, verbose print function and in the block handling EOF hole.
Introduce a new function factoring out the common code and replace the 3 sites
where the code is used with it. This reduces duplication and makes it apparent
when we are printing holes. No functional changes.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

last_logical and cur_extent are being passed by reference to the printing
functions and the in turn modify those variables. This makes it a bit harder to
reason about the code. So change the printing function to take those 2 arguemnts
by value and move the manipulation logic in fiemap_f. Furthermore, the printing
function now return the number of extents they have printed (either 1 or 2,
dependent on whether we've hit the -n limit). No functional changes

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

Fiemap has this rather convoluted logic to calculate the number of extents to
query. This introduces needless complexity with no real benefit. Remove
num_extents and instead hardcode the number of extents we query for in a single
go to 32. No functional changes

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

The blocksize variable was hardcoded to 512 bytes and was passed to various
functions. This introduced a lot of redundancy since we can just as well use
the BTOBBT macro. So let's do that and eliminate all usage of the blocksize var.
No functional changes.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

When pf_create_prefetch_thread fails, it tears down the args struct
and frees it.  This causes a use-after-free in prefetch_ag_range, which
then passes the now-invalid pointer to start_inode_prefetch.  The struct
is only freed when the queuing thread can't be started.  When we can't
start even one worker thread, we mark the args ready for processing and
allow it to proceed single-threaded.  Unfortunately, this only marks
the current args ready for processing and since we return immediately,
the call to pf_create_prefetch_thread at the end of pf_queuing_worker
never gets called and we wait forever for prefetch to start on the
next AG.

This patch factors out the cleanup into a new pf_skip_prefetch_thread
that is called when we fail to create either the queuing thread or
the first of the workers.  It marks the args ready for processing, marks
it done so start_inode_prefetch doesn't add another AG to the list, and
tries to start a new thread for the next AG in the list.  We also clear
->next_args and check for it in cleanup_inode_prefetch so this condition
is easier to catch should it arise again.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

When debugging prefetch failures, it's useful to have thread creation
failure messages that are output as warnings on stderr in the trace
log as well.  It's also helpful to see when an AG gets queued behind
another one rather than having the thread started directly, which
has a separate trace line.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

pf_queuing_worker and pf_create_prefetch_thread both try to handle
thread creation failure gracefully, but assume that pthread_create
doesn't modify the pthread_t when it fails.

From the pthread_create man page:
On  success,  pthread_create() returns 0; on error, it returns an error
number, and the contents of *thread are undefined.

In fact, glibc's pthread_create writes the pthread_t value before
calling clone().  When we join the created threads in
cleanup_inode_prefetch and the cleanup stage of pf_queuing_worker, we
assume that if the pthread_t is nonzero that it's a valid thread handle
and end up crashing in pthread_join.

This patch zeros out the handle after pthread_create failure.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

When seeking for data and holes, the lseek result must be greater than
or equal to the start offset.  Furthermore, assuming that the file
doesn't change under us, when switching between SEEK_HOLE and SEEK_DATA,
the seek position must increase monotonically.  Warn and abort if this
is not the case.

Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

In the main loop of fsrallfs, we exit when we've hit the timeout but
we increment fs before we get there.  If we're operating on the last
file system in the array, we'll hit an uninitialized fsdesc and
crash in fsrall_cleanup.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
[sandeen: change Jeff's for(; loop]
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

The middle and last bit flip fuzz verbs need to read the same bit that
they're trying to set.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

Add a missing underscore where it was omitted probably by a mistake.

Signed-off-by: Jan Tulak <jtulak@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

For a non-crc filesystem, make write/fuzz -c and -d work properly
instead of bailing out.  Since there's no checksum to update, both
cases collapse to setting the field value without calling the write
verifier.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Carlos Maiolino <cmaiolino@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

Fix a missed opportunity to free the field list when we fail out of the
fuzz command by refactoring the error clauses to use a common cleanup
clause.

Fixes-coverity-id: 1416141
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

On the off chance that someone runs metadump more than once with the
metadump file going to stdout and then not stdout, the stdout_metadump
variable will not be reset before the second invocation.  Clear the
status variable when we undo the stdout redirection.

Fixes-coverity-id: 1416140
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>

We can call the cursor push and pop functions directly from btdump,
so skip all the eval overhead.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@sandeen.net>