Permalink
Commits on Apr 19, 2015
  1. nilfs2-kmod7 v1.0 release

    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 18, 2015
  2. README: update versions of supported mainline kernel

    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 19, 2015
  3. README: correct spell of CentOS

    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 19, 2015
Commits on Apr 18, 2015
  1. nilfs2: improve execution time of NILFS_IOCTL_GET_CPINFO ioctl

    The older a filesystem gets, the slower lscp command becomes.  This is
    because nilfs_cpfile_do_get_cpinfo() function meets more hole blocks
    as the start offset of valid checkpoint numbers gets bigger.
    
    This reduces the overhead by skipping hole blocks efficiently with
    nilfs_mdt_find_block() helper.
    
    A measurement result of this patch is as follows:
    
    Before:
    $ time lscp
                     CNO        DATE     TIME  MODE  FLG      BLKCNT       ICNT
                 5769303  2015-02-22 19:31:33   cp    -          108          1
                 5769304  2015-02-22 19:38:54   cp    -          108          1
    
    real    0m0.182s
    user    0m0.003s
    sys     0m0.180s
    
    After:
    $ time lscp
                     CNO        DATE     TIME  MODE  FLG      BLKCNT       ICNT
                 5769303  2015-02-22 19:31:33   cp    -          108          1
                 5769304  2015-02-22 19:38:54   cp    -          108          1
    
    real    0m0.003s
    user    0m0.001s
    sys     0m0.002s
    
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 16, 2015
  2. nilfs2: add helper to find existent block on metadata file

    Add a new metadata file function, nilfs_mdt_find_block(), which finds
    an existent block on a metadata file in a given range of blocks.  This
    function skips continuous hole blocks efficiently by using
    nilfs_bmap_seek_key().
    
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 16, 2015
  3. nilfs2: add bmap function to seek a valid key

    Add a new bmap function, nilfs_bmap_seek_key(), which seeks a valid
    entry and returns its key starting from a given key.  This function
    can be used to skip hole blocks efficiently.
    
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Cc: Dan Carpenter <dan.carpenter@oracle.com>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 16, 2015
  4. nilfs2: unify type of key arguments in bmap interface

    The type of key arguments in block mapping interface varies depending
    on function.  For instance, nilfs_bmap_lookup_at_level() takes "__u64"
    for its key argument whereas nilfs_bmap_lookup() takes "unsigned
    long".
    
    This fits them to "__u64" to eliminate the variation.
    
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 16, 2015
  5. nilfs2: use bgl_lock_ptr()

    Simplify nilfs_mdt_bgl_lock() by utilizing bgl_lock_ptr() helper in
    <linux/blockgroup_lock.h>.
    
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 16, 2015
  6. nilfs2: use set_mask_bits() for operations on buffer state bitmap

    nilfs_forget_buffer(), nilfs_clear_dirty_page(), and
    nilfs_segctor_complete_write() are using a bunch of atomic bit operations
    against buffer state bitmap.
    
    This reduces the number of them by utilizing set_mask_bits() macro.
    
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    [segment.c: add reference to kern_feature.h to use set_mask_bits()]
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 16, 2015
  7. Add mimic of set_mask_bits() helper for old kernels

    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 18, 2015
  8. nilfs2: do not use async write flag for segment summary buffers

    The async write flag is introduced to nilfs2 in the commit 7f42ec394156
    ("nilfs2: fix issue with race condition of competition between segments
    for dirty blocks"), but the flag only makes sense for data buffers and
    btree node buffers.  It is not needed for segment summary buffers.
    
    This gets rid of the latter uses as part of refactoring of atomic bit
    operations on buffer state bitmap.
    
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Cc: Vyacheslav Dubeyko <slava@dubeyko.com>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 16, 2015
  9. nilfs2: use inode_set_flags() in nilfs_set_inode_flags()

    Use inode_set_flags() to atomically set i_flags instead of clearing out
    the S_IMMUTABLE, S_APPEND, etc.  flags and then setting them from the
    FS_IMMUTABLE_FL, FS_APPEND_FL flags to avoid a race where an immutable
    file has the immutable flag cleared for a brief window of time.
    
    This is a similar fix to commit 5f16f3225b06 ("ext4: atomically set
    inode->i_flags in ext4_set_inode_flags()").
    
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Cc: "Theodore Ts'o" <tytso@mit.edu>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 16, 2015
  10. Add mimic of inode_set_flags() for old kernels

    inode_set_flags() is used to atomically set inode->i_flags.
    This adds the function as an inline to make it available
    in kernel 3.15 and earlier.
    
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 17, 2015
  11. nilfs2: put out gfp mask manipulation from nilfs_set_inode_flags()

    nilfs_set_inode_flags() function adjusts gfp-mask of inode->i_mapping as
    well as i_flags, however, this coupling of operations is not appropriate.
    
    For instance, nilfs_ioctl_setflags(), one of three callers of
    nilfs_set_inode_flags(), doesn't need to reinitialize the gfp-mask at all.
     In addition, nilfs_new_inode(), another caller of
    nilfs_set_inode_flags(), doesn't either because it has already initialized
    the gfp-mask.
    
    Only __nilfs_read_inode(), the remaining caller, needs it.  So, this moves
    the gfp mask manipulation to __nilfs_read_inode() from
    nilfs_set_inode_flags().
    
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 16, 2015
  12. nilfs2: fix gcc warning at nilfs_checkpoint_is_mounted()

    Fix the following build warning:
    
     fs/nilfs2/super.c: In function 'nilfs_checkpoint_is_mounted':
     fs/nilfs2/super.c:1023:10: warning: comparison of unsigned expression < 0 is always false [-Wtype-limits]
       if (cno < 0 || cno > nilfs->ns_cno)
               ^
    
    This warning indicates that the comparision "cno < 0" is useless because
    variable "cno" has an unsigned integer type "__u64".
    
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Reported-by: David Binderman <dcb314@hotmail.com>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 16, 2015
  13. Fix build error due to removal of mapping->backing_dev_info

    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 18, 2015
  14. Fix build error due to removal of vm_ops->remap_pages()

    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 18, 2015
  15. Use d_obtain_root() and add its compat function

    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 18, 2015
  16. Fix build error due to change related to iov iterator at Linux 3.16

    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 18, 2015
  17. Fix build error due to change of bio struct at Linux 3.14

    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 18, 2015
  18. Fix build error due to change of readdir at Linux 3.11

    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 18, 2015
  19. Fix build error due to relocation of version.h

    The header file "version.h" was moved to include/generated/uapi/linux/
    at Linux 3.7 and this causes build errors for mainline kernels.  This
    patch fixes the issue.
    
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 18, 2015
Commits on Apr 5, 2015
  1. nilfs2-kmod-centos7 v0.1.5 release

    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 5, 2015
  2. Support change of truncate_pagecache() function

    The 'oldsize' argument of truncate_pagecache() was dropped by commit
    7caef26767c1 ("truncate: drop 'oldsize' truncate_pagecache()
    parameter") on the mainline, and this change was back-ported to
    RHEL/CentOS 7.1.
    
    This patch follows the change by adding nilfs_truncate_pagecache()
    wrapper which supports both truncate_pagecache() functions before and
    after the change.
    
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 5, 2015
Commits on Mar 13, 2015
  1. nilfs2-kmod-centos7 v0.1.4 release

    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Mar 13, 2015
  2. nilfs2: fix deadlock of segment constructor during recovery

    According to a report from Yuxuan Shui, nilfs2 in kernel 3.19 got stuck
    during recovery at mount time.  The code path that caused the deadlock was
    as follows:
    
      nilfs_fill_super()
        load_nilfs()
          nilfs_salvage_orphan_logs()
            * Do roll-forwarding, attach segment constructor for recovery,
              and kick it.
    
            nilfs_segctor_thread()
              nilfs_segctor_thread_construct()
               * A lock is held with nilfs_transaction_lock()
                 nilfs_segctor_do_construct()
                   nilfs_segctor_drop_written_files()
                     iput()
                       iput_final()
                         write_inode_now()
                           writeback_single_inode()
                             __writeback_single_inode()
                               do_writepages()
                                 nilfs_writepage()
                                   nilfs_construct_dsync_segment()
                                     nilfs_transaction_lock() --> deadlock
    
    This can happen if commit 7ef3ff2fea8b ("nilfs2: fix deadlock of segment
    constructor over I_SYNC flag") is applied and roll-forward recovery was
    performed at mount time.  The roll-forward recovery can happen if datasync
    write is done and the file system crashes immediately after that.  For
    instance, we can reproduce the issue with the following steps:
    
     < nilfs2 is mounted on /nilfs (device: /dev/sdb1) >
     # dd if=/dev/zero of=/nilfs/test bs=4k count=1 && sync
     # dd if=/dev/zero of=/nilfs/test conv=notrunc oflag=dsync bs=4k
     count=1 && reboot -nfh
     < the system will immediately reboot >
     # mount -t nilfs2 /dev/sdb1 /nilfs
    
    The deadlock occurs because iput() can run segment constructor through
    writeback_single_inode() if MS_ACTIVE flag is not set on sb->s_flags.  The
    above commit changed segment constructor so that it calls iput()
    asynchronously for inodes with i_nlink == 0, but that change was
    imperfect.
    
    This fixes the another deadlock by deferring iput() in segment constructor
    even for the case that mount is not finished, that is, for the case that
    MS_ACTIVE flag is not set.
    
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Reported-by: Yuxuan Shui <yshuiv7@gmail.com>
    Tested-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Cc: Al Viro <viro@zeniv.linux.org.uk>
    Cc: <stable@vger.kernel.org>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Mar 12, 2015
Commits on Mar 3, 2015
  1. nilfs2: fix potential memory overrun on inode

    Each inode of nilfs2 stores a root node of a b-tree, and it turned out to
    have a memory overrun issue:
    
    Each b-tree node of nilfs2 stores a set of key-value pairs and the number
    of them (in "bn_nchildren" member of nilfs_btree_node struct), as well as
    a few other "bn_*" members.
    
    Since the value of "bn_nchildren" is used for operations on the key-values
    within the b-tree node, it can cause memory access overrun if a large
    number is incorrectly set to "bn_nchildren".
    
    For instance, nilfs_btree_node_lookup() function determines the range of
    binary search with it, and too large "bn_nchildren" leads
    nilfs_btree_node_get_key() in that function to overrun.
    
    As for intermediate b-tree nodes, this is prevented by a sanity check
    performed when each node is read from a drive, however, no sanity check
    has been done for root nodes stored in inodes.
    
    This patch fixes the issue by adding missing sanity check against b-tree
    root nodes so that it's called when on-memory inodes are read from ifile,
    inode metadata file.
    
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Cc: <stable@vger.kernel.org>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Feb 27, 2015
Commits on Feb 7, 2015
  1. nilfs2-kmod-centos7 v0.1.3 release

    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Feb 7, 2015
  2. nilfs2: fix deadlock of segment constructor over I_SYNC flag

    Nilfs2 eventually hangs in a stress test with fsstress program.  This
    issue was caused by the following deadlock over I_SYNC flag between
    nilfs_segctor_thread() and writeback_sb_inodes():
    
      nilfs_segctor_thread()
        nilfs_segctor_thread_construct()
          nilfs_segctor_unlock()
            nilfs_dispose_list()
              iput()
                iput_final()
                  evict()
                    inode_wait_for_writeback()  * wait for I_SYNC flag
    
      writeback_sb_inodes()
         * set I_SYNC flag on inode->i_state
        __writeback_single_inode()
          do_writepages()
            nilfs_writepages()
              nilfs_construct_dsync_segment()
                nilfs_segctor_sync()
                   * wait for completion of segment constructor
        inode_sync_complete()
           * clear I_SYNC flag after __writeback_single_inode() completed
    
    writeback_sb_inodes() calls do_writepages() for dirty inodes after
    setting I_SYNC flag on inode->i_state.  do_writepages() in turn calls
    nilfs_writepages(), which can run segment constructor and wait for its
    completion.  On the other hand, segment constructor calls iput(), which
    can call evict() and wait for the I_SYNC flag on
    inode_wait_for_writeback().
    
    Since segment constructor doesn't know when I_SYNC will be set, it
    cannot know whether iput() will block or not unless inode->i_nlink has a
    non-zero count.  We can prevent evict() from being called in iput() by
    implementing sop->drop_inode(), but it's not preferable to leave inodes
    with i_nlink == 0 for long periods because it even defers file
    truncation and inode deallocation.  So, this instead resolves the
    deadlock by calling iput() asynchronously with a workqueue for inodes
    with i_nlink == 0.
    
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Cc: Al Viro <viro@zeniv.linux.org.uk>
    Tested-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Cc: <stable@vger.kernel.org>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Feb 5, 2015
Commits on Jan 17, 2015
  1. nilfs2: fix the nilfs_iget() vs. nilfs_new_inode() races

    Same story as in commit 41080b5a2401 ("nfsd race fixes: ext2") (similar
    ext2 fix) except that nilfs2 needs to use insert_inode_locked4() instead
    of insert_inode_locked() and a bug of a check for dead inodes needs to
    be fixed.
    
    If nilfs_iget() is called from nfsd after nilfs_new_inode() calls
    insert_inode_locked4(), nilfs_iget() will wait for unlock_new_inode() at
    the end of nilfs_mkdir()/nilfs_create()/etc to unlock the inode.
    
    If nilfs_iget() is called before nilfs_new_inode() calls
    insert_inode_locked4(), it will create an in-core inode and read its
    data from the on-disk inode.  But, nilfs_iget() will find i_nlink equals
    zero and fail at nilfs_read_inode_common(), which will lead it to call
    iget_failed() and cleanly fail.
    
    However, this sanity check doesn't work as expected for reused on-disk
    inodes because they leave a non-zero value in i_mode field and it
    hinders the test of i_nlink.  This patch also fixes the issue by
    removing the test on i_mode that nilfs2 doesn't need.
    
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Cc: Al Viro <viro@zeniv.linux.org.uk>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Dec 10, 2014
Commits on Oct 19, 2014
  1. nilfs2-kmod-centos7 v0.1.2 release

    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Oct 19, 2014
Commits on Oct 18, 2014
  1. nilfs2: improve the performance of fdatasync()

    Support for fdatasync() has been implemented in NILFS2 for a long time,
    but whenever the corresponding inode is dirty the implementation falls
    back to a full-flegded sync().  Since every write operation has to
    update the modification time of the file, the inode will almost always
    be dirty and fdatasync() will fall back to sync() most of the time.  But
    this fallback is only necessary for a change of the file size and not
    for a change of the various timestamps.
    
    This patch adds a new flag NILFS_I_INODE_SYNC to differentiate between
    those two situations.
    
     * If it is set the file size was changed and a full sync is necessary.
     * If it is not set then only the timestamps were updated and
       fdatasync() can go ahead.
    
    There is already a similar flag I_DIRTY_DATASYNC on the VFS layer with
    the exact same semantics.  Unfortunately it cannot be used directly,
    because NILFS2 doesn't implement write_inode() and doesn't clear the VFS
    flags when inodes are written out.  So the VFS writeback thread can
    clear I_DIRTY_DATASYNC at any time without notifying NILFS2.  So
    I_DIRTY_DATASYNC has to be mapped onto NILFS_I_INODE_SYNC in
    nilfs_update_inode().
    
    Signed-off-by: Andreas Rohner <andreas.rohner@gmx.net>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    zeitgeist87 committed with konis Oct 13, 2014
  2. nilfs2: add missing blkdev_issue_flush() to nilfs_sync_fs()

    Under normal circumstances nilfs_sync_fs() writes out the super block,
    which causes a flush of the underlying block device.  But this depends
    on the THE_NILFS_SB_DIRTY flag, which is only set if the pointer to the
    last segment crosses a segment boundary.  So if only a small amount of
    data is written before the call to nilfs_sync_fs(), no flush of the
    block device occurs.
    
    In the above case an additional call to blkdev_issue_flush() is needed.
    To prevent unnecessary overhead, the new flag nilfs->ns_flushed_device
    is introduced, which is cleared whenever new logs are written and set
    whenever the block device is flushed.  For convenience the function
    nilfs_flush_device() is added, which contains the above logic.
    
    Signed-off-by: Andreas Rohner <andreas.rohner@gmx.net>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    zeitgeist87 committed with konis Oct 13, 2014
Commits on Sep 27, 2014
  1. README: add Scientific Linux 7 to the list of supported distributions

    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Sep 27, 2014
  2. nilfs2-kmod-centos7 v0.1.1 release

    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Sep 27, 2014