Permalink
Commits on Oct 19, 2014
  1. nilfs2-kmod-centos7 v0.1.2 release

    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Oct 19, 2014
Commits on Oct 18, 2014
  1. nilfs2: improve the performance of fdatasync()

    Support for fdatasync() has been implemented in NILFS2 for a long time,
    but whenever the corresponding inode is dirty the implementation falls
    back to a full-flegded sync().  Since every write operation has to
    update the modification time of the file, the inode will almost always
    be dirty and fdatasync() will fall back to sync() most of the time.  But
    this fallback is only necessary for a change of the file size and not
    for a change of the various timestamps.
    
    This patch adds a new flag NILFS_I_INODE_SYNC to differentiate between
    those two situations.
    
     * If it is set the file size was changed and a full sync is necessary.
     * If it is not set then only the timestamps were updated and
       fdatasync() can go ahead.
    
    There is already a similar flag I_DIRTY_DATASYNC on the VFS layer with
    the exact same semantics.  Unfortunately it cannot be used directly,
    because NILFS2 doesn't implement write_inode() and doesn't clear the VFS
    flags when inodes are written out.  So the VFS writeback thread can
    clear I_DIRTY_DATASYNC at any time without notifying NILFS2.  So
    I_DIRTY_DATASYNC has to be mapped onto NILFS_I_INODE_SYNC in
    nilfs_update_inode().
    
    Signed-off-by: Andreas Rohner <andreas.rohner@gmx.net>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    zeitgeist87 committed with konis Oct 13, 2014
  2. nilfs2: add missing blkdev_issue_flush() to nilfs_sync_fs()

    Under normal circumstances nilfs_sync_fs() writes out the super block,
    which causes a flush of the underlying block device.  But this depends
    on the THE_NILFS_SB_DIRTY flag, which is only set if the pointer to the
    last segment crosses a segment boundary.  So if only a small amount of
    data is written before the call to nilfs_sync_fs(), no flush of the
    block device occurs.
    
    In the above case an additional call to blkdev_issue_flush() is needed.
    To prevent unnecessary overhead, the new flag nilfs->ns_flushed_device
    is introduced, which is cleared whenever new logs are written and set
    whenever the block device is flushed.  For convenience the function
    nilfs_flush_device() is added, which contains the above logic.
    
    Signed-off-by: Andreas Rohner <andreas.rohner@gmx.net>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    zeitgeist87 committed with konis Oct 13, 2014
Commits on Sep 27, 2014
  1. README: add Scientific Linux 7 to the list of supported distributions

    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Sep 27, 2014
  2. nilfs2-kmod-centos7 v0.1.1 release

    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Sep 27, 2014
  3. nilfs2: fix data loss with mmap()

    This bug leads to reproducible silent data loss, despite the use of
    msync(), sync() and a clean unmount of the file system.  It is easily
    reproducible with the following script:
    
      ----------------[BEGIN SCRIPT]--------------------
      mkfs.nilfs2 -f /dev/sdb
      mount /dev/sdb /mnt
    
      dd if=/dev/zero bs=1M count=30 of=/mnt/testfile
    
      umount /mnt
      mount /dev/sdb /mnt
      CHECKSUM_BEFORE="$(md5sum /mnt/testfile)"
    
      /root/mmaptest/mmaptest /mnt/testfile 30 10 5
    
      sync
      CHECKSUM_AFTER="$(md5sum /mnt/testfile)"
      umount /mnt
      mount /dev/sdb /mnt
      CHECKSUM_AFTER_REMOUNT="$(md5sum /mnt/testfile)"
      umount /mnt
    
      echo "BEFORE MMAP:\t$CHECKSUM_BEFORE"
      echo "AFTER MMAP:\t$CHECKSUM_AFTER"
      echo "AFTER REMOUNT:\t$CHECKSUM_AFTER_REMOUNT"
      ----------------[END SCRIPT]--------------------
    
    The mmaptest tool looks something like this (very simplified, with
    error checking removed):
    
      ----------------[BEGIN mmaptest]--------------------
      data = mmap(NULL, file_size - file_offset, PROT_READ | PROT_WRITE,
                  MAP_SHARED, fd, file_offset);
    
      for (i = 0; i < write_count; ++i) {
            memcpy(data + i * 4096, buf, sizeof(buf));
            msync(data, file_size - file_offset, MS_SYNC))
      }
      ----------------[END mmaptest]--------------------
    
    The output of the script looks something like this:
    
      BEFORE MMAP:    281ed1d5ae50e8419f9b978aab16de83  /mnt/testfile
      AFTER MMAP:     6604a1c31f10780331a6850371b3a313  /mnt/testfile
      AFTER REMOUNT:  281ed1d5ae50e8419f9b978aab16de83  /mnt/testfile
    
    So it is clear, that the changes done using mmap() do not survive a
    remount.  This can be reproduced a 100% of the time.  The problem was
    introduced in commit 136e8770cd5d ("nilfs2: fix issue of
    nilfs_set_page_dirty() for page at EOF boundary").
    
    If the page was read with mpage_readpage() or mpage_readpages() for
    example, then it has no buffers attached to it.  In that case
    page_has_buffers(page) in nilfs_set_page_dirty() will be false.
    Therefore nilfs_set_file_dirty() is never called and the pages are never
    collected and never written to disk.
    
    This patch fixes the problem by also calling nilfs_set_file_dirty() if the
    page has no buffers attached to it.
    
    [akpm@linux-foundation.org: s/PAGE_SHIFT/PAGE_CACHE_SHIFT/]
    Signed-off-by: Andreas Rohner <andreas.rohner@gmx.net>
    Tested-by: Andreas Rohner <andreas.rohner@gmx.net>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Cc: <stable@vger.kernel.org>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    zeitgeist87 committed with konis Sep 25, 2014
Commits on Jul 8, 2014
  1. nilfs2-kmod-centos7 v0.1 release

    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Jul 8, 2014
  2. Add d_count helper function for kernel-3.10

    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Jul 8, 2014
  3. helper for reading ->d_count

    Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Al Viro committed with konis Jul 5, 2013
  4. mm + fs: store shadow entries in page cache

    Reclaim will be leaving shadow entries in the page cache radix tree upon
    evicting the real page.  As those pages are found from the LRU, an
    iput() can lead to the inode being freed concurrently.  At this point,
    reclaim must no longer install shadow pages because the inode freeing
    code needs to ensure the page tree is really empty.
    
    Add an address_space flag, AS_EXITING, that the inode freeing code sets
    under the tree lock before doing the final truncate.  Reclaim will check
    for this flag before installing shadow pages.
    
    Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
    Reviewed-by: Rik van Riel <riel@redhat.com>
    Reviewed-by: Minchan Kim <minchan@kernel.org>
    Cc: Andrea Arcangeli <aarcange@redhat.com>
    Cc: Bob Liu <bob.liu@oracle.com>
    Cc: Christoph Hellwig <hch@infradead.org>
    Cc: Dave Chinner <david@fromorbit.com>
    Cc: Greg Thelen <gthelen@google.com>
    Cc: Hugh Dickins <hughd@google.com>
    Cc: Jan Kara <jack@suse.cz>
    Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
    Cc: Luigi Semenzato <semenzato@google.com>
    Cc: Mel Gorman <mgorman@suse.de>
    Cc: Metin Doslu <metin@citusdata.com>
    Cc: Michel Lespinasse <walken@google.com>
    Cc: Ozgun Erdogan <ozgun@citusdata.com>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Roman Gushchin <klamm@yandex-team.ru>
    Cc: Ryan Mallon <rmallon@gmail.com>
    Cc: Tejun Heo <tj@kernel.org>
    Cc: Vlastimil Babka <vbabka@suse.cz>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    hnaz committed with konis Apr 3, 2014
  5. Add kern_feature.h header file

    This file evens up kernel differences depending on versions and
    distributions.
    
    This initial version of kern_feature.h includes declaration of
    truncate_inode_pages_final() function which is added to kernel 3.15
    and backported to RHEL 7 GA kernel.
    
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Jul 8, 2014
  6. nilfs2: update project's web site in nilfs2.txt

    Project's web site was moved to nilfs.sourceforge.net from
    www.nilfs.org.  This updates the site information in
    Documentation/filesystems/nilfs2.txt with the new location.
    
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 3, 2014
  7. nilfs2: verify metadata sizes read from disk

    Add code to check sizes of on-disk data of metadata files such as inode
    size, segment usage size, DAT entry size, and checkpoint size.  Although
    these sizes are read from disk, the current implementation doesn't check
    them.
    
    If these sizes are not sane on disk, it can cause out-of-range access to
    metadata or memory access overrun on metadata block buffers due to
    overflow in sundry calculations.
    
    Both lower limit and upper limit of metadata sizes are verified to
    prevent these issues.
    
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Cc: Andreas Rohner <andreas.rohner@gmx.net>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Apr 3, 2014
  8. nilfs2: add FITRIM ioctl support for nilfs2

    Add support for the FITRIM ioctl, which enables user space tools to
    issue TRIM/DISCARD requests to the underlying device.  Every clean
    segment within the specified range will be discarded.
    
    Signed-off-by: Andreas Rohner <andreas.rohner@gmx.net>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    zeitgeist87 committed with konis Apr 3, 2014
  9. nilfs2: add nilfs_sufile_trim_fs to trim clean segs

    Add nilfs_sufile_trim_fs(), which takes an fstrim_range structure and
    calls blkdev_issue_discard for every clean segment in the specified
    range.  The range is truncated to file system block boundaries.
    
    Signed-off-by: Andreas Rohner <andreas.rohner@gmx.net>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    zeitgeist87 committed with konis Apr 3, 2014
  10. nilfs2: implementation of NILFS_IOCTL_SET_SUINFO ioctl

    With this ioctl the segment usage entries in the SUFILE can be updated
    from userspace.
    
    This is useful, because it allows the userspace GC to modify and update
    segment usage entries for specific segments, which enables it to avoid
    unnecessary write operations.
    
    If a segment needs to be cleaned, but there is no or very little
    reclaimable space in it, the cleaning operation basically degrades to a
    useless moving operation.  In the end the only thing that changes is the
    location of the data and a timestamp in the segment usage information.
    With this ioctl the GC can skip the cleaning and update the segment
    usage entries directly instead.
    
    This is basically a shortcut to cleaning the segment.  It is still
    necessary to read the segment summary information, but the writing of
    the live blocks can be skipped if it's not worth it.
    
    [konishi.ryusuke@lab.ntt.co.jp: add description of NILFS_IOCTL_SET_SUINFO ioctl]
    Signed-off-by: Andreas Rohner <andreas.rohner@gmx.net>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    zeitgeist87 committed with konis Apr 3, 2014
  11. nilfs2: add nilfs_sufile_set_suinfo to update segment usage

    Introduce nilfs_sufile_set_suinfo(), which expects an array of
    nilfs_suinfo_update structures and updates the segment usage information
    accordingly.
    
    This is basically a helper function for the newly introduced
    NILFS_IOCTL_SET_SUINFO ioctl.
    
    [konishi.ryusuke@lab.ntt.co.jp: use put_bh() instead of brelse() because we know bh != NULL]
    Signed-off-by: Andreas Rohner <andreas.rohner@gmx.net>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    zeitgeist87 committed with konis Apr 3, 2014
  12. nilfs2: add struct nilfs_suinfo_update and flags

    Add the nilfs_suinfo_update structure, which contains the information
    needed to update one segment usage entry.  The flags specify, which
    fields need to be updated.
    
    Signed-off-by: Andreas Rohner <andreas.rohner@gmx.net>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    zeitgeist87 committed with konis Apr 3, 2014
  13. nilfs2: add comments for ioctls

    Add comments for ioctls in fs/nilfs2/ioctl.c file and describe NILFS2
    specific ioctls in Documentation/filesystems/nilfs2.txt.
    
    Signed-off-by: Vyacheslav Dubeyko <slava@dubeyko.com>
    Reviewed-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Cc: Wenliang Fan <fanwlexca@gmail.com>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    dubeyko committed with konis Jan 23, 2014
  14. ] nilfs2: use atomic64_t type for inodes_count and blocks_count field…

    …s in nilfs_root struct
    
    The cp_inodes_count and cp_blocks_count are represented as __le64 type in
    on-disk structure (struct nilfs_checkpoint).  But analogous fields in
    in-core structure (struct nilfs_root) are represented by atomic_t type.
    
    This patch replaces atomic_t on atomic64_t type in representation of
    inodes_count and blocks_count fields in struct nilfs_root.
    
    Signed-off-by: Vyacheslav Dubeyko <slava@dubeyko.com>
    Acked-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Acked-by: Joern Engel <joern@logfs.org>
    Cc: Clemens Eisserer <linuxhippy@gmail.com>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    dubeyko committed with konis Jul 3, 2013
  15. nilfs2: implement calculation of free inodes count

    Currently, NILFS2 returns 0 as free inodes count (f_ffree) and current
    used inodes count as total file nodes in file system (f_files):
    
    df -i
    Filesystem      Inodes  IUsed   IFree IUse% Mounted on
    /dev/loop0           2      2       0  100% /mnt/nilfs2
    
    This patch implements real calculation of free inodes count.  First of
    all, it is calculated total file nodes in file system as
    (desc_blocks_count * groups_per_desc_block * entries_per_group).  Then, it
    is calculated free inodes count as difference the total file nodes and
    used inodes count.  As a result, we have such output for NILFS2:
    
    df -i
    Filesystem       Inodes   IUsed    IFree IUse% Mounted on
    /dev/loop0      4194304 2114701  2079603   51% /mnt/nilfs2
    
    Reported-by: Clemens Eisserer <linuxhippy@gmail.com>
    Signed-off-by: Vyacheslav Dubeyko <slava@dubeyko.com>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    Tested-by: Vyacheslav Dubeyko <slava@dubeyko.com>
    Cc: Joern Engel <joern@logfs.org>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    dubeyko committed with konis Jul 3, 2013
  16. Import source files

    This is an initial revision of nilfs2 kernel module for CentOS 7
    and other RHEL 7 clones.
    
    Source files of this package derive from those of nilfs2 in Linux
    kernel 3.10.47.
    
    Signed-off-by: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
    konis committed Jul 8, 2014