Latest release

zfs-0.7.9

@tonyhutter tonyhutter released this May 10, 2018 · 1640 commits to master since this release

Supported Kernels

  • Compatible with 2.6.32 - 4.16 Linux kernels.

Bug Fixes

ZFS

  • Remove DEBUG_STACKFLAGS to bypass compiler error
  • Fedora 28: Add BuildRequires: libtirpc-devel #7494 #7495
  • RHEL 7.5 compat: FMODE_KABI_ITERATE #7460 #7463
  • Add back iostat -y or -w descriptions #7479 #7483
  • Add test with two kinds of file creation orders #7411
  • Allow mounting datasets more than once #5796 #7207
  • Fix zfs_arc_max minimum tuning #7417
  • Linux compat 4.16: blk_queue_flag_{set,clear} #7428 #7431
  • Linux compat 4.16: blk_queue_flag_{set,clear} #7410
  • Fix 'zfs send/recv' hang with 16M blocks #7365 #7404
  • Clean up (k)shlib and cfg file shebangs
  • Fix "file is executable, but no shebang" warnings #7359 #7395
  • Exclude python scripts from RPM shebang check #7360 #7399
  • Update mmp_delay on sync or skipped, failed write #7330
  • Fedora 28: Fix misc bounds check compiler warnings #7361 #7368
  • Fix hung z_zvol tasks during 'zfs receive' #6330 #6890 #7343
  • Add support for nvme based devids #7356
  • chmod -x on etc/init.d/zfs-*.in automake files #7355 #7327
  • Fix mmap / libaio deadlock #7335 #7339
  • Remove libattr requirement #7344 #7351
  • Fedora 28: Fix "Macro %_dracutdir has empty body" #7326 #7328
  • modprobe zfs during dracut mount #7322
  • Add support for nvme disk detection #7304
  • Report pool suspended due to MMP #7296
  • Add zfs_scan_ignore_errors tunable #7293
  • Allow to limit zed's syslog chattiness #6886 #7260
  • Record skipped MMP writes in multihost_history #7212
  • Introduce a destroy_dataset helper #7224 #7246 #7249 #7267

SPL

  • Fix undefined RPM macros #703
  • Exclude python scripts from RPM shebang check
  • Linux compat 4.16: SECTOR_SIZE #697
  • Remove sysevents #696

zfs-0.7.8

@tonyhutter tonyhutter released this Apr 10, 2018 · 1640 commits to master since this release

This is a one-patch release to fix a major regression: #7401.

Supported Kernels

  • Compatible with 2.6.32 - 4.16 Linux kernels.

Bug Fixes

ZFS

  • Revert "Handle zap_add() failures in mixed ... " #7416

zfs-0.7.7

@tonyhutter tonyhutter released this Mar 19, 2018 · 31 commits to zfs-0.7-release since this release

Updated 4/9/18: We've found a serious regression in 0.7.7 (#7401). Please use 0.7.6 or 0.7.8.

Supported Kernels

  • Compatible with 2.6.32 - 4.16 Linux kernels.

Bug Fixes

ZFS

  • Fix MMP write frequency for large pools #7205 #7289
  • Handle zio_resume and mmp => off #7286
  • Fix zfs-kmod builds when using rpm >= 4.14 #7284
  • zdb and inuse tests don't pass with real disks #6939 #7261
  • Take user namespaces into account in policy checks #6800 #7270
  • Detect long config lock acquisition in mmp #7212
  • Linux 4.16 compat: get_disk_and_module() #7264
  • Change checksum & IO delay ratelimit values #7252
  • Increment zil_itx_needcopy_bytes properly #6988 #7176
  • Fix some typos #7237
  • Fix zpool(8) list example to match actual format #7244
  • Add SMART self-test results to zpool status -c #7178
  • Add scrub after resilver zed script #4662 #7086
  • Fix free memory calculation on v3.14+ #7170
  • Report duration and error in mmp_history entries #7190
  • Do not initiate MMP writes while pool is suspended #7182
  • Linux 4.16 compat: use correct *_dec_and_test()
  • Allow modprobe to fail when called within systemd #7174
  • Add SMART attributes for SSD and NVMe #7183 #7193
  • Correct count_uberblocks in mmp.kshlib #7191
  • Fix config issues: frame size and headers #7169
  • Clarify zinject(8) explanation of -e #7172
  • OpenZFS 8857 - zio_remove_child() panic due to already destroyed parent zio #7168
  • 'zfs receive' fails with "dataset is busy" #7129 #7154
  • contrib/initramfs: add missing conf.d/zfs #7158
  • mmp should use a fixed tag for spa_config locks #6530 #7155
  • Handle zap_add() failures in mixed case mode #7011 #7054
  • Fix zdb -ed on objset for exported pool #7099 #6464
  • Fix zdb -E segfault #7099
  • Fix zdb -R decompression #7099 #4984
  • Fix racy assignment of zcb.zcb_haderrors #7099
  • Fix zle_decompress out of bound access #7099
  • Fix zdb -c traverse stop on damaged objset root #7099
  • Linux 4.11 compat: avoid refcount_t name conflict #7148
  • Linux 4.16 compat: inode_set_iversion() #7148
  • OpenZFS 8966 - Source file zfs_acl.c, function zfs_aclset_common contains a use after end of the lifetime of a local variable #7141
  • Remove deprecated zfs_arc_p_aggressive_disable #7135
  • Fix default libdir for Debian/Ubuntu #7083 #7101
  • Bug fix in qat_compress.c for vmalloc addr check #7125
  • Fix systemd_ RPM macros usage on Debian-based distributions #7074 #7100
  • Emit an error message before MMP suspends pool #7048
  • ZTS: Fix create-o_ashift test case #6924 #6977
  • Fix --with-systemd on Debian-based distributions (#6963) #6591 #6963
  • Remove vn_rename and vn_remove dependency zfsonlinux/spl#648 #6753
  • Fix "--enable-code-coverage" debug build #6674
  • Update codecov.yml #6669
  • Add support for "--enable-code-coverage" option #6670
  • Make "-fno-inline" compile option more accessible #6605
  • Add configure option to enable gcov analysis #6642
  • Implement --enable-debuginfo to force debuginfo #2734
  • Make --enable-debug fail when given bogus args #2734

SPL

zfs-0.7.6

@tonyhutter tonyhutter released this Feb 6, 2018 · 1640 commits to master since this release

Supported Kernels

  • Compatible with 2.6.32 - 4.15 Linux kernels.

Bug Fixes

ZFS

  • Fix 'zfs receive -o' when used with '-e|-d' #7088
  • Extend zloop.sh for automated testing #6999
  • Cleanup zloop working directory after each pass #6663
  • OpenZFS 8835 - Speculative prefetch in ZFS not working for misaligned reads #7062
  • Fix Debian packaging on ARMv7/ARM64 #7046 #7058
  • Fix shellcheck v0.4.6 warnings #7040
  • Remove l2arc_nocompress from zfs-module-parameters(5) #7043
  • Fix incompatibility with Reiser4 patched kernels #6241 #7021
  • Use zap_count instead of cached z_size for unlink #7019
  • Revert raidz_map and _col structure types #6981 #7023
  • zhack: fix getopt return type #7016
  • Fix ARC hit rate #6171 #6852 #6989
  • Fix 'zpool add' handling of nested interior VDEVs #6678 #6996
  • Call commit callbacks from the tail of the list #6986
  • Handle broken pipes in arc_summary #6965 #6969
  • Handle invalid options in arc_summary #6983
  • OpenZFS 8794 - cstyle generates warnings with recent perl #6973
  • Update for cppcheck v1.80 #6879
  • Fix data on evict_skips in arc_summary.py #6882 #6883
  • Minor code cleanups in arc_python.py #6869
  • Fix arc_summary.py -d crash with Python3 #6849 #6850
  • Sort output of tunables in arc_summary.py #6828
  • Add documentation strings to arc_summary.py #6818
  • Rewrite fHits() in arc_summary.py with SI units #6815
  • Minor code cleanup in arc_summary.py #6802
  • Rewrite of function fBytes() in arc_summary.py #6784
  • Fix bug in distclean which removes needed files #6636
  • dmu_objset: release bonus buffer in failure path #6575
  • Fix zfs_ioc_pool_sync should not use fnvlist #6529
  • vdev_mirror: load balancing fixes #6461
  • Use /sbin/openrc-run for openrc init scripts #6519

SPL

zfs-0.7.5

@tonyhutter tonyhutter released this Dec 19, 2017 · 1640 commits to master since this release

Supported Kernels

  • Compatible with 2.6.32 - 4.14 Linux kernels.

Bug Fixes

  • Fix multihost stale cache file import #6933 #6971
  • Fix ZTS MMP tests and ztest -M behavior #6666
  • Enable QAT support in zfs-dkms RPM #6932
  • Add zfs-import.target services in spec file #6955
  • Enable zfs-import.target in systemd preset #6968

zfs-0.7.4

@tonyhutter tonyhutter released this Dec 12, 2017 · 1640 commits to master since this release

Supported Kernels

  • Compatible with 2.6.32 - 4.14 Linux kernels.

Important Notices

  • The new systemd zfs-import.target file was added to the RPM packages but not automatically enabled at install time. This leads to an incorrect unit ordering on startup and missing mounted file systems. This issue can be resolved by running systemctl enable zfs-import.target after installing the packages. #6953

Bug Fixes

  • Allow test-runner to filter test groups by tag #6788
  • Fix NFS sticky bit permission denied error #6889 #6910
  • Add /usr/bin/env to COPY_EXEC_LIST initramfs hook #5360 #6913
  • Fix 'zpool create|add' replication level check #6907 #6911
  • Preserve itx alloc size for zio_data_buf_free() #6912
  • Fix 'zfs get {user|group}objused@' functionality #6908
  • Linux 4.14 compat: CONFIG_GCC_PLUGIN_RANDSTRUCT #5390 #6903
  • initramfs: Honor canmount=off #6897
  • initramfs: Honor mountpoint=none/legacy #6897
  • zpool(8): Fix "zpool import -t" #6894
  • Fix column alignment with long zpool names #6786
  • Emit history events for 'zpool create' #6712 #6486
  • Fix dirty check in dmu_offset_next() #3125 #6867
  • Disable automatic dependencies in zfs-test package #6868
  • Fix truncate(2) mtime and ctime handling #6811 #6819
  • OpenZFS 7531 - Assign correct flags to prefetched buffers (https://www.illumos.org/issues/7531)
  • Handle compressed buffers in __dbuf_hold_impl() #5742 #6797
  • Fix undefined %{systemd_svcs} in RPM scriptlets #6838 #6841
  • Bug fix in qat_compress.c when compressed size is < 4KB #6827
  • Disable automatic dependencies in DKMS package #6467 #6835
  • Initramfs fixes #6807
  • systemd zfs-import.target and documentation #6764
  • Update zfs module parameters man5 #6785
  • Fix status command options in zpool(8) #6792 #6794
  • arcstat: flush stdout / outfile after each line #6775
  • Ensure arc_size_break is filled in arc_summary.py #5526 #6770
  • Correct flake8 errors after STYLE builder update #6776
  • Support integration with new QAT products #6767
  • Reimplement vdev_random_leaf and rename it #6631 #6665
  • Linux 4.14 compat: vfs_read & vfs_write #656 #667
  • Update spl module parameters man5 with missing parameter details #664
  • Pool io stat shows wlentime instead of rlentime #652 #651
  • spl-module-parameters.5 manpage: fix macro #643
  • splat.1 manpage: fix spelling of 'hexadecimal' #642
  • Fix use-after-free in taskq_seq_show_impl #638 #640

zfs-0.7.3

@tonyhutter tonyhutter released this Oct 19, 2017 · 153 commits to zfs-0.7-release since this release

Supported Kernels

  • Compatible with 2.6.32 - 4.13 Linux kernels.

Bug Fixes

  • Add DKMS package on Debian-based distributions (zfs) #6044 #6731
  • Fix function documentation to correctly mirror code #6754
  • Increase default zloop.sh vdev size #6758
  • Typo in dsl_dataset.h #6756 #6273
  • Fix chattr/cleanup failure #6749
  • Fixes for SPARC support #6733 #6738 #6750
  • Explicitly depend on icp module in initramfs hook #6751
  • Fix boot from ZFS issues #6700 #6747
  • Skip FREEOBJECTS for objects which can't exist #5699 #6507 #6616
  • Free objects when receiving full stream as clone #5699 #6507 #6616
  • Fix intra-pool resumable 'zfs send -t ' #6618 #6619 #6623
  • Fix ARC behavior on 32-bit systems #5352 #6734
  • Fix inclusion of libgcc_s.so on Void #6715
  • Use bitwise '&' instead of logical '&&' #6684 #6722
  • Remove unnecessary equality check #6685
  • Use linear abd in vdev_copy_uberblocks() #6718 #6713
  • receive_freeobjects() skips freeing some objects #6694 #6695
  • Scale the dbuf cache with arc_c #6561
  • Add DKMS package on Debian-based distributions (spl) #657
  • Add parenthesis to btop and ptob macros (#662) #660

zfs-0.7.2

@tonyhutter tonyhutter released this Sep 25, 2017 · 1640 commits to master since this release

Supported Kernels

  • Compatible with 2.6.32 - 4.13 Linux kernels.

Bug Fixes

  • Correct cppcheck errors #6662 #6653
  • Increase default arc_c_min #6659
  • Export symbol dmu_tx_mark_netfree() #6660
  • ZTS fix slog_replay_volume.ksh failure #6654
  • Linux 4.14 compat: IO acct, global_page_state, etc #6655 #6635
  • Modifying XATTRs doesnt change the ctime #3644 #6586
  • Fix volume WR_INDIRECT log replay #6620 #6603
  • Improved dnode allocation and dmu_hold_impl() #6611 #6611
  • Linux 4.8+ compatibility fix for vm stats #6528
  • Disable mount(8) canonical paths in do_mount() #1791 #6429 #6437
  • Fix range locking in ZIL commit codepath #6238 #6315 #6356 #6477
  • Fix remounting snapshots read-write #6510 #6515
  • Fix ZTS grow_pool/setup #6499 #6516
  • vdev_id: implement slot numbering by port id #6484
  • Fix NULL pointer when O_SYNC read in snapshot #6478 #6494
  • zio_dva_throttle_done() should allow zinjected ZIO #6383 #6384
  • Man page fixes #6492
  • Crash in dbuf_evict_one with DTRACE_PROBE #6463
  • Tag zfs-0.7.1- Fix dnode allocation race #6414

zfs-0.7.1

@tonyhutter tonyhutter released this Aug 8, 2017 · 1640 commits to master since this release

Supported Kernels

  • Compatible with 2.6.32 - 4.12 Linux kernels.

Bug Fixes

zfs-0.7.0

@behlendorf behlendorf released this Jul 26, 2017 · 1640 commits to master since this release

New Features

  • Resumable zfs send/receive - Allow an interrupted zfs receive to be resumed if the stream was prematurely terminated (e.g. due to remote system or network failure).

  • Compressed zfs send/receive - Use the zfs send -c option to directly send the compressed data in the ARC or on-disk to another pool without needing to decompress it.

  • Multiple Import Protection - Prevents a shared pool in a fail-over configuration from being imported on different hosts at the same time. When the multihost pool property is on, perform an activity check prior to importing the pool to verify it is not in use.

  • Customized zpool iostat|status columns - Additional columns can be added to the zpool iostat and zpool status output to show more information. Several useful scripts are provided which can report drive temperature, SMART data, enclosure LED status, and more. Administrators and users can add additional scripts to meet their needs.

  • Latency and request size histograms - Use the zpool iostat -l option to show on-the-fly latency stats and zpool iostat -w to generate a histogram showing the total latency of each IO. The zpool iostat -r option can be used to show the size of each IO. These statistics are available per-disk to aid in finding misbehaving devices.

  • Scrub Pause - The zpool scrub -p option can be used to pause/resume an active scrub without having to cancel it.

  • Delegations - The zfs allow and zfs unallow subcommands can be used to delegate ZFS administrative permissions for the file systems to non-privileged users.

  • Large dnodes - This feature improves metadata performance allowing extended attributes, ACLs, and symbolic links with long target names to be stored in the dnode. This benefits workloads such as SELinux, distributed filesystems like Lustre and Ceph, and any application which makes use of extended attributes.

  • User/group object accounting and quota - This feature adds per-object user/group accounting and quota limits to the existing space accounting and quota functionality. The zfs userspace and zfs groupspace subcommands have been extended to set quota limits and report on object usage.

  • Cryptographic checksums - Stronger SHA-512, Skein, or Edon-R checksums are available.

  • JBOD Management

    • Automatic drive online - Newly detected devices which are determined to be part of an imported pool are automatically brought online.
    • Automatic drive replacement - When the autoreplace pool property is on, any new device found in the same physical location as a device that previously belonged to the pool, is automatically formatted and replaced.
    • Automatic hot spares - When a device is faulted start a rebuild to a hot-spare device if available.
    • Fault LEDs - Set the fault LED for a device when it's faulted, clear it when it has been replaced.
    • Drive health monitoring - Automatically fault a device when an excessive number of read, write, or checksum errors are detected.
    • Force fault - Use zpool offline -f to proactively fault a problematic device.
    • Multipath aware - Can be used with advanced multipath configurations.

Performance

  • ARC Buffer Data (ABD) - Allocates ARC data buffers using scatter lists of pages instead of virtual memory. This approach minimizes fragmentation on the system allowing for a more efficient use of memory. The reduced demand for virtual memory also improves stability and performance on 32-bit architectures.
  • Compressed ARC - Cached file data is compressed by default in memory and uncompressed on demand. This allows for an larger effective cache which improves overall performance.
  • Vectorized RAIDZ - Hardware optimized RAIDZ which reduces CPU usage.
    Supported SIMD instructions: sse2, ssse3, avx2, avx512f, and avx512bw, neon, neonx2
  • Vectorized checksums - Hardware optimized Fletcher-4 checksums which reduce CPU usage.
    Supported SIMD instructions: sse2, ssse3, avx2, avx512f, neon
  • GZIP compression offloading - Hardware optimized GZIP compression offloading with QAT accelerator.
  • Metadata performance - Overall improved metadata performance. Optimizations include a multi-threaded allocator, batched quota updates, improved prefetching, and streamlined call paths.
  • Faster RAIDZ resilver - When resilvering RAIDZ intelligently skips sections of the device which don't need to be rebuilt.

Changes in Behavior

  • Non-privileged users are allowed to run zpool list, zpool iostat, zpool status, zpool get, zfs list, and zfs get. These commands no longer need to be added to the /etc/sudoers file.
  • The permissions of the /dev/zfs device have changed from 0600 to 0666 to let ZFS do access control in kernel space and make zfs allow and zfs unallow work properly. If you have been changing permissions / group owner of the device file yourself your change won't work correctly anymore and breaks proper behavior of zfs allow. From this release forward you should be able to satisfy your use-case with the officially supported zfs allow command.
  • By default task queues are now dynamic and worker threads will be created and destroyed as needed. This allows the system to automatically tune itself to ensure the optimal number of threads are used for the active workload which can result in a performance improvement.
  • Accessing snapshots over NFS now requires the crossmnt option be added to the /etc/exports file. The nfsd service is now aware that snapshots are different filesystems. A result of this change is that older distributions, like CentOS 6.x, can no longer provide access to snapshots over NFS.

Supported Kernels

  • Compatible with 2.6.32 - 4.12 Linux kernels.

Module Options

  • The default values for the module options were selected to yield good performance for the majority of workloads and configurations. They should not need to be tuned for most systems but are available for performance analysis and tuning. See the zfs-module-parameters(5) man page for a more complete description of the options and what they control.
  • Added:
    • dbuf_cache_hiwater_pct - Percent over dbuf_cache_max_bytes when dbufs must be evicted
    • dbuf_cache_lowater_pct - Percent below dbuf_cache_max_bytes when dbufs stop being evicted
    • dbuf_cache_max_bytes - Maximum size in bytes of the dbuf cache
    • dbuf_cache_max_shift - Cap the size of the dbuf cache to a log2 fraction of arc size
    • dmu_object_alloc_chunk_shift - CPU-specific allocator grabs 2^N objects at once
    • send_holes_without_birth_time - Ignore hole_birth txg for zfs send
    • zfetch_max_distance - Max bytes to prefetch per stream
    • zfs_abd_scatter_enabled - Toggle whether ABD allocations must be linear
    • zfs_abd_scatter_max_order - Maximum order allocation used for a scatter ABD
    • zfs_arc_dnode_limit - Minimum bytes of dnodes in ARC
    • zfs_arc_dnode_limit_percent - Percent of ARC meta buffers for dnodes
    • zfs_arc_dnode_reduce_percent - Percentage of excess dnodes to try to unpin
    • zfs_arc_meta_limit_percent - Percent of arc size for arc meta limit
    • zfs_arc_pc_percent - Percent of pagecache to reclaim ARC to
    • zfs_compressed_arc_enabled - Disable compressed arc buffers
    • zfs_deadman_checktime_ms - Dead I/O check interval in milliseconds
    • zfs_delete_blocks - Delete files larger than N blocks asynchronously
    • zfs_dmu_offset_next_sync - Enable forcing txg sync to find holes
    • zfs_free_bpobj_enabled - Enable processing of the free_bpobj
    • zfs_metaslab_segment_weight_enabled - Enable segment-based metaslab selection
    • zfs_metaslab_switch_threshold - Metaslab selection max buckets before switching
    • zfs_multihost_fail_intervals - Max allowed period without a successful mmp write
    • zfs_multihost_history - Historical statistics for last N multihost writes
    • zfs_multihost_import_intervals - Number of zfs_multihost_interval periods to wait for activity
    • zfs_multihost_interval - Milliseconds between mmp writes to each leaf
    • zfs_multilist_num_sublists - Number of sublists used in each multilist
    • zfs_per_txg_dirty_frees_percent - Percentage of dirtied blocks from frees in one TXG
    • zfs_sync_taskq_batch_pct - Percentage of CPUs to run an IO worker thread
    • zfs_vdev_mirror_non_rotating_inc - Non-rotating media load increment for non-seeking I/O's
    • zfs_vdev_mirror_non_rotating_seek_inc - Non-rotating media load increment for seeking I/O's
    • zfs_vdev_mirror_rotating_inc - Rotating media load increment for non-seeking I/O's
    • zfs_vdev_mirror_rotating_seek_inc - Rotating media load increment for seeking I/O's
    • zfs_vdev_mirror_rotating_seek_offset - Offset in bytes from the last I/O to trigger seek increment
    • zfs_vdev_queue_depth_pct - Queue depth percentage for each top-level vdev
    • zfs_vdev_raidz_impl - Select RAIDZ implementation.
    • zil_slog_bulk - Limit in bytes slog sync writes per commit
    • zio_dva_throttle_enabled - Throttle block allocations in the ZIO pipeline
    • zvol_request_sync - Synchronously handle bio requests
    • zvol_threads - Max number of threads to handle I/O requests
    • zvol_volmode - Default volmode property value
    • spl_max_show_tasks - Max number of tasks shown in taskq proc
    • spl_panic_halt - Cause kernel panic on assertion failures
  • Removed:
    • l2arc_nocompress - Skip compressing L2ARC buffers
    • zfetch_block_cap - Max number of blocks to fetch at a time
    • zfs_arc_num_sublists_per_state - Number of sublists used in each of the ARC state lists
    • zfs_disable_dup_eviction - Disable duplicate buffer eviction
    • zfs_vdev_mirror_switch_us - Switch mirrors every N microseconds
    • zil_slog_limit - Max commit bytes to separate log device
  • Changed:
    • zfs_admin_snapshot - Enable mkdir/rmdir/mv in .zfs/snapshot