Skip to content

zfs-0.8.0
Compare
Choose a tag to compare
@behlendorf behlendorf released this 23 May 17:13
· 4039 commits to master since this release
zfs-0.8.0
This tag was signed with the committer’s verified signature.
behlendorf Brian Behlendorf
GPG key ID: 0AB9E991C6AF658B
Learn about vigilant mode.
e34c3ee

New Features

  • Native encryption #5769 - The encryption property enables the creation of encrypted filesystems and volumes. The aes-256-ccm algorithm is used by default. Per-dataset keys are managed with zfs load-key and associated subcommands.

  • Raw encrypted 'zfs send/receive' #5769 - The zfs send -w option allows an encrypted dataset to be sent and received to another pool without decryption. The received dataset is protected by the original user key from the sending side. This allows datasets to be efficiently backed up to an untrusted system without fear of the data being compromised.

  • Device removal #6900 - This feature allows single and mirrored top-level devices to be removed from the storage pool with zpool remove. All data is copied in the background to the remaining top-level devices and the pool capacity is reduced accordingly.

  • Pool checkpoints #7570 - The zpool checkpoint subcommand allows you to preserve the entire state of a pool and optionally revert back to that exact state. It can be thought of as a pool wide snapshot. This is useful when performing complex administrative actions which are otherwise irreversible (e.g. enabling a new feature flag, destroying a dataset, etc).

  • Pool TRIM #8419 - The zpool trim subcommand provides a way to notify the underlying devices which sectors are no longer allocated. This allows an SSD to more efficiently manage itself and helps prevent performance from degrading. Continuous background trimming can be enabled via the new autotrim pool property.

  • Pool initialization #8230 - The zpool initialize subcommand writes a pattern to all the unallocated space. This eliminates the first access performance penalty, which may exist on some virtualized storage (e.g. VMware VMDKs).

  • Project accounting and quota #6290 - This features adds project based usage accounting and quota enforcement to the existing space accounting and quota functionality. Project quotas add an additional dimension to traditional user/group quotas. The zfs project and zfs projectspace subcommands have been added to manage projects, set quota limits and report on usage.

  • Channel programs #6558 - The zpool program subcommand can be used to perform compound ZFS administrative actions via Lua scripts in a sandboxed environment (with time and memory limits).

  • Pyzfs #7230 - The new pyzfs library is intended to provide a stable interface for the programmatic administration of ZFS. This wrapper provides a one-to-one mapping for the libzfs_core API functions, but the signatures and types are more natural to Python.

  • Python 3 compatibility #8096 - The arcstat, arcsummary, and dbufstat utilities have been updated to be compatible with Python 3.

  • Direct IO #7823 - Adds support for Linux's direct IO interface.

Performance

  • Sequential scrub and resilver #6256 - When scrubbing or resilvering a pool the process has been split into two phases. The first phase scans the pool metadata in order to determine where the data blocks are stored on disk. This allows the second phase to issue scrub I/O as sequentially as possible, greatly improving performance.

  • Allocation classes #5182 - Allows a pool to include a small number of high-performance SSD devices that are dedicated to storing specific types of frequently accessed blocks (e.g. metadata, DDT data, or small file blocks). A pool can opt-in to this feature by adding a special or dedup top-level device.

  • Administrative commands #7668 - Improved performance due to targeted caching of the metadata required for administrative commands like zfs list and zfs get.

  • Parallel allocation #7682 - The allocation process has been parallelized by creating multiple "allocators" per-metaslab group. This results in improved allocation performance on high-end systems.

  • Deferred resilvers #7732 - This feature allows new resilvers to be postponed if an existing one is already in progress. By waiting for the running resilver to complete redundancy is restored as quickly as possible.

  • ZFS Intent Log (ZIL) #6566 - New log blocks are created and issued while there are still outstanding blocks being serviced by the storage, effectively reducing the overall latency observed by the application.

  • Volumes #8615 - When a pool contains a large number of volumes they are more promptly registered with the system and made available for use after a zpool import.

  • QAT #7295 #7282 #6767 - Support for accelerated SHA256 checksums, AES-GCM encryption, and the new QAT Intel(R) C62x Chipset / Atom(R) C3000 Processor Product Family SoC.

Changes in Behavior

  • Relaxed (ref)reservation constraints on volumes, they may now be set larger than the volume size.

  • The arcstat.py, arc_summary.py, and dbufstat.py commands have been renamed arcstat, arc_summary, and dbufstat respectively.

  • The SPL source is now included in the ZFS repository removing the need for separate packages.

  • The dedupditto pool property and zfs send -D option have been deprecated and will be removed in a future release.

Additional Information

  • Supported kernels - Compatible with 2.6.32 - 5.1* Linux kernels.
    * SIMD acceleration is currently not supported for Linux 5.0 and newer kernels.

  • Change log - Complete v0.7.0 - v0.8.0 change log

  • Module options - The default values for the module options were selected to yield good performance for the majority of workloads and configurations. They should not need to be tuned for most systems but are available for performance analysis and tuning. See the zfs-module-parameters(5) man page for the complete list of the options and what they control.

Updated module options
  • Added:
    • dbuf_metadata_cache_max_bytes* - Maximum size in bytes of the dbuf metadata cache
    • dbuf_cache_shift - Set the size of the dbuf cache to a log2 fraction of arc size
    • icp_aes_impl - Select aes implementation
    • icp_gcm_impl - Select gcm implementation
    • metaslab_force_ganging - Blocks larger than this size are forced to be gang blocks
    • spa_load_print_vdev_tree - Print vdev tree to zfs_dbgmsg during pool import
    • spl_hostid - The system hostid
    • spl_hostid_path - The system hostid file (/etc/hostid)
    • spl_panic_halt - Cause kernel panic on assertion failures
    • vdev_removal_max_span - Largest span of free chunks a remap segment can span
    • vdev_validate_skip - Bypass vdev_validate()
    • zio_slow_io_ms - Max I/O completion time (milliseconds) before marking it as slow
    • zio_deadman_log_all - Log all slow ZIOs, not just those with vdevs
    • zfs_key_max_salt_uses - Max number of times a salt value can be used for generating encryption keys before it is rotated
    • zfs_commit_timeout_pct - ZIL block open timeout percentage
    • zil_nocacheflush - Disable ZIL cache flushes
    • zfs_unlink_suspend_progress - Set to prevent async unlinks (debug - leaks space into the unlinked set)
    • zfs_lua_max_instrlimit - Max instruction limit that can be specified for a channel program
    • zfs_lua_max_memlimit - Max memory limit that can be specified for a channel program
    • zfs_trim_extent_bytes_max - Max size of TRIM commands, larger will be split
    • zfs_trim_extent_bytes_min - Min size of TRIM commands, smaller will be skipped
    • zfs_trim_metaslab_skip - Skip metaslabs which have never been initialized
    • zfs_trim_txg_batch - Min number of txgs to aggregate frees before issuing TRIM
    • zfs_trim_queue_limit - Max queued TRIMs outstanding per leaf vdev
    • zfs_removal_ignore_errors - Ignore hard IO errors when removing device
    • zfs_remove_max_segment - Largest contiguous segment to allocate when removing device
    • zfs_removal_suspend_progress - Pause device removal after this many bytes are copied (debug use only - causes removal to hang)
    • zfs_vdev_aggregation_limit_non_rotating - Max vdev I/O aggregation size for non-rotating media
    • zfs_vdev_aggregate_trim - Allow TRIM I/O to be aggregated
    • zfs_vdev_initializing_max_active - Max active initializing I/Os per vdev
    • zfs_vdev_initializing_min_active - Min active initializing I/Os per vdev
    • zfs_vdev_removal_max_active - Max active removal I/Os per vdev
    • zfs_vdev_removal_min_active - Min active removal I/Os per vdev
    • zfs_vdev_trim_max_active - Max active trim/discard I/Os per vdev
    • zfs_vdev_trim_min_active - Min active trim/discard I/Os per vdev
    • zfs_initialize_value - Value written during zpool initialize
    • zfs_condense_indirect_vdevs_enable - Whether to attempt condensing indirect vdev mappings
    • zfs_condense_min_mapping_bytes - Minimum size of vdev mapping to condense
    • zfs_condense_max_obsolete_bytes - Minimum size obsolete spacemap to attempt condensing
    • zfs_condense_indirect_commit_entry_delay_ms - Delay while condensing vdev mapping
    • zfs_reconstruct_indirect_combinations_max - Maximum number of combinations when reconstructing split segments
    • zfs_vdev_default_ms_count - Target number of metaslabs per top-level vdev
    • zfs_vdev_min_ms_count - Minimum number of metaslabs per top-level vdev
    • zfs_vdev_ms_count_limit - Practical upper limit of total metaslabs per top-level vdev
    • zfs_slow_io_events_per_second - Rate limit slow IO (delay) events to this many per second
    • zfs_checksum_events_per_second - Rate limit checksum events to this many checksum errors per second
    • zfs_nocacheflush - Disable cache flushes
    • zfs_deadman_ziotime_ms - IO expiration time in milliseconds
    • zfs_deadman_failmode - Failmode for deadman timer
    • zfs_ddt_data_is_special - Place DDT data into the special class
    • zfs_user_indirect_is_special - Place user data indirect blocks into the special class
    • zfs_special_class_metadata_reserve_pct - Small file blocks in special vdevs depends on this much free space available
    • zfs_spa_discard_memory_limit - Maximum memory for prefetching checkpoint space map per top-level vdev while discarding checkpoint
    • zfs_max_missing_tvds - Allow importing pool with up to this number of missing top-level vdevs (in read-only mode)
    • zfs_scan_vdev_limit - Max bytes in flight per leaf vdev for scrubs and resilvers
    • zfs_scrub_min_time_ms - Min millisecs to scrub per txg
    • zfs_obsolete_min_time_ms - Min millisecs to obsolete per txg
    • zfs_scan_suspend_progres - Set to prevent scans from progressing
    • zfs_async_block_max_blocks - Max number of blocks freed in one txg
    • zfs_scan_mem_lim_fact - Fraction of RAM for scan hard limit
    • zfs_scan_issue_strategy - IO issuing strategy during scrubbing. 0 = default, 1 = LBA, 2 = size
    • zfs_scan_legacy - Scrub using legacy non-sequential method
    • zfs_scan_checkpoint_intval - can progress on-disk checkpointing interval
    • zfs_scan_max_ext_gap - Max gap in bytes between sequential scrub / resilver I/Os
    • zfs_scan_mem_lim_soft_fact - Fraction of hard limit used as soft limit
    • zfs_scan_strict_mem_lim - Tunable to attempt to reduce lock contention
    • zfs_scan_fill_weight - Tunable to adjust bias towards more filled segments during scans
    • zfs_resilver_disable_defer - Process all resilvers immediately
    • zfs_zil_clean_taskq_nthr_pct - Max percent of CPUs that are used per dp_sync_taskq
    • zfs_zil_clean_taskq_minalloc - Number of taskq entries that are pre-populated
    • zfs_zil_clean_taskq_maxalloc - Max number of taskq entries that are cached
    • zfs_disable_ivset_guid_check - Set to allow raw receives without IVset guids
    • zfs_override_estimate_recordsize - Record size calculation override for zfs send estimates
    • zfs_send_unmodified_spill_blocks - Send unmodified spill blocks
    • zfs_arc_min_prefetch_ms - Min life of prefetch block in ms
    • zfs_arc_min_prescient_prefetch_ms - Min life of prescient prefetched block in ms
    • zfs_abd_scatter_min_size - Minimum size of scatter allocations
  • Removed:
    • dbuf_cache_max_shift - Cap the size of the dbuf cache to a log2 fraction of arc size
    • metaslabs_per_vdev - Divide added vdev into approximately (but no more than) this number of metaslabs
    • zio_delay_max - Max zio millisec delay before posting event
    • zfs_nocacheflush - Disable cache flushes
    • zfs_delays_per_second - Rate limit delay events to this many IO delays per second
    • zfs_checksums_per_second - Rate limit checksum events to this many checksum errors per second
    • zfs_top_maxinflight - Max I/Os per top-level
    • zfs_resilver_delay - Number of ticks to delay resilver
    • zfs_scrub_delay - Number of ticks to delay scrub
    • zfs_scan_idle - Idle window in clock ticks
    • zfs_scan_min_time_ms - Min millisecs to scrub per txg
    • zfs_free_max_blocks - Max number of blocks freed in one txg
    • zfs_dirty_data_sync - Sync txg when this much dirty data
    • zfs_mdcomp_disable - Disable meta data compression
    • zfs_arc_min_prefetch_lifespan - Min life of prefetch block
Assets 5