From 0d7ef114236e2acadbac867d2abf3135f978f9ec Mon Sep 17 00:00:00 2001 From: Mrt134 Date: Mon, 22 Feb 2021 12:31:06 -0500 Subject: [PATCH] DOCS-1700: Rewrite L2ARC reference. --- content/CORE/Notices/PersistentL2ARCin12.0.md | 29 +++++++ content/References/L2ARC.md | 44 ++++++++++ content/References/L2arcReference.md | 83 ------------------- ...s-2020.md => MicrosoftLDAPDefaults2020.md} | 0 .../References/{SMB-references.md => SMB.md} | 0 .../{ZFS-references.md => ZFSPrimer.md} | 0 6 files changed, 73 insertions(+), 83 deletions(-) create mode 100644 content/CORE/Notices/PersistentL2ARCin12.0.md create mode 100644 content/References/L2ARC.md delete mode 100644 content/References/L2arcReference.md rename content/References/{ms-ldap-defaults-2020.md => MicrosoftLDAPDefaults2020.md} (100%) rename content/References/{SMB-references.md => SMB.md} (100%) rename content/References/{ZFS-references.md => ZFSPrimer.md} (100%) diff --git a/content/CORE/Notices/PersistentL2ARCin12.0.md b/content/CORE/Notices/PersistentL2ARCin12.0.md new file mode 100644 index 0000000000..d09e483310 --- /dev/null +++ b/content/CORE/Notices/PersistentL2ARCin12.0.md @@ -0,0 +1,29 @@ +--- +title: "Persistent L2ARC in TrueNAS 12.0" +weight: 20 +--- + +Persistent L2ARC, or L2ARC rebuild, is a ZFS feature that was added to TrueNAS 12.0. +By default, the L2ARC cache empties when the system reboots. +When persistent L2ARC is enabled, a sysctl repopulates the cache device mapping during the reboot process. +This is intended to preserve L2ARC performance even after a system reboot. + +However, persistent L2ARC for very large data pools can drastically slow the reboot process, degrading middleware and web interface performance. +Because of this, persistent L2ARC is disabled by default in TrueNAS 12.0 and must be manually activated. + +## Activating Persistent L2ARC + +Go to **System > Tunables > ADD**. +For the *Variable*, enter `vfs.zfs.l2arc.rebuild_enabled`. Set the *Value* to *1* and the *Type* to *sysctl*. +It is often helpful to note in the *Description* that this is the Persistent L2ARC activation. +Make sure *Enabled* is set and click *SUBMIT*. + +![PersistentL2ARCTunable](/images/CORE/12.0/SystemTunablesL2ARCRebuild.png "Persistent L2ARC Activation") + +{{< expand "CLI Instructions" "v" >}} +{{< hint warning >}} +Settings changed through the CLI are not written to the configuration database and will be reset on reboot. +{{< /hint >}} +In a command line, enter `sysctl vfs.zfs.l2arc.rebuild_enabled=1`. +When successful, the output reads: `vfs.zfs.l2arc.rebuild_enabled: 0 -> 1` +{{< /expand >}} diff --git a/content/References/L2ARC.md b/content/References/L2ARC.md new file mode 100644 index 0000000000..382ceef689 --- /dev/null +++ b/content/References/L2ARC.md @@ -0,0 +1,44 @@ +--- +title: "L2ARC" +weight: 50 +--- + +{{< toc >}} + +ZFS has several features to help improve performance for frequent access data read operations. One is Adaptive Replacement Cache (ARC), which is uses the server memory (RAM). The other is second level adaptive replacement cache (L2ARC), or cache drives added to ZFS storage pools. These cache drives are multi-level cell (MLC) SSD drives and, while slower than system memory, still much faster than standard hard drives. ZFS (and TrueNAS) uses all of the RAM installed in a system to make the ARC as large as possible, but this can be very expensive. Cache drives provide a cheaper alternative to RAM for frequently accessed data. + +## How Does L2ARC Work? + +When a system gets read requests, ZFS uses ARC (RAM) to serve those requests. When the ARC is full and there are L2ARC drives are allocated to a ZFS pool, ZFS uses the L2ARC to serve the read requests that "overflowed" from the ARC. This minimizes how often the slower hard drives are accessed and increases system performance. + +### Implementation in TrueNAS + +TrueNAS integrates L2ARC management in the **Storage > Pools** section of the web interface. Specifically, adding a *Cache* vdev to a new or existing pool and allocating drives to that pool enables L2ARC for that specific storage pool. + +Cached drives are not mirrored, but always striped. +To increase the size of an existing L2ARC, stripe another cache device with it. +Dedicated L2ARC devices cannot be shared between ZFS pools. + +A cache device failure does not affect the integrity of the pool, but can impact read performance. +This depends on the workload and the ratio of dataset size to cache size. + +### Device Recommendations + +Like all complicated features, trying to decide if using L2ARC is effective requires a strong understanding of your storage environment, performance goals, and the software being used. +However, there are a few recommendations for L2ARC devices: + +* Using multiple L2ARC devices helps reduce latency and improve performance. + +* Random Read Heavy workloads can benefit from large capacity L2ARC SSDs that are faster than the existing data storage drives. + +* Sequential or streaming workloads need very fast and low-latency L2ARC devices. [Enterprise-grade NVMe devices are recommended](https://www.snia.org/sites/default/files/SDC/2019/presentations/File_Systems/McKenzie_Ryan_Best_Practices_for_OpenZFS_L2ARC_in_the_Era_of_NVMe.pdf). The device capacity is dependent on how much faster the L2ARC device is over the data storage devices. The faster the L2ARC device over the storage, larger capacity becomes more useful. + +## Resources + +* "Explanation of ARC and L2ARC" : https://www.zfsbuild.com/2010/04/15/explanation-of-arc-and-l2arc/ +* "Best Practices for OpenZFS L2ARC in the Era of NVMe" : https://www.snia.org/sites/default/files/SDC/2019/presentations/File_Systems/McKenzie_Ryan_Best_Practices_for_OpenZFS_L2ARC_in_the_Era_of_NVMe.pdf +* Open ZFS Repository: https://github.com/openzfs/zfs +* "ARC: A Self-Tuning, Low Overhead Replacement Cache": https://www.usenix.org/conference/fast-03/arc-self-tuning-low-overhead-replacement-cache +* "Activity of the ZFS ARC": http://dtrace.org/blogs/brendan/2012/01/09/activity-of-the-zfs-arc/ +* "ZFS L2ARC": http://www.brendangregg.com/blog/2008-07-22/zfs-l2arc.html +* "FreeBSD Mastery: Advanced ZFS": https://www.amazon.com/FreeBSD-Mastery-Advanced-ZFS/dp/164235001X diff --git a/content/References/L2arcReference.md b/content/References/L2arcReference.md deleted file mode 100644 index 1c3ebdeb35..0000000000 --- a/content/References/L2arcReference.md +++ /dev/null @@ -1,83 +0,0 @@ ---- -title: "L2ARC" -description: "What L2ARC does, and how to utilize it within TrueNAS." -tags: ["ZFS", "zpool"] ---- - -There are features within ZFS that help to improve performance for read operations of data that is accessed on a frequent basis. One is adaptive replacement cache (ARC), which is located in a servers memory (RAM). The other is second level adaptive replacement cache (L2ARC), or cache drives within ZFS systems. These cache drives are multi-level cell (MLC) SSD drives, and while slower than system memory, they are faster than standard hard drives. Generally speaking, you want as much RAM in a system as possible to make the ARC as large as possible, but this can be a costly endeavor. Cache drives provide a cheaper alternative to RAM for frequently accessed content. - -## How Does L2ARC Work - -If cache drives are present within a ZFS pool, these drives will cache data that is frequently accessed that did not fit in ARC. So, when a system gets read requests, ZFS will try to use ARC to serve those requests. If the data is not in the ARC, ZFS will then try to serve those requests from the L2ARC. The result of this is the slow hard drives within your system will be accessed less resulting in better performace. - -## Creating a Pool with A Cached Drive - -Go to **Storage > Pools > ADD > CREATE POOL** Select two hard drives from your available disks and add them to *Data Vdevs* (or however many drives you need for your requirements). From the **ADD VDEV** dropdown menu select *Cache*. - -

- -

- -From the *Available Disks* section, select an SSD and add to *Cache Vdev* then click **CREATE**. Check the *Confirm* box and click **CREATE POOL**. - -

- -

- -It is a wise practice to check the *Status* of a pool after it has successfully been created. Select   (pool Operations) to the right of the pool you just created and choose **Status**. This will take you to the *Pool Status* page. Verify your configuration contains the hard drives in their appropriate format (Mirror, Raid-z, Raid-z2 etc.) and the cached drive is present. - -

- -

- -{{% pageinfo %}} -Cached drives do not get mirrored. To increase the size of an existing L2ARC, stripe another cache device with it. The web interface will always stripe L2ARC, not mirror it, as the contents of L2ARC are recreated at boot. Failure of an individual SSD from an L2ARC pool will not affect the integrity of the pool, but may have an impact on read performance, depending on the workload and the ratio of dataset size to cache size. Note that dedicated L2ARC devices cannot be shared between ZFS pools. -{{% /pageinfo %}} - -## Adding an Additional Cache Drive to an Existing Pool - -Go to **Storage > Pools** and select   to the right of the pool you wish to add an additional cached drive. From the menu, select *Add Vdevs*. Access the **ADD VDEV** dropdown menu and select *Cache*. Select an SSD and add to *Cache Vdev* then click **ADD VDEVS**. Check the *Confirm* box and click **ADD VDEVS**. - -

- -

- -

- -

- -

- -

- -## Persistent L2ARC - -Persistent L2ARC, or L2ARC rebuild, is a new feature of TrueNAS-12.0. Upon rebooting your system, the L2ARC gets emptied, but if you enable persistent L2ARC it will populate -the cache device mapping after reboot via *sysctl*. - -{{% pageinfo %}} -Persistent L2ARC is disabled by default due to performance impact when rebuilding with large amounts of data. Reactivating persistent L2ARC can result in degraded UI -and Middleware performance in specific situations where large amounts of data is loaded into L2ARC and a rebuild is triggered. -{{% /pageinfo %}} - -**To Reactivate Persistent L2ARC via the WebUI** - -Go to **System->Tunables->ADD**. *Variable* = `vfs.zfs.l2arc.rebuild_enabled`, *Value* = `1`, *Type* = `sysctl`, enter a description if desired and ensure the *Enabled* box is checked. Click **SUBMIT**. - -

- -

- -**To Reactivate Persistent L2ARC via the CLI** - -Enter the following command : `sysctl vfs.zfs.l2arc.rebuild_enabled=1` - -Command output should read : `vfs.zfs.l2arc.rebuild_enabled: 0 -> 1` - -{{% alert title=Warning color=warning %}} -Settings changed through the CLI are not written to the configuration database and will be reset on reboot. -{{% /alert %}} - - -*For more information, see ZFS Build - Explanation of ARC and L2ARC : https://www.zfsbuild.com/2010/04/15/explanation-of-arc-and-l2arc/* - diff --git a/content/References/ms-ldap-defaults-2020.md b/content/References/MicrosoftLDAPDefaults2020.md similarity index 100% rename from content/References/ms-ldap-defaults-2020.md rename to content/References/MicrosoftLDAPDefaults2020.md diff --git a/content/References/SMB-references.md b/content/References/SMB.md similarity index 100% rename from content/References/SMB-references.md rename to content/References/SMB.md diff --git a/content/References/ZFS-references.md b/content/References/ZFSPrimer.md similarity index 100% rename from content/References/ZFS-references.md rename to content/References/ZFSPrimer.md