ISPN-13157 file store docs

documentation/src/main/asciidoc/stories/assembly_creating_custom_cache_stores.adoc

@@ -1,5 +1,5 @@
-[id='creating_stores']
-:context: custom
+[id='creating-custom-cache-stores']
+:context: custom-cache-stores
 = Implementing Custom Cache Stores
 You can create custom cache stores through the {brandname} persistent SPI.
 

documentation/src/main/asciidoc/stories/assembly_persistence.adoc

@@ -1,5 +1,5 @@
 [id='persistence']
-:context: configure
+:context: persistence
 = Setting Up Persistent Storage
 {brandname} can persist in-memory data to external storage, giving you
 additional capabilities to manage your data such as:
@@ -15,10 +15,28 @@ Data overflow::
 Using eviction and passivation techniques ensures that {brandname} keeps only
 frequently used data in-memory and writes older entries to persistent storage.
 
-include::assembly_cache_stores.adoc[leveloffset=+1]
-include::assembly_cache_store_implementations.adoc[leveloffset=+1]
-include::assembly_migrating_cache_stores.adoc[leveloffset=+1]
+include::{topics}/proc_configuring_global_persistent_location.adoc[leveloffset=+1]
 
+include::{topics}/proc_configuring_cache_stores.adoc[leveloffset=+1]
+include::{topics}/con_passivation.adoc[leveloffset=+2]
+include::{topics}/ref_passivation_behavior.adoc[leveloffset=+3]
+include::{topics}/con_cache_loaders_transactional.adoc[leveloffset=+2]
+include::{topics}/con_segmented_cache_stores.adoc[leveloffset=+2]
+include::{topics}/con_write_through.adoc[leveloffset=+2]
+include::{topics}/con_write_behind.adoc[leveloffset=+2]
+
+include::{topics}/con_file_based_cache_stores.adoc[leveloffset=+1]
+include::{topics}/proc_configuring_file_stores.adoc[leveloffset=+2]
+include::{topics}/proc_configuring_single_file_stores.adoc[leveloffset=+2]
+
+include::{topics}/ref_cache_store_cluster.adoc[leveloffset=+1]
+include::{topics}/ref_cache_store_jdbc.adoc[leveloffset=+1]
+include::{topics}/ref_cache_store_jdbc_connection.adoc[leveloffset=+2]
+include::{topics}/ref_cache_store_jdbc_configuration.adoc[leveloffset=+2]
+include::{topics}/ref_cache_store_jpa.adoc[leveloffset=+1]
+include::{topics}/ref_cache_store_jpa_example.adoc[leveloffset=+2]
+include::{topics}/ref_cache_store_remote.adoc[leveloffset=+1]
+include::{topics}/ref_cache_store_rocksdb.adoc[leveloffset=+1]
 
 // Restore the parent context.
 ifdef::parent-context[:context: {parent-context}]

documentation/src/main/asciidoc/titles/configuring/stories.adoc

@@ -9,4 +9,6 @@ include::{stories}/assembly_configuration.adoc[leveloffset=+1]
 include::{stories}/assembly_configuring_data_container.adoc[leveloffset=+1]
 include::{stories}/assembly_statistics.adoc[leveloffset=+1]
 include::{stories}/assembly_persistence.adoc[leveloffset=+1]
+include::{stories}/assembly_creating_custom_cache_stores.adoc[leveloffset=+2]
+include::{stories}/assembly_migrating_cache_stores.adoc[leveloffset=+2]
 include::{stories}/assembly_partition_handling.adoc[leveloffset=+1]

documentation/src/main/asciidoc/topics/code_examples/GlobalPersistentLocation.java

@@ -0,0 +1 @@
+new GlobalConfigurationBuilder().globalState().enable().persistentLocation("example", "my.data");

documentation/src/main/asciidoc/topics/con_file_based_cache_stores.adoc

@@ -0,0 +1,76 @@
+[id='file-stores_{context}']
+= File-Based Cache Stores
+
+File-based cache stores provide persistent storage on the local host filesystem where {brandname} is running.
+For clustered caches, file-based cache stores are unique to each {brandname} node.
+
+[WARNING]
+====
+Never use filesystem-based cache stores on shared file systems, such as an NFS or Samba share, because they do not provide file locking capabilities and data corruption can occur.
+
+Additionally if you attempt to use transactional caches with shared file systems, unrecoverable failures can happen when writing to files during the commit phase.
+====
+
+[discrete]
+== Soft-Index File Stores
+
+`SoftIndexFileStore` is the default implementation for file-based cache stores and stores data in a set of append-only files.
+
+When append-only files:
+
+* Reach their maximum size, {brandname} creates a new file and starts writing to it.
+* Reach the compaction threshold of less than 50% usage, {brandname} overwrites the entries to a new file and then deletes the old file.
+
+.B+ trees
+
+To improve performance, append-only files in a `SoftIndexFileStore` are indexed using a **B+ Tree** that can be stored both on disk and in memory.
+The in-memory index uses Java soft references to ensure it can be rebuilt if removed by Garbage Collection (GC) then requested again.
+
+Because `SoftIndexFileStore` uses Java soft references to keep indexes in memory, it helps prevent out-of-memory exceptions.
+GC removes indexes before they consume too much memory while still falling back to disk.
+
+You can configure any number of B+ trees.
+By default {brandname} creates up to 16 B+ trees, which means there can be up to 16 indexes.
+Having multiple indexes prevents bottlenecks from concurrent writes to an index.
+It also minimizes the number of entries that can be read at the same time because {brandname} reads all entries at the same time when it iterates over a soft-index file store.
+
+Each entry in the B+ tree is a node.
+By default, the size of each node is limited to 4096 bytes.
+`SoftIndexFileStore` throws an exception if keys are longer after serialization occurs.
+
+.Expired entries
+
+`SoftIndexFileStore` cannot detect expired entries, which can lead to excessive usage of space on the file system.
+
+.Segmentation
+
+Soft-index file stores are always segmented.
+
+[NOTE]
+====
+The `AdvancedStore.purgeExpired()` method is not implemented in `SoftIndexFileStore`.
+====
+
+[discrete]
+== Single File Cache Stores
+
+[NOTE]
+====
+Single file cache stores are now deprecated and planned for removal.
+====
+
+Single File cache stores, `SingleFileStore`, persist data to file.
+{brandname} also maintains an in-memory index of keys while keys and values are stored in the file.
+
+Because `SingleFileStore` keeps an in-memory index of keys and the location of values, it requires additional memory, depending on the key size and the number of keys.
+For this reason, `SingleFileStore` is not recommended for use cases where the keys are larger or there can be a larger number of them.
+
+In some cases, `SingleFileStore` can also become fragmented.
+If the size of values continually increases, available space in the single file is not used but the entry is appended to the end of the file.
+Available space in the file is used only if an entry can fit within it.
+Likewise, if you remove all entries from memory, the single file store does not decrease in size or become defragmented.
+
+.Segmentation
+
+Single file cache stores are segmented by default with a separate instance per segment, which results in multiple directories.
+Each directory is a number that represents the segment to which the data maps.

documentation/src/main/asciidoc/topics/config_examples/global_persistent_location.xml

@@ -0,0 +1,5 @@
+<cache-container default-cache="myCache">
+   <global-state>
+      <persistent-location path="example" relative-to="my.data"/>
+   </global-state>
+</cache-container>

documentation/src/main/asciidoc/topics/config_examples/persistence_custom_file_store.xml

@@ -4,7 +4,7 @@
       <store class="org.acme.CustomStore"
              fetch-state="false"
              preload="true"
-             shared="false"
+             shared="true"
              purge="true"
              read-only="false"
              segmented="true">

documentation/src/main/asciidoc/topics/config_examples/persistence_file_store.xml

@@ -2,10 +2,9 @@
    <file-store shared="false"
                preload="true"
                fetch-state="true"
-               read-only="false"
-               purge="true">
-      <data path="${java.io.tmpdir}/data"/>
-      <index path="${java.io.tmpdir}/index"/>
+               read-only="false">
+      <data path="data"/>
+      <index path="index"/>
       <write-behind modification-queue-size="123" />
    </file-store>
 </persistence>

documentation/src/main/asciidoc/topics/config_examples/persistence_single_file_store.xml

@@ -0,0 +1,6 @@
+<persistence passivation="false">
+  <single-file-store shared="false"
+                     preload="true"
+                     fetch-state="true"
+                     read-only="false"/>
+</persistence>

documentation/src/main/asciidoc/topics/proc_configuring_cache_stores.adoc

@@ -1,5 +1,9 @@
-[id='configuring_cache_stores-{context}']
+[id='configuring-cache-stores_{context}']
 = Configuring Cache Stores
+
+Cache stores connect {brandname} to persistent data sources and implement the
+`NonBlockingStore` interface.
+
 Add cache stores to {brandname} caches in a chain either declaratively or
 programmatically. Cache read operations check each cache store in the
 configured order until they locate a valid non-null element of data. Write
@@ -14,7 +18,7 @@ only.
 Use either the `shared` attribute declaratively or the `shared(false)` method programmatically.
 +
 . Configure other cache stores properties as appropriate. Custom cache stores can also include `property` parameters.
-+
+
 [NOTE]
 ====
 Configuring cache stores as shared or not shared (local only) determines which
@@ -27,29 +31,20 @@ state and purge on startup. However, if the cache store is shared, then you
 should not fetch state or purge on startup.
 ====
 
-.Local (non-shared) file store
+.Local, non-shared file store
 
 [source,xml,options="nowrap",subs=attributes+]
 ----
 include::config_examples/persistence_file_store.xml[]
 ----
 
-.Shared custom cache store
+.Custom, shared cache store
 
 [source,xml,options="nowrap",subs=attributes+]
 ----
 include::config_examples/persistence_custom_file_store.xml[]
 ----
 
-.SoftIndex file store
-
-[source,java]
-----
-include::code_examples/SoftIndexFileStore.java[]
-----
-
-.Reference
-
+[role="_additional-resources"]
+.Additional resources
 * link:{configdocroot}[{brandname} Configuration Schema]
-* link:#cache_store_implementations[{brandname} Cache Store Implementations]
-* link:#create_custom_cache_store[Creating Custom Cache Stores]

documentation/src/main/asciidoc/topics/proc_configuring_file_stores.adoc

@@ -0,0 +1,40 @@
+[id='configuring-file-stores_{context}']
+= Configuring File-Based Cache Stores
+Add file-based cache stores to {brandname} to persist data on the host filesystem.
+
+.Prerequisites
+
+* Install {brandname} Server if you use remote caches.
+* Enable global state and configure a global persistent location if you use embedded caches.
+
+.Procedure
+
+. Add the `persistence` element to your cache configuration.
+. Optionally specify `true` as the value for the `passivation` attribute to write to the file-based cache store only when data is evicted from memory.
+. Include the `file-store` element and configure attributes as appropriate.
+. Specify `false` as the value for the `shared` attribute.
++
+File-based cache stores should always be unique to each {brandname} instance.
+If you want to use the same persistent across a cluster, configure shared storage such as a JDBC string-based cache store .
++
+. Configure the `index` and `data` elements to specify the location where {brandname} creates indexes and stores data.
+. Include the `write-behind` element to configure the cache store as write behind instead of as write through.
+
+.Directory location
+
+{brandname} creates file-based cache stores in the directory that you configure, relative to the global persistent location.
+For {brandname} this is the `{server_home}/server/data` directory.
+For embedded caches, {brandname} defaults to the `user.dir` system property.
+
+You should never configure an absolute path for a file-based cache store that is outside the global persistent location.
+If you do, {brandname} writes the following exception to logs:
+
+----
+ISPN000558: "The store location 'foo' is not a child of the global persistent location 'bar'"
+----
+
+.File-based cache store configuration
+[source,xml,options="nowrap",subs=attributes+]
+----
+include::config_examples/persistence_file_store.xml[]
+----