flux-content - access content service
flux content load [--bypass-cache] blobref
flux content store [--bypass-cache] [--dry-run]
flux content flush
flux content dropcache
Each Flux instance implements an append-only, content addressable storage service, which stores blobs of arbitrary content under message digest keys termed "blobrefs".
flux content store accepts a blob on standard input, stores it, and prints the blobref on standard output.
flux content load accepts a blobref argument, retrieves the corresponding blob, and writes it to standard output.
After a store operation completes on any rank, the blob may be retrieved from any other rank.
The content service includes a cache on each broker which improves scalability. The flux content flush command initiates store requests for any dirty entries in the local cache and waits for them to complete. This is mainly used in testing. The flux content dropcache command drops all non-essential entries in the local cache; that is, entries which can be removed without data loss.
- -b, --bypass-cache
-
Bypass the in-memory cache, and directly access the backing store, if available (see below).
- -d, --dry-run
-
When used with flux content store, the blobref is computed locally and displayed on standard output, but the blob is not stored.
The rank 0 cache retains all content until a module providing the "content-backing" service is loaded which can offload content to some other place. The content-sqlite module provides this service, and is loaded by default.
Content database files are stored persistently on rank 0 if the persist-directory broker attribute is set to a directory name for the session. Otherwise they are stored in the directory defined by the scratch-directory attribute and are cleaned up when the instance terminates.
When one of these modules is loaded, it informs the rank 0 cache of its availability, which triggers the cache to begin offloading entries. Once entries are offloaded, they are eligible for expiration from the rank 0 cache.
If the module is unloaded, its contents are transferred back to the cache. This operation may fail if more content has been stored than can fit in memory, and so is only advisable early in the life of an instance.
The parameters affecting local cache expiration may be tuned with flux-setattr(1):
- content-purge-target-entries
-
The cache is purged to bring the number of cache entries less than or equal to this value (default 1048576).
- content-purge-target-size
-
The cache is purged to bring the sum of the size of cached blobs less than or equal to this value (default 16777216)
- content-purge-old-entry
-
Only entries that have not been accessed in old-entry heartbeat epochs are eligible for purge (default 5).
- content-purge-large-entry
-
Only entries with blob size greater than or equal to large-entry are purged to reach the size target (default 256).
Expiration becomes active on every heartbeat, when the cache exceeds one or both of the targets configured above. Dirty or invalid entries are not eligible for purge.
Some accounting info for the local cache can be viewed with flux-getattr(1):
- content-acct-entries
-
The total number of cache entries.
- content-acct-size
-
The sum of the size of cached blobs.
- content-acct-dirty
-
The number of dirty cache entries.
- content-acct-valid
-
The number of valid cache entries.
The cache is write-through with respect to the rank 0 cache; that is, a store operation does not receive a response until it is valid in the rank 0 cache.
The cache on rank 0 is write-back with respect to the backing store, if any; that is, a store operation may receive a response before it has been stored on the backing store.
The cache is hierarchical. Rank 0 (the root of the tree based overlay network) holds all blobs stored in the instance. Other ranks keep only what a they heuristically determine to be of benefit. On ranks > 0, a load operation that cannot be fulfilled from the local cache is "faulted" in from the level above it. A store operation that reaches a level that has already cached the same content is "squashed"; that is, it receives a response without traveling further up the tree.
Github: http://github.com/flux-framework