Skip to content

FCPv2 ClientPutDiskDir

DC* edited this page Aug 29, 2020 · 3 revisions

ClientPutDiskDir inserts an entire on-disk directory (including sub-directories) under a single key (technically, as a manifest file), so that each of the inserted files is located using the same key like this:

 CHK@NOSdw7FF88S....4BgOPxSPqv~bNg7YsgM,AAEC--8/file1.txt
 CHK@NOSdw7FF88S....4BgOPxSPqv~bNg7YsgM,AAEC--8/file2.jpg
 CHK@NOSdw7FF88S....4BgOPxSPqv~bNg7YsgM,AAEC--8/subdir/file3.html
 CHK@NOSdw7FF88S....4BgOPxSPqv~bNg7YsgM,AAEC--8/subdir/file4.ogg
 CHK@NOSdw7FF88S....4BgOPxSPqv~bNg7YsgM,AAEC--8/foo/bar/file5.pdf

Most of the fields below have the same usage as in ClientPut.

Example

 ClientPutDiskDir
 Identifier=My Identifier
 Verbosity=1023
 MaxRetries=999
 PriorityClass=2
 URI=CHK@
 GetCHKOnly=true
 DontCompress=true
 ClientToken=My Client Token
 Persistence=forever
 Global=true
 DefaultName=index.html
 Filename=/path/to/directory
 AllowUnreadableFiles=false // unless true, any unreadable files cause the whole request to fail; optional
 IncludeHiddenFiles=true // unless true, any "hidden" files will be ignored (from 1424)
 RealTimeFlag=false
 EndMessage

Parameters

>
Field Possible values Mandatory Default Description
Identifier Arbitrary text string yes This is just for client to be able to identify files that have been inserted.
Global [true,false] no false Whether the insert is visible on the global queue or not. If you set this as true, you should probably do a WatchGlobal, or you won't get any PutSuccessful / PutFailed message
URI CHK@
KSK@name
SSK@privateKey/docname-1
USK@privateKey/docname/0/
  
yes The type of key to insert. When inserting an SSK key, you explicitly specifiy the version number. For a USK key, use a zero and it should automatically use the correct version number.
ClientToken Arbitrary string no Returned to the client in the PersistentPut message if this is a persistent request.
Codecs a comma separated list of either names or ids no The node will try to compress the data with the given codec(s). If more then one codec is given the node will try to find the best in the given order. The node tells the supported codecs in the NodeHello message. This setting was introduced in 1231.
DontCompress [true,false] no false Hint to node: don't try to compress the data, it's already compressed
Filename yes The absolute name of the directory that is to be inserted
GetCHKOnly [true,false] no false If set to true, it won't actually insert the data, just return the key it would generate. If the key is USK, you may want to transform it into a SSK to prevent the node spending time searching for an unused index.
MaxRetries Integer -1 to 999999+? no 0 Number of times to retry if the first time doesn't work. -1 means retry forever.
PriorityClass [0..2..6] no 2 (Semi-interactive priority class) How to prioritize the insert.
Verbosity Bitmask no 0 Allows additional insertion status updates beyond the standard PutSuccessful and PutFailed updates.
Persistence [connection,reboot,forever] no connection Whether the insert stays on the queue across new client connections, Freenet restarts, or forever
TargetFilename A filename (no slashes) no site Filename to be appended to a CHK insert. Technically it creates a one-file manifest with this filename pointing to the file being uploaded. Ignored for all types other than CHK since other types have human-readable filenames anyway. Empty means no filename.
EarlyEncode [true,false] no false See EarlyEncode
ForkOnCacheable [true,false] Depends on node version true Override insert behaviour: If true, fork an insert after its HTL becomes low enough for it to be cacheable. If false, don't fork the insert at this point. Inserts are not cached while their HTL is above 15, after that they can be cached, but this can be many hops because of probabilistic decrement, hence they may go over the "ideal" nodes or "sinks" where they should have been cached while they are still not allowed to, and the insert will not go back to those nodes because it's already been there. Forking deals with this by creating a new insert, which can go back over nodes we've already been to if necessary.
ExtraInsertsSingleBlock Integer >= 0 no 2 (from 1297) Insert single blocks (not blocks above splitfiles, and not blocks in splitfiles) additional times. Testing has shown that a value of 2 (3 inserts total) can dramatically improve data persistence. We are still trying to figure out why.
ExtraInsertsSplitfileHeaderBlock Integer >=0 no 2 Insert blocks above splitfiles additional times. E.g. the CHK at the top of a splitfile, the CHK you actually fetch, is a single block. The splitfile itself has redundancy but the top block doesn't. So we insert it multiple times (3 by default) to improve the number of nodes it gets stored on. Tests show this dramatically improves data persistence, we are still trying to figure out why.
CompatibilityMode [COMPAT_CURRENT,] no COMPAT_CURRENT Insert the data in a way that is identical to how older nodes inserted it, not taking advantage of improvements in the metadata format, so that we can generate the same CHK and do reinserts, so that the old keys still work. COMPAT_1250 actually uses the 1251 version of even segment splitting, but doesn't add an extra block to smaller segments (so doesn't cause pre-1250 nodes to crash); COMPAT_1250_EXACT inserts segments of exactly 128 blocks except for the last one, so inserts exactly the same as older nodes. 1251 introduces even segment splitting (with extra blocks for sub-128 block segments), 1254 introduces better even segment splitting and a variety of other changes e.g. hashes, single splitfile key. It is likely that 1255 will include further metadata changes for large files.
LocalRequestOnly [true,false] no False If true, data will be inserted into the datastore, not sent to the network. Only useful for testing, as nobody will be able to find the data; also unsafe in that it inserts local content directly to your store.
OverrideSplitfileCryptoKey 256 bit key in hex no If set, the bottom layer of any splitfiles inserted will be encrypted using this key, rather than a key derived from the hash of the content. This is useful for reinserting data that was inserted to an SSK. SSKs automatically use a random key for the bottom layer splitfile as of 1254, unless CompatibilityMode is set to an earlier version. Note that upper layers (i.e. multi-level metadata) are still encrypted according to the data hash; they are not predictable so there is no need to complicate matters.
RealTimeFlag no false Whether to fetch the data with the real-time flag set to realtime (true) or bulk (false). (since 1311)
AllowUnreadableFiles [true,false] no false If false, unreadable files (e.g. files with insufficient permissions) are skipped; if true, unreadable files will result in a ProtocolError with a Code of 9.
includeHiddenFiles [true,false] no false Whether to include hidden files or not. On UNIX systems files with a leading dot (.) are hidden, on Windows systems files can have special attributes.
DefaultName no auto-detected The name of the file to return when the empty path (SSK@key/site-edition/) is requested, comparable to an index.html on a normal web server. If none is specified, index.html, index.htm, default.html, and default.htm are searched for in the inserted directory, first case-sensitive, then case-insensitive; the first one to be found is used. Otherwise there will be no default file, and users with a browser will receive an error page.
WriteToClientCache [true,] no false
IgnoreUSKDatehints [true,] no false If true, does not insert USK date hints.
Clone this wiki locally