From 745f82e0f994ea045ed54f20cfc6d0e107ffd721 Mon Sep 17 00:00:00 2001 From: Daniel Aleksandersen Date: Sun, 13 Jan 2019 05:05:10 +0000 Subject: [PATCH] Clarify that chunker sizes are in bytes Clarify that the Rabin fingerprint chunker is in bytes, and recommend using larger chunk sizes than shown in the provided examples. People are getting confused over chunker sizes and incorrectly assuming they're in kiB and not bytes. License: MIT Signed-off-by: Daniel Aleksandersen --- core/commands/add.go | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/core/commands/add.go b/core/commands/add.go index b315c1a4e01..357eba6dc59 100644 --- a/core/commands/add.go +++ b/core/commands/add.go @@ -78,12 +78,16 @@ You can now refer to the added file in a gateway, like so: The chunker option, '-s', specifies the chunking strategy that dictates how to break files into blocks. Blocks with same content can -be deduplicated. The default is a fixed block size of +be deduplicated. Different chunking strategies will produce different +hashes for the same file. The default is a fixed block size of 256 * 1024 bytes, 'size-262144'. Alternatively, you can use the -rabin chunker for content defined chunking by specifying -rabin-[min]-[avg]-[max] (where min/avg/max refer to the resulting -chunk sizes). Using other chunking strategies will produce -different hashes for the same file. +Rabin fingerprint chunker for content defined chunking by specifying +rabin-[min]-[avg]-[max] (where min/avg/max refer to the desired +chunk sizes in bytes), e.g. 'rabin-262144-524288-1048576'. + +The following examples use very small byte sizes to demonstrate the +properties of the different chunkers on a small file. You'll likely +want to use a 1024 times larger chunk sizes for most files. > ipfs add --chunker=size-2048 ipfs-logo.svg added QmafrLBfzRLV4XSH1XcaMMeaXEUhDJjmtDfsYU95TrWG87 ipfs-logo.svg