Feat/v2 commp cid arbitrary size #6
Conversation
| if size*128 < size { | ||
| return 0, fmt.Errorf("unsupported size: too big") |
There was a problem hiding this comment.
TODO: This was just easier for now so I didn't have to worry about integer overflow things to get something basic up, but something that should probably be dealt with
| if dataSize < 127 { | ||
| return 0, 0, fmt.Errorf("unsupported size: too small") |
There was a problem hiding this comment.
TODO: Needs some decision here filecoin-project/FIPs#808 (comment)
There was a problem hiding this comment.
FWIW I am going forward with mandatory 127-pad https://github.com/ribasushi/go-toolbox-interplanetary/blob/master/fil/cid_test.go#L43-L65
aschmahmann
force-pushed
the
feat/v2-commp-cid-arbitrary-size
branch
from
d86ce67
to
ec6db9a
Compare
| // Fr32PaddedSizeToV1TreeHeight calculates the height of the piece tree given data that's been FR32 padded. Because | ||
| // pieces are only defined on binary trees if the size is not a power of 2 it will be rounded up to the next one under | ||
| // the assumption that the rest of the tree will be padded out (e.g. with zeros) | ||
| func Fr32PaddedSizeToV1TreeHeight(size uint64) uint8 { |
There was a problem hiding this comment.
I would drop this function entirely: what is its purpose...?
| // tree before any FR32 padding is applied. Because pieces are only defined on binary trees of FR32 encoded data if the | ||
| // size is not a power of 2 after the FR32 padding is applied it will be rounded up to the next one under the assumption | ||
| // that the rest of the tree will be padded out (e.g. with zeros) | ||
| func UnpaddedSizeToV1TreeHeight(size uint64) (uint8, error) { |
There was a problem hiding this comment.
Drop this entire function - just have the next one alone.
| // encoded data if the size is not a power of 2 after the FR32 padding is applied it will be rounded up to the next one | ||
| // under the assumption that the rest of the tree will be padded out (e.g. with zeros). The amount of data padding that | ||
| // is needed to be applied is returned alongside the tree height. | ||
| func UnpaddedSizeToV1TreeHeightAndPadding(dataSize uint64) (uint8, uint64, error) { |
There was a problem hiding this comment.
Clearer? The entire FR32 "padding/unpadding" "expansion" business is an internal implementation detail anyway...
| func UnpaddedSizeToV1TreeHeightAndPadding(dataSize uint64) (uint8, uint64, error) { | |
| func PayloadSizeToV1TreeHeightAndPadding(dataSize uint64) (uint8, uint64, error) { |
| // - hash type: multihash.SHA2_256_TRUNC254_PADDED_BINARY_TREE | ||
| // | ||
| // The helpers UnpaddedSizeToV1TreeHeight and Fr32PaddedSizeToV1TreeHeight may help in computing tree height | ||
| func DataCommitmentV1ToPieceMhCID(commD []byte, unpaddedDataSize uint64) (cid.Cid, error) { |
There was a problem hiding this comment.
Allow one to supply 0 as "don't care" ?
| func DataCommitmentV1ToPieceMhCID(commD []byte, unpaddedDataSize uint64) (cid.Cid, error) { | |
| func DataCommitmentV1ToPieceMhCID(commD []byte, optionalPayloadSize uint64) (cid.Cid, error) { |
| // piece multihash CID | ||
| // | ||
| // The helpers UnpaddedSizeToV1TreeHeight and Fr32PaddedSizeToV1TreeHeight may help in computing tree height | ||
| func ConvertDataCommitmentV1V1CIDtoPieceMhCID(v1PieceCid cid.Cid, unpaddedDataSize uint64) (cid.Cid, error) { |
There was a problem hiding this comment.
Too many V1s ? Also... the helper seems overly trivial, perhaps drop it? If kept: simpler name below:
| func ConvertDataCommitmentV1V1CIDtoPieceMhCID(v1PieceCid cid.Cid, unpaddedDataSize uint64) (cid.Cid, error) { | |
| func PieceCidV2FromV1(v1PieceCid cid.Cid, unpaddedDataSize uint64) (cid.Cid, error) { |
|
|
||
| // ConvertDataCommitmentV1PieceMhCIDToV1CID takes a piece multihash CID and produces a v1 piece CID along with the | ||
| // size of the unpadded data | ||
| func ConvertDataCommitmentV1PieceMhCIDToV1CID(pieceMhCid cid.Cid) (cid.Cid, uint64, error) { |
There was a problem hiding this comment.
Mixing "MH" into the names of things that do not return MHs ( but cid.Cid ) is... nasty.
| func ConvertDataCommitmentV1PieceMhCIDToV1CID(pieceMhCid cid.Cid) (cid.Cid, uint64, error) { | |
| func PieceCidV1FromV2(pcidV2 cid.Cid) (cid.Cid, uint64, error) { |
Builds off of #5 with an alternative v2 cid representation that handles arbitrary sizes