Fix warnings on block-cid mismatch when adding #1632
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Because we are adding blocks on a single call, and we choose the format parameter based on the prefix of the first block, IPFS will return block CIDs based on that option. This caused warnings when adding content that has multiple CID prefixes: for example, any cid-version=1 file will include both dag-pb CIDs and raw CIDs. Since the first block is usually a leave, IPFS will only return raw-cids, and cause a warning because of the CID-mistmatch. This fixes things by comparing multihashes only. But! We might be writing blocks with the wrong CID and then the good CID won't work! Correct, we might, in some corner cases. In go-ipfs >= 0.12.0, all blocks are addressed by multihash so CID prefixes are irrelevant. This problem does not exist in that case. In go-ipfs < 0.12.0, if a read for a CIDv1 DAG-PB fails, it is retried as it it was raw. This means that if we wrote something with cidv1/format=raw, that should have been a cidv1/format=dag-pb, the read will still work. That covers some common cases (i.e. adding with cid-version=1) because the first block should be a raw-leaf. Default-params (cidv0) is not affected since everything is raw multihashes. However, there are still possible CAR layouts etc. where cluster will write blocks wrongly to older IPFS versions.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Because we are adding blocks on a single call, and we choose the format
parameter based on the prefix of the first block, IPFS will return block CIDs
based on that option.
This caused warnings when adding content that has multiple CID prefixes: for
example, any cid-version=1 file will include both dag-pb CIDs and raw
CIDs. Since the first block is usually a leave, IPFS will only return
raw-cids, and cause a warning because of the CID-mistmatch.
This fixes things by comparing multihashes only.
But! We might be writing blocks with the wrong CID and then the good CID won't
work!
Correct, we might, in some corner cases.
In go-ipfs >= 0.12.0, all blocks are addressed by multihash so CID prefixes
are irrelevant. This problem does not exist in that case.
In go-ipfs < 0.12.0, if a read for a CIDv1 DAG-PB fails, it is retried as it
it was raw. This means that if we wrote something with cidv1/format=raw, that
should have been a cidv1/format=dag-pb, the read will still work. That covers
some common cases (i.e. adding with cid-version=1) because the first block
should be a raw-leaf. Default-params (cidv0) is not affected since everything
is raw multihashes. However, there are still possible CAR layouts etc. where
cluster will write blocks wrongly to older IPFS versions.