Make flatfs robust #64
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Stebalien
force-pushed
the
feat/improve-flatfs
branch
from
b9df39e
to
9ba2c75
Compare
dirkmc
reviewed
Feb 14, 2020
hsanjuan
requested changes
Feb 14, 2020
While this datastore is still quite limited, we might as well support everything we easily can.
That way, we're finally implementing everything.
Only allow keys of the form `/[0-9A-Z+-_=]`. That is, upper-case alphanumeric keys in the root namespace (plus some special characters). Why? We don't encode keys before writing them to the filesystem. This change ensures that: 1. Case sensitivity doesn't matter because we only allow upper-case keys. 2. Path separators and special characters doesn't matter. For context, go-ipfs only uses flatfs for storing blocks. Every block CID is encoded as uppercase alphanumeric text (specifically, uppercase base32). We could be less restrictive, but this is safer and easier to understand. Unfortunately, we _can't_ allow mixed case (Windows) and can't allow lowercase because we're already using uppercase keys. fixes #23
Stebalien
force-pushed
the
feat/improve-flatfs
branch
from
9ba2c75
to
4ca877d
Compare
General maintenance. I needed to pull through a fix for prefix handling in the query logic. |
hsanjuan
approved these changes
Feb 14, 2020
|
Errors in tests, seems circle is building with old Go ? |
Stebalien
force-pushed
the
feat/improve-flatfs
branch
from
35b5c51
to
9e32f65
Compare
Updated go-log. |
|
Oh, and probably old go. |
This was referenced May 6, 2020
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Importantly, this change only allows keys of the form
/[0-9A-Z+-_=]. That is, upper-case alphanumeric keys in the root namespace (plus some special characters).Why? We don't encode keys before writing them to the filesystem. This change ensures that:
For context, go-ipfs only uses flatfs for storing blocks. Every block CID is encoded as uppercase alphanumeric text (specifically, uppercase base32).
We could be less restrictive, but this is safer and easier to understand. Unfortunately, we can't allow mixed case (Windows) and can't allow lowercase because we're already using uppercase keys.
fixes #32, #61, #40, #22
tests with the query test suite