-
Notifications
You must be signed in to change notification settings - Fork 38
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unify blobstor interfaces #1584
Conversation
Codecov Report
@@ Coverage Diff @@
## master #1584 +/- ##
==========================================
- Coverage 33.21% 32.58% -0.64%
==========================================
Files 332 337 +5
Lines 22752 22651 -101
==========================================
- Hits 7558 7380 -178
- Misses 14574 14658 +84
+ Partials 620 613 -7
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- is it worth changing the
writecache
behavior in that PR? - don't you think we could get problems with general parameters? (e.g. some
Storage
implementation starts requiring some parameter that another does not need at all) - commits do not start with the issue number
pkg/local_object_storage/blobstor/blobovniczatree/blobovnicza.go
Outdated
Show resolved
Hide resolved
cmd/neofs-node/config/engine/shard/blobstor/blobovnicza/config.go
Outdated
Show resolved
Hide resolved
Yes, it reduces the amount of code that needs to be written. This implementation was tested on the experimental branch, it leads to a more uniform speed.
This is the point -- each object storage should have the same interface. If any implementation starts requiring some additional parameter, we will provide it in the generic structure, it will be unused by the others. However, I cannot imagine this will ever become true.
Fixed. |
Rebased on master. |
) | ||
|
||
// CConfig represents common compression-related configuration. | ||
type CConfig struct { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why double C
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It was meant as Compression Config
and embedded struct named Config
would be rather misleading. I guess I'll just remove confusing embeddings and use compression.Config
as a name.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed
require.ErrorIs(t, err, expectedErr) | ||
}) | ||
t.Skip() | ||
//dir := t.TempDir() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Commented because can't be compiled?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, it became a bit harder to test. I have created a #1686 to get more test coverage: we could test iterate in every component, and then use a dummy one when testing blobstor. Because in blobstor we are really interested only in how errors from subcomponents can be returned to the caller.
…arate package Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
… operation Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
…ange` operation Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
…te` operation Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
…ts` operation Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
…e` methods Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Allow to extend blobstor with more storage sub-systems. Currently objects stored in the FSTree have empty byte slice descriptor and object from blobovnicza tree have the same id as earlier. Each such change in the identifier formation should be accompanied with metabase version increase. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
…ate` operation Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
1. Remove in-memory cache. It doesn't persist objects and if we want more speed, `NoSync` option can be used for the bolt DB. 2. Put to the metabase in a synchronous fashion. This considerably simplifies overall logic and plays nicely with the metabase bolt DB batch settings. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
1. Move compression parameters to the `shard` section. 2. Allow to use multiple sub-storage components in the blobstor. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Close #1523.
Preparation for the #1522 .