Various ADR regarding JMAP and blobStore performance enhancements #170
Conversation
|
|
||
| An HybridBlobStore will be created to choose between Cassandra and ObjectStorage implementations depending on the policies. | ||
|
|
||
| DeletedMessageVault, BlobExport & MailRepository will rely on LowCostStoragePolicy. Other BlobStore users will rely on SizeBasedStoragePolicy. |
There was a problem hiding this comment.
do we really need to use the policy based blob store for all our blob store uses? I'm not sure LowCostStoragePolicy and PerformantStoragePolicy are really worth it.
There was a problem hiding this comment.
LowCostStoragePolicy -> For DeletedMessageVault (mail headers)
There was a problem hiding this comment.
PerformantStoragePolicy might not be worth it though...
There was a problem hiding this comment.
I mean why configuring DeletedMessageVault with a particular blob store policy while we probably ever want to store its data in swift?
There was a problem hiding this comment.
because having to declare what we expect in term of storage policy is better than knowing that store X has this specific behavior
|
|
||
| Some performance tests will be run in order to evaluate the improvements. | ||
|
|
||
| ## Consequences |
There was a problem hiding this comment.
I wonder how it will interact with the blob garbage collection we plan to implement
There was a problem hiding this comment.
Good question!
And I have an answer for it: as you plan to add a generation component in the blob id, blobs will be different, and this idList will thus be specific for a generation. We can thus safely remove entries from it as we know there will be no new blobs generated with the old generation.
However ADR for blob deletion are not presented yet, thus detailing interactions is hard.
Another argument I have: blob deletion acts at the BlobStore level, while this proposed implementation will be implemented at the ObjectStorageBlobDAO level, which should limit conflicts.
chibenwa
force-pushed
the
JAMES-2920-adrs
branch
from
612b085
to
d7de4c6
Compare
|
|
||
| In case of a less than 5% improvement, the code will not be added to the codebase and the proposal will get the status 'rejected'. | ||
|
|
||
| We expect more data to be stored in Cassandra. |
There was a problem hiding this comment.
it would be nice to tell how much when we'll get the results
There was a problem hiding this comment.
That depends on the data, and proportion of small mails. Here only "production" like data would be relevant IMO
|
Merged |
The goal of these architecture decisions is to ensure a GetMessageList call does not end up calling the blob store.