JAMES-3105 Corrective task for fixing mailbox counters #3185
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Resetting counters is intrinsecly flowed in the face of concurrent operations.
Algorithm: - List existing mailboxes - List their messages - Check them against their source of truth - Compute mailbox counter values - And reset the value of the counter if needed Limitations: This won't be friendly in the face of concurrent operations, which would be ignored during the recomputation of the counter of the given mailbox. However the source of truth is unaffected hence, upon rerunning the algorithm, the result will be eventually correct. We rely on the "listing messages by mailbox" projection (that we recheck). Missing entries in there will be ignored until the given projection is healed (currently unsupported).
A simple wrapper around the already existing service...
Proposed endpoint: ``` curl -XPST http://ip:port/mailboxes?task=RecomputeMailboxCounters ```
…ers recomputation We can piggy back a partial check of the message denormalization upon counter recomputation (partial because we cannot detect missing entries in the "list messages in mailbox" denormalization table)
Arsnael
reviewed
Mar 9, 2020
...sandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxCounterDAO.java
Outdated
Show resolved
Hide resolved
...sandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxCounterDAO.java
Outdated
Show resolved
Hide resolved
...sandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxCounterDAO.java
Outdated
Show resolved
Hide resolved
...sandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxCounterDAO.java
Outdated
Show resolved
Hide resolved
...sandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxCounterDAO.java
Outdated
Show resolved
Hide resolved
...t/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersServiceTest.java
Outdated
Show resolved
Hide resolved
...src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersTask.java
Outdated
Show resolved
Hide resolved
trantienduchn
approved these changes
Mar 9, 2020
Arsnael
approved these changes
Mar 9, 2020
blackheaven
reviewed
Mar 9, 2020
.../james/mailbox/cassandra/mail/task/RecomputeMailboxCountersTaskAdditionalInformationDTO.java
Outdated
Show resolved
Hide resolved
...sandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxCounterDAO.java
Outdated
Show resolved
Hide resolved
...sandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxCounterDAO.java
Show resolved
Hide resolved
blackheaven
approved these changes
Mar 9, 2020
Arsnael
approved these changes
Mar 9, 2020
will fix it |
chibenwa
added
waiting_merge
|
@chibenwa I know that you are optimistic, but please, wait for the CI to be green before labeling "waiting_merge" |
I merge when it's green, that's the meaning |
remk
approved these changes
Mar 9, 2020
...sandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxCounterDAO.java
Outdated
Show resolved
Hide resolved
...sandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxCounterDAO.java
Outdated
Show resolved
Hide resolved
|
Merged |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cassandra maintains a per mailbox projection for message count and unseen message count.
As with any projection, it can go out of sync, leading to inconsistent results being returned to the client, which is not acceptable.
We need to expose a corrective task, resetting mailbox counters to their correct value.
The proposed endpoint is:
This endpoints will:
Of course, this endpoint won't be friendly in the face of concurrent operations, which would be ignored during the recomputation of the counter of the given mailbox. However the source of truth is unaffected hence, upon rerunning the task, the result will be eventually correct.
To be noted that we rely on the "listing messages by mailbox" projection (that we recheck). Missing entries in there will be ignored until the given projection is healed (currently unsupported). We furthermore can piggy back a partial check of the message denormalization upon counter recomputation (partial because we cannot detect missing entries in the "list messages in mailbox" denormalization table)