-
Notifications
You must be signed in to change notification settings - Fork 63
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
JAMES-3105 Corrective task for fixing mailbox counters #3185
Conversation
Resetting counters is intrinsecly flowed in the face of concurrent operations.
Algorithm: - List existing mailboxes - List their messages - Check them against their source of truth - Compute mailbox counter values - And reset the value of the counter if needed Limitations: This won't be friendly in the face of concurrent operations, which would be ignored during the recomputation of the counter of the given mailbox. However the source of truth is unaffected hence, upon rerunning the algorithm, the result will be eventually correct. We rely on the "listing messages by mailbox" projection (that we recheck). Missing entries in there will be ignored until the given projection is healed (currently unsupported).
A simple wrapper around the already existing service...
Proposed endpoint: ``` curl -XPST http://ip:port/mailboxes?task=RecomputeMailboxCounters ```
…ers recomputation We can piggy back a partial check of the message denormalization upon counter recomputation (partial because we cannot detect missing entries in the "list messages in mailbox" denormalization table)
...sandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxCounterDAO.java
Outdated
Show resolved
Hide resolved
...sandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxCounterDAO.java
Outdated
Show resolved
Hide resolved
...sandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxCounterDAO.java
Outdated
Show resolved
Hide resolved
...sandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxCounterDAO.java
Outdated
Show resolved
Hide resolved
...sandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxCounterDAO.java
Outdated
Show resolved
Hide resolved
...t/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersServiceTest.java
Outdated
Show resolved
Hide resolved
...src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersTask.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Read it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 with the fixups
.../james/mailbox/cassandra/mail/task/RecomputeMailboxCountersTaskAdditionalInformationDTO.java
Outdated
Show resolved
Hide resolved
...sandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxCounterDAO.java
Outdated
Show resolved
Hide resolved
...sandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxCounterDAO.java
Show resolved
Hide resolved
will fix it |
@chibenwa I know that you are optimistic, but please, wait for the CI to be green before labeling "waiting_merge" |
I merge when it's green, that's the meaning |
...sandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxCounterDAO.java
Outdated
Show resolved
Hide resolved
...sandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxCounterDAO.java
Outdated
Show resolved
Hide resolved
Merged |
Cassandra maintains a per mailbox projection for message count and unseen message count.
As with any projection, it can go out of sync, leading to inconsistent results being returned to the client, which is not acceptable.
We need to expose a corrective task, resetting mailbox counters to their correct value.
The proposed endpoint is:
This endpoints will:
Of course, this endpoint won't be friendly in the face of concurrent operations, which would be ignored during the recomputation of the counter of the given mailbox. However the source of truth is unaffected hence, upon rerunning the task, the result will be eventually correct.
To be noted that we rely on the "listing messages by mailbox" projection (that we recheck). Missing entries in there will be ignored until the given projection is healed (currently unsupported). We furthermore can piggy back a partial check of the message denormalization upon counter recomputation (partial because we cannot detect missing entries in the "list messages in mailbox" denormalization table)