ARTEMIS-4065 Non Persistent Page Counters #4265
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
I stil have both versions of the counters in the codebase. I am going to remove the other implementation of the PageCounter and only keep the non persistent. This PR is for a draft only.. and I will finish after fixing the testsuite (the built server would work fine though) |
clebertsuconic
commented
Oct 21, 2022
...java/org/apache/activemq/artemis/core/paging/cursor/impl/NonPersistentPagingCounterImpl.java
Outdated
Show resolved
Hide resolved
clebertsuconic
commented
Oct 21, 2022
clebertsuconic
force-pushed
the
page-counter
branch
8 times, most recently
from
9826735
to
e994c51
Compare
clebertsuconic
force-pushed
the
page-counter
branch
2 times, most recently
from
7320153
to
21e2942
Compare
|
I updated this PR with a new commit. From now on we save snapshots of the counter upon shutdown. |
clebertsuconic
force-pushed
the
page-counter
branch
6 times, most recently
from
8116f5c
to
e289d3f
Compare
clebertsuconic
force-pushed
the
page-counter
branch
8 times, most recently
from
b6b84d6
to
53d5d26
Compare
gemmellr
reviewed
Dec 12, 2022
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/SizeAwareMetric.java
Outdated
Show resolved
Hide resolved
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/SizeAwareMetric.java
Outdated
Show resolved
Hide resolved
artemis-commons/src/main/java/org/apache/activemq/artemis/utils/SizeAwareMetric.java
Outdated
Show resolved
Hide resolved
artemis-server/src/main/java/org/apache/activemq/artemis/core/paging/PagingManager.java
Outdated
Show resolved
Hide resolved
...-server/src/main/java/org/apache/activemq/artemis/core/paging/cursor/PageCursorProvider.java
Outdated
Show resolved
Hide resolved
...main/java/org/apache/activemq/artemis/core/paging/cursor/impl/PageCounterRebuildManager.java
Outdated
Show resolved
Hide resolved
...main/java/org/apache/activemq/artemis/core/paging/cursor/impl/PageCounterRebuildManager.java
Outdated
Show resolved
Hide resolved
...rc/main/java/org/apache/activemq/artemis/core/paging/cursor/impl/PageCursorProviderImpl.java
Show resolved
Hide resolved
...in/java/org/apache/activemq/artemis/core/paging/cursor/impl/PageSubscriptionCounterImpl.java
Outdated
Show resolved
Hide resolved
.../src/main/java/org/apache/activemq/artemis/core/paging/cursor/impl/PageSubscriptionImpl.java
Outdated
Show resolved
Hide resolved
clebertsuconic
force-pushed
the
page-counter
branch
5 times, most recently
from
8bd24b5
to
120d102
Compare
|
@gemmellr I will do some testing to make sure it's solid and I should merge it.. unless you have any other concerns. |
|
@clebertsuconic still need to look over the rest, plus whatever changed since yesterday. |
clebertsuconic
force-pushed
the
page-counter
branch
from
120d102
to
4282193
Compare
gemmellr
reviewed
Dec 13, 2022
...mis-server/src/main/java/org/apache/activemq/artemis/core/paging/impl/PagingManagerImpl.java
Show resolved
Hide resolved
...mis-server/src/main/java/org/apache/activemq/artemis/core/paging/impl/PagingManagerImpl.java
Outdated
Show resolved
Hide resolved
artemis-server/src/main/java/org/apache/activemq/artemis/core/server/ActiveMQMessageBundle.java
Outdated
Show resolved
Hide resolved
artemis-server/src/test/java/org/apache/activemq/artemis/tests/util/ActiveMQTestBase.java
Outdated
Show resolved
Hide resolved
tests/compatibility-tests/src/main/resources/pageCounter/checkMessages.groovy
Show resolved
Hide resolved
tests/compatibility-tests/src/main/resources/pageCounter/sendMessages.groovy
Show resolved
Hide resolved
...test/java/org/apache/activemq/artemis/tests/integration/paging/PageCountSyncOnNonTXTest.java
Outdated
Show resolved
Hide resolved
clebertsuconic
force-pushed
the
page-counter
branch
8 times, most recently
from
7ce9e7f
to
eb075e2
Compare
- From now on we will save snapshots of page-counters on the journal (basically for compatibility with previous verions). And we will recount the records on startup. - While the rebuild is being done the value from the previous snapshot is still available with current updates.
clebertsuconic
force-pushed
the
page-counter
branch
from
eb075e2
to
7a64afb
Compare
|
I had a few successful run with this PR over the full CI now. |
clebertsuconic
force-pushed
the
page-counter
branch
from
aa84ea0
to
7a64afb
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Instead of relying on the journal to store the value of the counters, and have it eventually getting out of sync we should just use real counters from reading the pages on startup.
In a test performed, reading 700 files didn't take more than 1 minute in a modest laptop. And besides the data will be available while the rebuild is being done.