Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GapAwareToken should minimize the number of gaps #391

Closed
abuijze opened this issue Aug 1, 2017 · 1 comment
Closed

GapAwareToken should minimize the number of gaps #391

abuijze opened this issue Aug 1, 2017 · 1 comment
Assignees
@abuijze
Labels
Priority 1: Must Highest priority. A release cannot be made if this issue isn’t resolved. Status: Resolved Use to signal that work on this issue is done. Type: Enhancement Use to signal an issue enhances an already existing feature of the project.
Milestone
Release 3.0.6

Comments

@abuijze
Copy link
Member

abuijze commented Aug 1, 2017

The JpaEventStorageEngine (as well as JdbcEventStorageEngine) use a GapAwareTrackingToken to ensure that they don't accidentally skip an event that may have been inserted before, but committed after another event.

However, gaps that are older than a certain threshold (e.g. a minute) don't need to be reported as gaps anymore, as they don't represent an uncommitted event, but rather a rolled back one or a gap in the sequence provider.

Therefore, the Jpa and JdbcEventStorageEngine should remove gaps from any incoming and/or outgoing tokens if they were older than this threshold. The gaps age can be calculated from the timestamp of the event following any gap.
@abuijze abuijze added the Type: Enhancement Use to signal an issue enhances an already existing feature of the project. label Aug 1, 2017
@abuijze abuijze added this to the Release 3.0.6 milestone Aug 1, 2017
abuijze added a commit that referenced this issue Aug 14, 2017
@abuijze
Added mechanism for cleaning large amounts of gaps in tokens
c0102bf 
When consuming old events, skipped sequence numbers will not register a gap in
 the token. When a token is submitted with a large amount of gaps, the Event
 Storage engine will attempt to clean the gaps before issuing a query for
 events.

Resolves issue #391
@abuijze abuijze closed this as completed Aug 14, 2017
@askfor
Copy link
Contributor

askfor commented Nov 7, 2017

AxonFramework/core/src/main/java/org/axonframework/eventsourcing/eventstore/jpa/JpaEventStorageEngine.java

Line 163 in c0102bf

"WHERE e.globalIndex > :token OR e.globalIndex IN (:gaps) ORDER BY e.globalIndex ASC",

.setParameter("gaps", previousToken.getGaps());

Oracle has a limit of 1000 items for in clause. So this could cause some problem?

@smcvb smcvb added Priority 1: Must Highest priority. A release cannot be made if this issue isn’t resolved. Status: Resolved Use to signal that work on this issue is done. labels Feb 28, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@abuijze abuijze

Labels
Priority 1: Must Highest priority. A release cannot be made if this issue isn’t resolved. Status: Resolved Use to signal that work on this issue is done. Type: Enhancement Use to signal an issue enhances an already existing feature of the project.
Projects
None yet
Milestone
Release 3.0.6
Development

No branches or pull requests
3 participants
@askfor @abuijze @smcvb