Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(sdk): EventCache fully uses RoomEvents/LinkedChunk #3230

Merged
merged 20 commits into from Mar 27, 2024

Conversation

Hywan
Copy link
Member

@Hywan Hywan commented Mar 18, 2024

EventCache uses a custom EventCacheStore trait to store all events for all rooms.

The idea of this PR is to use the newly introduced LinkedChunk in #3166. Instead of having one store for all events for all rooms, now there is one LinkedChunk/store per room.

Of course, @bnjbvr and I are discussing a lot, which explains why the actual EventCacheStore was already adopting some patterns from LinkedChunk. The migration was easier.

Several PR were necessary to make this work possible:

Briefly, this PR removes the EventCacheStore and the MemoryStore, and replace them by LinkedChunk.


@Hywan Hywan force-pushed the feat-sdk-event-cache-use-room-events branch 4 times, most recently from 6990ec4 to cd3671f Compare March 18, 2024 14:20
@Hywan Hywan force-pushed the feat-sdk-event-cache-use-room-events branch 2 times, most recently from 9720873 to 6222bbd Compare March 20, 2024 12:42
@Hywan Hywan force-pushed the feat-sdk-event-cache-use-room-events branch 3 times, most recently from 7acc135 to 1f0e3b7 Compare March 20, 2024 21:05
@Hywan Hywan marked this pull request as ready for review March 20, 2024 21:08
@Hywan Hywan requested a review from a team as a code owner March 20, 2024 21:08
@Hywan Hywan requested review from poljar and removed request for a team March 20, 2024 21:08
@Hywan
Copy link
Member Author

Hywan commented Mar 20, 2024

Marking the PR as Ready for review just to get the CI feedback, but it's not ready for a review :-). Everything seems in-place though, just checking. The history needs to be rewritten.

Copy link

codecov bot commented Mar 21, 2024

Codecov Report

Attention: Patch coverage is 88.11881% with 12 lines in your changes are missing coverage. Please review.

Project coverage is 83.60%. Comparing base (ab9e4f7) to head (11c3799).
Report is 1 commits behind head on main.

Files Patch % Lines
crates/matrix-sdk/src/event_cache/mod.rs 87.64% 11 Missing ⚠️
crates/matrix-sdk/src/event_cache/store.rs 91.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3230      +/-   ##
==========================================
+ Coverage   83.56%   83.60%   +0.04%     
==========================================
  Files         238      238              
  Lines       24598    24585      -13     
==========================================
- Hits        20556    20555       -1     
+ Misses       4042     4030      -12     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Hywan Hywan force-pushed the feat-sdk-event-cache-use-room-events branch 2 times, most recently from 30e91e6 to be9079a Compare March 21, 2024 12:56
@Hywan Hywan requested review from bnjbvr and removed request for poljar March 21, 2024 13:01
Copy link
Member

@bnjbvr bnjbvr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's sweet to finally connect the linked chunks implementation to the event cache, nicely done!

A few comments below, and I'd like to take another look after those have been addressed. Thanks!

crates/matrix-sdk/src/event_cache/store.rs Show resolved Hide resolved
crates/matrix-sdk/src/event_cache/store.rs Outdated Show resolved Hide resolved
crates/matrix-sdk/src/event_cache/store.rs Outdated Show resolved Hide resolved
crates/matrix-sdk/tests/integration/event_cache.rs Outdated Show resolved Hide resolved
crates/matrix-sdk/tests/integration/event_cache.rs Outdated Show resolved Hide resolved
crates/matrix-sdk/src/event_cache/mod.rs Outdated Show resolved Hide resolved
crates/matrix-sdk/src/event_cache/mod.rs Outdated Show resolved Hide resolved
crates/matrix-sdk/src/event_cache/mod.rs Outdated Show resolved Hide resolved
crates/matrix-sdk/src/event_cache/mod.rs Show resolved Hide resolved
crates/matrix-sdk/src/event_cache/mod.rs Outdated Show resolved Hide resolved
@Hywan Hywan requested a review from bnjbvr March 25, 2024 11:18
@Hywan Hywan force-pushed the feat-sdk-event-cache-use-room-events branch from bf88bea to 70fb96f Compare March 25, 2024 12:43
Copy link
Member

@bnjbvr bnjbvr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yay, great job, thank you!

crates/matrix-sdk/src/event_cache/mod.rs Show resolved Hide resolved
crates/matrix-sdk/src/event_cache/mod.rs Outdated Show resolved Hide resolved
crates/matrix-sdk/src/event_cache/mod.rs Show resolved Hide resolved
crates/matrix-sdk/src/event_cache/store.rs Outdated Show resolved Hide resolved
…` and `events`.

This patch implements the following wrapper methods (over
`LinkedChunk`): `push_gap`, `replace_gap_at` and `events`. This patch
also implements the `reset` method that clears/drops all chunks in the
`LinkedChunk`.
@Hywan Hywan force-pushed the feat-sdk-event-cache-use-room-events branch from 70fb96f to 755034e Compare March 27, 2024 08:09
Hywan added 17 commits March 27, 2024 09:22
The test `test_reset_while_backpaginating` was expecting a
race-condition, which no longer exists. It first initially tried to
assert a workaround about this race-condition. It doesn't hold anymore.
Rewrite the test to assert the (correct) new behaviour.
This patch ensures that operations on `RoomEvents` happen in one block,
by sharing the same lock.

2 new methods are created: `replace_all_events_by` and
`append_new_events`.
… unknown token.

Prior to this patch, in `RoomEventCacheInner::backpaginate`, when the
`token` validity was checked, and it was invalid:

* before calling `/messages`, `Err(EventCacheError::UnknownBackpaginationToken)` was returned,
* after calling `/messages`, `Ok(BackPaginationOutput::UnknownBackpaginationToken)` was returned.

This patch tries to uniformize this by only returning
`Ok(BackPaginationOutput::UnknownBackpaginationToken)`.

That's a tradeoff. It will probably be refactor later.

The idea is also to call `/messages` **before** taking the write-lock
of `RoomEvents`, otherwise it can keep the lock for up to 30secs in
this case. Also, checking the validity of the `token` **before** and
**after** `/messages` is not necessary: it can be done only after.
@Hywan Hywan force-pushed the feat-sdk-event-cache-use-room-events branch from 755034e to 11c3799 Compare March 27, 2024 08:41
@Hywan
Copy link
Member Author

Hywan commented Mar 27, 2024

I've rewritten the entire history of this patch. I think we are good!

@Hywan Hywan merged commit ac0bc95 into matrix-org:main Mar 27, 2024
35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants