-
-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Optimize backfill receiving to have less missing prev_event
thrashing (scratch)
#13864
Commits on Sep 21, 2022
-
Fix
have_seen_event
cache not being invalidatedFix #13856 `_invalidate_caches_for_event` doesn't run in monolith mode which means we never even tried to clear the `have_seen_event` and other caches. And even in worker mode, it only runs on the workers, not the master (AFAICT). Additionally there is bug with the key being wrong so `_invalidate_caches_for_event` never invalidates the `have_seen_event` cache even when it does run. Wrong: ```py self.have_seen_event.invalidate((room_id, event_id)) ``` Correct: ```py self.have_seen_event.invalidate(((room_id, event_id),)) ```
Configuration menu - View commit details
-
Copy full SHA for a847a35 - Browse repository at this point
Copy the full SHA a847a35View commit details -
Configuration menu - View commit details
-
Copy full SHA for f6393db - Browse repository at this point
Copy the full SHA f6393dbView commit details
Commits on Sep 22, 2022
-
Fix
have_seen_event
cache not being invalidatedFix #13856 `_invalidate_caches_for_event` doesn't run in monolith mode which means we never even tried to clear the `have_seen_event` and other caches. And even in worker mode, it only runs on the workers, not the master (AFAICT). Additionally there is bug with the key being wrong so `_invalidate_caches_for_event` never invalidates the `have_seen_event` cache even when it does run. Wrong: ```py self.have_seen_event.invalidate((room_id, event_id)) ``` Correct: ```py self.have_seen_event.invalidate(((room_id, event_id),)) ```
Configuration menu - View commit details
-
Copy full SHA for d07947d - Browse repository at this point
Copy the full SHA d07947dView commit details -
Configuration menu - View commit details
-
Copy full SHA for f2a5c70 - Browse repository at this point
Copy the full SHA f2a5c70View commit details -
Calculate the stream_ordering from newest -> oldest (in the correct o…
…rder) and persist in the oldest -> newest to get the least missing prev_event fetch thrashing
Configuration menu - View commit details
-
Copy full SHA for b23b3e4 - Browse repository at this point
Copy the full SHA b23b3e4View commit details -
Configuration menu - View commit details
-
Copy full SHA for a25821d - Browse repository at this point
Copy the full SHA a25821dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 5a5c324 - Browse repository at this point
Copy the full SHA 5a5c324View commit details -
Merge branch 'develop' into madlittlemods/13856-fix-have-seen-events-…
…not-being-invalidated
Configuration menu - View commit details
-
Copy full SHA for 1054f91 - Browse repository at this point
Copy the full SHA 1054f91View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2162ab5 - Browse repository at this point
Copy the full SHA 2162ab5View commit details -
Fix
@cachedList
on_have_seen_events_dict
As mentioned by @erikjohnston, #13865 (comment)
Configuration menu - View commit details
-
Copy full SHA for 0cdc7bf - Browse repository at this point
Copy the full SHA 0cdc7bfView commit details
Commits on Sep 23, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 5b9b645 - Browse repository at this point
Copy the full SHA 5b9b645View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9fb750d - Browse repository at this point
Copy the full SHA 9fb750dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 4fa8f05 - Browse repository at this point
Copy the full SHA 4fa8f05View commit details -
Configuration menu - View commit details
-
Copy full SHA for b9be6c5 - Browse repository at this point
Copy the full SHA b9be6c5View commit details -
Configuration menu - View commit details
-
Copy full SHA for f8dc17b - Browse repository at this point
Copy the full SHA f8dc17bView commit details -
Merge branch 'madlittlemods/13856-fix-have-seen-events-not-being-inva…
…lidated' into maddlittlemods/msc2716-many-batches-optimization Conflicts: tests/storage/databases/main/test_events_worker.py
Configuration menu - View commit details
-
Copy full SHA for 44e9746 - Browse repository at this point
Copy the full SHA 44e9746View commit details
Commits on Sep 24, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 33d12a5 - Browse repository at this point
Copy the full SHA 33d12a5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 78b4434 - Browse repository at this point
Copy the full SHA 78b4434View commit details
Commits on Sep 26, 2022
-
Merge branch 'madlittlemods/13356-messages-investigation-scratch-v1' …
…into maddlittlemods/msc2716-many-batches-optimization Conflicts: synapse/handlers/federation.py synapse/storage/databases/main/cache.py synapse/storage/databases/main/event_federation.py
Configuration menu - View commit details
-
Copy full SHA for 85451b9 - Browse repository at this point
Copy the full SHA 85451b9View commit details
Commits on Sep 27, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 31e2c10 - Browse repository at this point
Copy the full SHA 31e2c10View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7a3ded0 - Browse repository at this point
Copy the full SHA 7a3ded0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6423938 - Browse repository at this point
Copy the full SHA 6423938View commit details -
Configuration menu - View commit details
-
Copy full SHA for 20f4d1c - Browse repository at this point
Copy the full SHA 20f4d1cView commit details
Commits on Sep 28, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 62f35ea - Browse repository at this point
Copy the full SHA 62f35eaView commit details
Commits on Sep 29, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 4dcb2f6 - Browse repository at this point
Copy the full SHA 4dcb2f6View commit details -
I think working same as Complement reverse_chronological, only the in…
…sertion event rejected
Configuration menu - View commit details
-
Copy full SHA for 50b11cb - Browse repository at this point
Copy the full SHA 50b11cbView commit details -
Configuration menu - View commit details
-
Copy full SHA for cfa5e57 - Browse repository at this point
Copy the full SHA cfa5e57View commit details -
Configuration menu - View commit details
-
Copy full SHA for aaa9679 - Browse repository at this point
Copy the full SHA aaa9679View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1ed0276 - Browse repository at this point
Copy the full SHA 1ed0276View commit details -
Configuration menu - View commit details
-
Copy full SHA for 61c1296 - Browse repository at this point
Copy the full SHA 61c1296View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5faebbd - Browse repository at this point
Copy the full SHA 5faebbdView commit details -
WIP: Connect state_chain to prev_event and the batch to the state_cha…
…in so everyhting is valid We are going to lose the benefit of keeping the join noise out of the timeline. And will probably have to hide "historical" state on the client.
Configuration menu - View commit details
-
Copy full SHA for 8f8c1a0 - Browse repository at this point
Copy the full SHA 8f8c1a0View commit details
Commits on Sep 30, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 3f8fef2 - Browse repository at this point
Copy the full SHA 3f8fef2View commit details
Commits on Oct 13, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 8a69706 - Browse repository at this point
Copy the full SHA 8a69706View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0b104e3 - Browse repository at this point
Copy the full SHA 0b104e3View commit details -
Configuration menu - View commit details
-
Copy full SHA for c5b1ce8 - Browse repository at this point
Copy the full SHA c5b1ce8View commit details