v8 heap usage is larger than I'd expect with LL enabled.

[ara4n]

Steps to reproduce

I just did a heap snapshot about 20 minutes after launching Element Nightly and was surprised to see it contained 192,329 MatrixEvent objects and 157,401 RoomMember objects:

(heap available if needed).

This feels crazily high, given the whole point of membership Lazy Loading is to only load members for the rooms your client is interacting with, and I had only viewed 3 rooms, each with <50 users present, at that point in time.

It's possible that given this is after ~12h of incremental sync, processing E2EE traffic in all my other rooms had caused their memberships to be loaded (in order to speak E2EE) - but i'm still surprised this resulted in as many as 157K room members. The 35K non-member events also feels way too high.

Is it possible that threads is leaking events, or triggering membership loads which then undermines the memory improvements of lazy loading?

This might also be contributing to the OOM in element-hq/element-desktop#680

Outcome

What did you expect?

Roughly 300MB of heap to be in use on a fresh launch

What happened instead?

Roughly 600MB of heap to be in use, which may leak further, and might also be causing OOMs overnight. Will update this bug with further heapdumps which will hopefully differentiate between LL membership warming up and actual leaks.

Operating system

macOS 12.3 on M1

Application version

Nightly

How did you install the app?

No response

Homeserver

No response

Will you send logs?

No

[ara4n]

(This is with threads turned on)

[ara4n]

This is with 'only' 95 Thread objects running around, btw - although 95 is still high, given i hadn't consciously interacted with any threads, or viewed many rooms:

[ara4n]

It's not that related to processing a big incremental sync, as after a clean restart (after the app has immediately been running) it has roughly the same profile:

[ara4n]

Counting my number of encrypted rooms, and the total number of members in each room (counting duplicates twice):

e=0; me=0; for (r of mxMatrixClientPeg.get().getRooms()) { if (r.currentState.events.get('m.room.encryption')) { e++; me += r.currentState.events.get('m.room.member').size; } }; console.log(e,me);

1311 6419

So, i still have ~100K room members unaccounted for.

[ara4n]

...and looking at all rooms, i only have 34K different member events:

e=0; me=0; for (r of mxMatrixClientPeg.get().getRooms()) { if (true) { e++; me += r.currentState.events.get('m.room.member').size; } }; console.log(e,me);
rageshake.ts:72 3907 34445

I guess it's possible that these are being stored 4 times (two sentinels per room, across oldState and currentState or something?), which could bring us up to the ~120K mark.

This only averages 34445/3907 = 8 members per room, which isn't implausible, given we load the 20 most recent messages in each room, or whatever.

So, I don't think LL is being undermined (as from memory there are around 400K different room members visible from my account when LL is disabled) - the surprising number of members is more likely if we're cloning them 4x.

Separately, we may still have a leak somewhere accounting for the additional ~40K of events... but across 4000 rooms, this is only an additional 10 events per room, which sounds plausible (a few msgs of timeline + create + name + topic + PLs + any other random state). We might want to doublecheck that we're not preloading too much timeline.

Finally, we may have a leak causing element-hq/element-desktop#680 - i'll heapdump again later in the day to see what the difference is.