Improve how users see the size of a Realm file

[kraenhansen]

With the current implementation of showing Realm sizes we have a few limitations:

Limitations

L1: Unnecessary computations

Sizes are computed for every Realm file - regardless of whether or not they've changed, this puts a lot of unnecessary work on a server.

L2: Major delay from change to visual feedback

Sizes are computed by the sync server on a schedule once every 30 minutes - which means that users can add a lot of data and then cannot trust the UI.

L3: Minor delay from change to visual feedback (on Realm Cloud)

On Realm Cloud we use Prometheus to store and retrieve the Realm size metrics and because these are read out from ROS on a schedule every 30 seconds, users will experience up to 30 seconds delay from the server emits stats and these are displayable in the UI.

L4: Non-reactive UI

Sizes are fetched only once whenever the user navigates to the list of Realms in Studio - and they wont update if newly computed sizes become available.

Potential solutions

S1: Compute on change

The sync server could compute the size of a Realm whenever it has integrated a change. This should probably be implemented with a sort of back-off period to prevent recomputing this over and over again for a Realm experiencing changes at a high frequency.

S2: In-memory cache

The sync server could implement an in-memory cache that ensures that it doesn't re-compute the size of a Realm that hasn't changed.

S3: In-file counter

Realm core could update an internal representation of the size of a realm state size (and perhaps other statistics) to make it less computationally intensive (essentially constant time) to get the size of a particular Realm.

S4: Use a Realm to store the size of Realms

When ROS core receives a statsd message from the sync server, ROS could store the newly computed size of the Realm in a synchronized Realm file.

Consequences

Either S1, S2 or S3 we would alleviate L1. And because it would be less intensive to calculate the size of a Realm, we could probably lower the delay between calculations, which would help alleviate L2.

In-fact with S1 we probably would not even have to re-compute on a schedule.

S4 would help alleviate L3 and L4 as the UI would simply be updating due to changes in the Realm. It would however introduce a Realm with a lot of changes, which would potentially build up large serverside histories in the way the Realm sync server is currently handling histories. Implementing this solution would make very little sense if L2 persists.

[astigsen]

S4: Use a Realm to store the size of Realms
When ROS core receives a statsd message from the sync server, ROS could store the newly computed size of the Realm in a synchronized Realm file.

Wouldn't it make sense to store the sizes in the admin realm together with the rest of the information about the realms? Then Studio could just listen for changes to the admin realm (as it already does) and update instantly.

[kraenhansen]

@astigsen I've been discussing this with @alebsack, @nirinchev, @kspangsege and @simonask - perhaps one of them would like to tip in here?

[kspangsege]

@astigsen , as @kraenhansen explained to me, the problem with storing these sizes in the admin Realm, is that they change a lot, and therefore cause a lot of sync activity and growth of the history for every client that already synchronize with the admin Realm. By putting it into a new Realm, only those clients (Studio in particular) that needs to see these sizes will be impacted.

[kraenhansen]

See @nirinchev's comment https://github.com/realm/realm-object-server-private/issues/1080#issuecomment-399088078 for context on ☝️

[kraenhansen]

@astigsen you moved this to the top of the backlog, should we perhaps wait a bit and see if and how anyone will be using this before improving the feature?

[roberhofer]

See as well internal discussion here: https://github.com/realm/realm-studio-private/issues/13 and here to improve the size calculation: https://github.com/realm/realm-core-private/issues/259

[kraenhansen]

@nirinchev just told me that the way we are currently querying Realm sizes (as instant values), we only get results if the metric was emitted within the previous 5 minutes in addition to this the current implementation is polling ROS which is polling Prometheus, which introduces a worst-case 15 seconds lag and unnecessary strain on ROS.

I believe we should start investigating the solution S4 proposed above ☝️which is to use a Realm for its persistence and real-time features. This leaves with the potential issue of a growing server-side history of the Realm with a high frequency changes, which we need to address somehow. I have two suggestions which I've created issues for in the Realm Sync repository:

1. https://github.com/realm/realm-sync/issues/2557: As a client I would like to communicate a history TTL, such that when I disconnect the server can forget about me (and compact the Realm) faster than the server history TTL prescribes.
2. https://github.com/realm/realm-sync/issues/2558: Compact history in-between read-only clients and a single writer.

[kraenhansen]

I added an issue to the ROS-private repository to track the implementation of the proposed solution S4: https://github.com/realm/realm-object-server-private/issues/1351

[kraenhansen]

Now that we implemented S4, I'm closing this in favor of another issue in the sync repository https://github.com/realm/realm-sync/issues/2674.