POC: Rollup view

[RobertJoonas]

Changes

This PR is a proof of concept that aims to uncover possible issues/limitations querying clickhouse for rollup sites. It does not touch any database fields yet, but allows viewing the dashboard of multiple sites at once. To be reverted as soon as we get the necessary insights.

Tests

• This PR does not require tests

Changelog

• This PR does not make a user-facing change

Documentation

• This change does not need a documentation update

Dark mode

• This PR does not change the UI

[github-actions]

Preview environment👷🏼‍♀️🏗️
PR-5719

[aerosol]

I think this is good to evaluate, though querying for IDs alone multiple times per dashboard is probably unnecessary and we could utilize Sites Cache to retrieve them.

A bit weird that unescaped URI like rollup:3e6b15a0-64a3-495b-9b0d-7303562cea77 shows a blank page, not a dashboard nor 404 even.

[ukutaht]

querying for IDs alone multiple times per dashboard is probably unnecessary and we could utilize Sites Cache to retrieve them.

This also bothered me slightly about this implementation although it's good enough for preliminary testing.

A couple thoughts:

1. Using Site Cache on the dashboard side is crossing a virtual boundary I keep in my head between ingestion / dashboard. We've talked about that separation for a long time. With this direction we'll need two caches to maintain clear separation. Not a deal breaker for sure but something that makes me think whether there are other ways of solving the same issue.
2. This isn't the only bit of work that we have to do for every dashboard endpoint. It's bothered me for ages that the initial dashboard request will fetch user, team, site, authenticate/authorize, etc and a few ms later we do all that work again usually 6 times for the full dashboard load.
3. For a second imagine the dashboard was a liveview. All this stuff could be loaded on the initial request and subsequent requests could reuse the state and authenticated connection.
4. Before I was thinking along the lines of keeping some dedicated dashboard cache state in server memory that expires and can be accessed via signed token by the dashboard that also expires. Now I'm thinking it's extra motivation to start moving towards liveview.

Zooming out a bit, the solutions seem quite heavy and not sure if proportional to the size of the problem. Overall I agree it's not optimal as is.

[aerosol]

With this direction we'll need two caches to maintain clear separation.

Yup, good point, I only thought of that as a quick PoC.

This isn't the only bit of work that we have to do for every dashboard endpoint. It's bothered me for ages that the initial dashboard request will fetch user, team, site, authenticate/authorize, etc and a few ms later we do all that work again usually 6 times for the full dashboard load.

That sounds unnecessary indeed.

For a second imagine the dashboard was a liveview. All this stuff could be loaded on the initial request and subsequent requests could reuse the state and authenticated connection.

Asynchronously - for sure. Perhaps we should look at scroll depth data to understand whether we should load everything at the start or resume when in viewport etc, similar to how it's done currently. Keeping the common context loaded once for all is definitely desirable.

[RobertJoonas]

A bit weird that unescaped URI like rollup:3e6b15a0-64a3-495b-9b0d-7303562cea77 shows a blank page, not a dashboard nor 404 even.

Yeah, that's because the React router expects a basePath which is the site.domain, which is URL-encoded. Using colon in the URL, it will not match the basePath, which is why you get a blank page.

Fixing this would involve passing site.rollup into the frontend (via two different layers) and building a conditional in the getRouterBasePath(site) function. Since this is for testing only, didn't think it's worth that much extra code.

[aerosol]

Thanks @RobertJoonas! Sounds reasonable