Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace posh with ratom #784

Closed

Conversation

pithyless
Copy link
Contributor

This is a continuation of the work done by @lambduhh on #665. Kudos to @lambduhh for the idea!

The supporting code has changed significantly, though, so I thought it would be easier to open a separate PR.

How to reproduce

The TL;DR to run this branch:

  1. Uncomment athens.tracing in shadow-cljs.edn: (-> :renderer :devtools :preloads)
  2. Choose your poison in athens.posh/version (:posh is original flavor, :dbrx is brand new fizzy flavor)
  3. Rebuild (lein dev) and run Athens (yarn run electron .), open the DevTools Console, and click around.

Two reports will show up. Epoch Stats report percentiles for profiled functions between epochs (as understood by re-frame-10x). Total Stats is a running total for the entire session. Epochs can help you identify the performance characteristics of a specific action, while Totals can help approximate the general performance of the app during regular usage.

report1

What are the results?

For my tests, I'm using a basic Welcome db + a large tagged markdown file.

:posh - original flavor

Open/Close sections of the Welcome page:

posh1

Open/Close sections of the Markdown report:

posh2

⚠️ 408 ms just to process the transact!. So, with additional parsing and rendering we have failed the Doherty Threshold!

:dbrx - brand new fizzy flavor

Open/Closing sections of the Welcome page:

dbrx1

Open/Close sections of the Markdown report:

dbrx2

💯 Max 40 ms response times.

Next Steps

This should also benefit #570

Athens is still noticeably slow when working with large files, but this is the first of several problems that needed to be fixed; the next biggest bottleneck may be in parsing and/or rendering.

In the meantime, we need to test and make sure this approach doesn't break on some edge cases. More 👀 and user reports would be most welcome!

PR Code Review

The last two commits are intended for profiling and may be optionally merged, depending on preference. I split them out to make it easier to cherry-pick this branch.

  1. 75b1d57
  2. d0f18d3

Original code alows executing a pull for `[:block/uid nil]`
which is not a valid lookup ref (and throws an exception
in DataScript). It was not a problem up to now, because
Posh does allow this unspecified behavior (and just returns nil).

Refactored the code to separate page-title and navigation concerns.
Solution was originally proposed by @lambduhh in
athensresearch#665.
See PR discussion for details why posh is considered
slow and a mismatch for Athens UI performance right now.

This version of the solution introduces a new namespace
`athens.posh` as a form of indirection. This way the rest
of the codebase does not need any further changes (aside from
the namespace require) and also allows easy switching between
the two different databases to compare performance and
correctness (via `athens.posh/version`).

The assumption is that later this hybrid duality can be elided
and `athens.posh` refactored into a more robust data fetching layer.
Added athens.tracing which depends on tufte for profiling CLJS
functions. It's not a great integration, as currently Athens is on an
old version of re-frame-10x that does not have the appropriate
monkeypatchin hooks.

The assumption is that a developer will enable it by uncommenting the
`athens.tracing` in shadow-cljs preloads and checking the results in
console.log.
@tangjeff0
Copy link
Collaborator

Results from testing open/close of a bullet on my personal DB.

posh
image

dbrx

image


Results from testing open/close of a bullet the ego db. Can test by adding athens.tracing to the :app :preloads vector, and also watching shadow-cljs watch main renderer app.

Posh

image

dbrx

image


Time seems to be about the same or in some cases higher?

@pithyless
Copy link
Contributor Author

@tangjeff0 any chance you did not restart the app after changing the athens.posh/version? That's the only thing I can think of right now to explain your observations. Here is how it looks on my machine:

https://www.youtube.com/watch?v=8lAth0JH3TQ

Also, as I'm posting this now, I realize I made a mistake in how I interpret the results for the new dbrx pull* queries. Because it's all wrapped in an ratom, the profiler does not actually measure how long an operation took (laziness strikes again!). Sorry about the confusion and misreporting.

But this should not change the observations about the performance of transact!, which is a blocking operation. I think this PR is still valid to reduce the time for transact!, but it looks like further work is necessary to refactor the components to make them reactive and only do the minimum amount of work when fetching/transforming data for rendering (as @lambduhh and @neotyk have already been pointing out). That probably should be raised as an independent issue and PR, to avoid growing this one further.

From my point of view, there are two open questions:

  1. Make sure this PR is actually improving transact! times. @tangjeff0 if you could double-check your observations, that'd be great. But it would also be beneficial if someone else could have a look over this PR and point out if I've made any goofball assumptions (like the one about profiling laziness).
  2. If the PR does look OK, would we want to merge it or wait until we have a more comprehensive solution with the reactive read/renders resolved as well?

@tangjeff0
Copy link
Collaborator

tangjeff0 commented Mar 11, 2021

My results testing on my actual db. Demo'd here

posh
image

dbrx
image

@tangjeff0 tangjeff0 marked this pull request as draft April 16, 2021 07:27
@tangjeff0 tangjeff0 closed this Apr 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants