Skip to content

Conversation

@aaron-steinfeld
Copy link
Contributor

@aaron-steinfeld aaron-steinfeld commented Aug 9, 2021

Description

The motivation here is that the previous client API had major performance issues. Given the purpose of this api, high traffic writes, reducing to a simpler API makes sense for the safe of perf.

There were two major issues previously:

  1. Each call returned a single, which requires a new subscription for each invocation. The subscriptions aren't too heavy, but that's still a lot of unnecessary churn considering we don't actually read the result anywhere today.
  2. By creating a subscription, the entire observable pipeline, including its lambda captures are retained in memory until the subscription is resolved (by default, 15s). This means that we're always holding 15s of ingestion data for no good reason. This needs to be fixed on the caller side, since the client can't control the context it's called in.

This was done as a breaking change, since these only currently have one consumer (enrichment) and that consumer needs to migrate.

Testing

Updated UTs, compiled and tested full system locally.

Checklist:

  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • Any dependent changes have been merged and published in downstream modules

@codecov
Copy link

codecov bot commented Aug 9, 2021

Codecov Report

Merging #123 (3a0e3a4) into main (870168e) will decrease coverage by 0.15%.
The diff coverage is 100.00%.

Impacted file tree graph

@@             Coverage Diff              @@
##               main     #123      +/-   ##
============================================
- Coverage     59.77%   59.61%   -0.16%     
+ Complexity      295      292       -3     
============================================
  Files            39       39              
  Lines          2978     2974       -4     
  Branches        368      368              
============================================
- Hits           1780     1773       -7     
- Misses         1026     1029       +3     
  Partials        172      172              
Flag Coverage Δ
integration 59.61% <100.00%> (-0.16%) ⬇️
unit 39.73% <100.00%> (-0.19%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...entity/data/service/rxclient/EntityDataClient.java 62.50% <ø> (-12.50%) ⬇️
...data/service/rxclient/EntityDataCachingClient.java 94.91% <100.00%> (-1.86%) ⬇️
...rtrace/entity/data/service/rxclient/EntityKey.java 86.95% <100.00%> (-0.55%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 870168e...3a0e3a4. Read the comment docs.

@github-actions

This comment has been minimized.

responseObservers.forEach(updateResult::subscribe);
EntityDataCachingClient.this
.createOrUpdateEntity(entityKey, condition)
.doOnError(error -> log.error("Error upserting entity", error))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one downside of the void return is the error will never make it back - doesn't seem to me like it's worth creating a subscription per call though.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we still throw the exception back, if there was an exception creating/updating the entity?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, we've taken off the return of this method, and since it's not blocking this code is executed long after it returns. Abstractly, I prefer the alternative, which I had working - returning a single. But practically, that means that we'll be maintaining a subscription for each call/span for that "eventually" duration (which is currently 15s), just to turn around and take this same action.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This did remind me though, I need to break the habit of using .doOnError like this - since it doesn't catch it or subscribe to it. The logging still happens, but it'd be followed by a

io.reactivex.rxjava3.exceptions.UndeliverableException
The exception could not be delivered to the consumer because it has already canceled/disposed the flow or the exception has nowhere to go to begin with. Further reading: https://github.com/ReactiveX/RxJava/wiki/What's-different-in-2.0#error-handling | 

The following commit illustrates how this is supposed to be handled.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didnt quite get this. Isnt doOnError like any other subscriber similar to the blockingSubscribe below?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's an operator - internally that does mean a subscription, but the key thing here is that doOnError only acts on the error but doesn't resolve it. You can think of observables as pipelines - if an error makes it to the final subscriber and that final subscriber doesn't handle it, it's considered "undeliverable" and you get the above message. In other words, the doOnError impl was the equivalent of catching, logging and rethrowing - the subscribe version I switched to is just catching and logging.

@avinashkolluru
Copy link
Contributor

lgtm

@aaron-steinfeld aaron-steinfeld marked this pull request as ready for review August 10, 2021 12:15
@aaron-steinfeld aaron-steinfeld requested a review from a team August 10, 2021 12:15
@github-actions

This comment has been minimized.

skjindal93
skjindal93 previously approved these changes Aug 10, 2021
@github-actions

This comment has been minimized.

@aaron-steinfeld aaron-steinfeld merged commit e2ae441 into main Aug 10, 2021
@aaron-steinfeld aaron-steinfeld deleted the caching-client-memory-footprint branch August 10, 2021 13:17
@github-actions
Copy link

Unit Test Results

  30 files  ±0    30 suites  ±0   23s ⏱️ -1s
137 tests  - 4  137 ✔️  - 4  0 💤 ±0  0 ❌ ±0 

Results for commit e2ae441. ± Comparison against base commit 870168e.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants