Skip to content

Conversation

@aaron-steinfeld
Copy link
Contributor

@aaron-steinfeld aaron-steinfeld commented Oct 15, 2021

Description

Added support for a new upsertAll api. This can be used for upserting multiple configs in fewer calls. Eventually, we'll optimize its internals to also reduce the roundtrips to the persistence store, but this change encapsulates that tech debt.

Testing

Added and updated existing UTs

Checklist:

  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • Any dependent changes have been merged and published in downstream modules

@codecov
Copy link

codecov bot commented Oct 15, 2021

Codecov Report

Merging #72 (b57d6bf) into main (1fc8b19) will decrease coverage by 1.17%.
The diff coverage is 80.23%.

Impacted file tree graph

@@             Coverage Diff              @@
##               main      #72      +/-   ##
============================================
- Coverage     85.37%   84.20%   -1.18%     
- Complexity      276      286      +10     
============================================
  Files            36       36              
  Lines          1361     1412      +51     
  Branches         46       45       -1     
============================================
+ Hits           1162     1189      +27     
- Misses          170      194      +24     
  Partials         29       29              
Flag Coverage Δ
integration 84.20% <80.23%> (-1.18%) ⬇️
unit 83.33% <73.25%> (-1.19%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
.../org/hypertrace/config/service/ConfigResource.java 75.00% <ø> (-5.00%) ⬇️
.../hypertrace/config/service/ConfigServiceUtils.java 96.77% <ø> (-0.11%) ⬇️
...pertrace/config/service/ConfigServiceGrpcImpl.java 70.64% <64.28%> (-13.81%) ⬇️
...race/config/objectstore/IdentifiedObjectStore.java 84.48% <69.04%> (-3.57%) ⬇️
...pertrace/config/service/ConfigResourceContext.java 88.88% <88.88%> (ø)
...rtrace/config/service/store/ConfigDocumentKey.java 88.88% <100.00%> (ø)
...race/config/service/store/DocumentConfigStore.java 100.00% <100.00%> (+1.16%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1fc8b19...b57d6bf. Read the comment docs.

@github-actions

This comment has been minimized.

* @param userId
* @return the upserted configs
*/
List<UpsertedConfig> writeAllConfigs(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the error handling behavior ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, it's identical to calling N write operations sequentially. If a later one fails, it aborts at that point leaving the original (this is the same behavior in the current abstraction one layer up in ObjectStore). Ideally, with a doc store API change we can make this transactional with all or nothing semantics (this came up with Entity Service in conversations with @saxenakshitiz too, but I believe as of today doc store itself doesn't support this - another reason I paused this change where I did).

public List<UpsertedConfig> writeAllConfigs(
Map<ConfigResourceContext, Value> resourceContextValueMap, String userId) throws IOException {
// TODO keep pushing this down into doc store
List<UpsertedConfig> list = new ArrayList<>(resourceContextValueMap.size());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so would docstore be able to leverage bulk upsert APIs ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the idea - just split the PR. The current PR handles all the API changes, which is a fairly significant refactor, but doesn't really change the implementation in any meaningful way. A future PR needs only swap out this implementation to use the doc store's bulk upsert.

Copy link
Collaborator

@GurtejSohi GurtejSohi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@aaron-steinfeld aaron-steinfeld merged commit 32a08ce into main Oct 19, 2021
@aaron-steinfeld aaron-steinfeld deleted the upsert-all branch October 19, 2021 17:04
@github-actions
Copy link

Unit Test Results

21 files  ±0  21 suites  ±0   36s ⏱️ +3s
77 tests ±0  77 ✔️ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit 32a08ce. ± Comparison against base commit 1fc8b19.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants