Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: extend channel priming logic to also send fake requests #398

Merged
merged 8 commits into from Sep 1, 2020

Conversation

igorbernstein2
Copy link
Contributor

@igorbernstein2 igorbernstein2 commented Aug 28, 2020

This PR upgrades the existing channel refresher to send a fake read row request to ensure that server side caches are warmed up. The new refresher will create a temporary stub around the new channel, which will reuse all of the existing request decorations (resource headers and user agent)

@igorbernstein2 igorbernstein2 added the do not merge label Aug 28, 2020
@google-cla google-cla bot added the cla: yes label Aug 28, 2020
@igorbernstein2 igorbernstein2 changed the title feat: extend channel priming logic to also send fake requests [WIP] feat: extend channel priming logic to also send fake requests Aug 28, 2020
@igorbernstein2 igorbernstein2 marked this pull request as ready for review Aug 28, 2020
@igorbernstein2 igorbernstein2 requested a review from kolea2 Aug 28, 2020
@igorbernstein2 igorbernstein2 removed the do not merge label Aug 28, 2020
@product-auto-label product-auto-label bot added the api: bigtable label Aug 29, 2020
@codecov
Copy link

@codecov codecov bot commented Aug 31, 2020

Codecov Report

Merging #398 into master will increase coverage by 0.11%.
The diff coverage is 84.09%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #398      +/-   ##
============================================
+ Coverage     80.24%   80.36%   +0.11%     
- Complexity     1102     1113      +11     
============================================
  Files           105      105              
  Lines          6815     6897      +82     
  Branches        364      369       +5     
============================================
+ Hits           5469     5543      +74     
- Misses         1148     1155       +7     
- Partials        198      199       +1     
Impacted Files Coverage Δ Complexity Δ
...e/cloud/bigtable/data/v2/BigtableDataSettings.java 63.63% <0.00%> (-4.11%) 7.00 <0.00> (ø)
...d/bigtable/data/v2/stub/BigtableChannelPrimer.java 78.87% <78.87%> (ø) 8.00 <8.00> (?)
...ble/data/v2/stub/EnhancedBigtableStubSettings.java 96.80% <85.71%> (-0.44%) 22.00 <1.00> (+1.00) ⬇️
...ud/bigtable/data/v2/stub/EnhancedBigtableStub.java 96.05% <98.00%> (+0.31%) 24.00 <4.00> (+3.00)
...d/bigtable/data/v2/stub/metrics/MetricsTracer.java 81.01% <0.00%> (+2.53%) 15.00% <0.00%> (+1.00%)
...ata/v2/stub/readrows/FilterMarkerRowsCallable.java 96.77% <0.00%> (+12.90%) 2.00% <0.00%> (ø%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a56a0f8...5e1a94d. Read the comment docs.

throws IOException {
EnhancedBigtableStubSettings.Builder builder = settings.toBuilder();

if (settings.isRefreshingChannel()) {
Copy link
Collaborator

@kolea2 kolea2 Aug 31, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add some tests around this to ensure the proper values are set in the settings?

Copy link
Contributor Author

@igorbernstein2 igorbernstein2 Sep 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a test for testChannelPrimerConfigured, which should cover the behavior of it

@@ -760,17 +787,10 @@ public EnhancedBigtableStubSettings build() {
Preconditions.checkState(projectId != null, "Project id must be set");
Preconditions.checkState(instanceId != null, "Instance id must be set");

// Set ChannelPrimer on TransportChannelProvider so channels will gracefully refresh
// connections to Cloud Bigtable service
if (isRefreshingChannel) {
Preconditions.checkArgument(
getTransportChannelProvider() instanceof InstantiatingGrpcChannelProvider,
Copy link
Collaborator

@kolea2 kolea2 Aug 31, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this check be moved to where we are setting the transport provider, https://github.com/googleapis/java-bigtable/pull/398/files#diff-dad9668e639c962562540f7f70a24138R144?

Copy link
Contributor Author

@igorbernstein2 igorbernstein2 Sep 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I want the validation logic to be as close to usage as possible.

Copy link
Contributor

@tonytanger tonytanger left a comment

Channel priming logic looks good!

I'm not the most familiar with the stub and stub settings. A second opinion on that would be useful.

}
}

private void primeChannelUnsafe(ManagedChannel managedChannel) throws IOException {
Copy link
Contributor

@tonytanger tonytanger Sep 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understand why this is called unsafe?

Copy link
Contributor Author

@igorbernstein2 igorbernstein2 Sep 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because it could throw an exception, does the priming api expect to handle the exceptions?

builder.setTransportChannelProvider(
transportProvider
.toBuilder()
.setChannelPrimer(
Copy link
Contributor

@tonytanger tonytanger Sep 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to have a comment here explaining the consequence of "setChannelPrimer" that the channel is created once, but periodically creates a new channel, runs BigtableChannelPrimer on the newly created channel and replaces the existing one with the new one.

Copy link
Contributor Author

@igorbernstein2 igorbernstein2 Sep 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont quite follow, can you rephrase?

kolea2
kolea2 approved these changes Sep 1, 2020
@kolea2 kolea2 merged commit 6f1ead2 into googleapis:master Sep 1, 2020
16 checks passed
@igorbernstein2 igorbernstein2 deleted the channel-priming branch Sep 1, 2020
gcf-merge-on-green bot pushed a commit that referenced this issue Sep 2, 2020
🤖 I have created a release \*beep\* \*boop\* 
---
## [1.15.0](https://www.github.com/googleapis/java-bigtable/compare/v1.14.0...v1.15.0) (2020-09-01)


### Features

* extend channel priming logic to also send fake requests ([#398](https://www.github.com/googleapis/java-bigtable/issues/398)) ([6f1ead2](https://www.github.com/googleapis/java-bigtable/commit/6f1ead2097150a87cb9712bcf35c6eaa9d57440c))
* **deps:** adopt flatten plugin and google-cloud-shared-dependencies ([#350](https://www.github.com/googleapis/java-bigtable/issues/350)) ([2298596](https://www.github.com/googleapis/java-bigtable/commit/2298596dab8a1ef87c0f48d3abe8bc3955417eb1))


### Bug Fixes

* temporarily disable reporting to unblock releases ([#395](https://www.github.com/googleapis/java-bigtable/issues/395)) ([a56a0f8](https://www.github.com/googleapis/java-bigtable/commit/a56a0f8c9caf675b68d02587b042e1feeb261ccb))


### Dependencies

* update dependency com.google.cloud:google-cloud-shared-dependencies to v0.8.6 ([#377](https://www.github.com/googleapis/java-bigtable/issues/377)) ([bdae336](https://www.github.com/googleapis/java-bigtable/commit/bdae336074d80815dcaaf8c71befafc0ed66c079))
* update dependency com.google.cloud:google-cloud-shared-dependencies to v0.9.0 ([#402](https://www.github.com/googleapis/java-bigtable/issues/402)) ([08f7d84](https://www.github.com/googleapis/java-bigtable/commit/08f7d84333c6a74bf03e0a57707b878a29400dd4))
---


This PR was generated with [Release Please](https://github.com/googleapis/release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigtable cla: yes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants