Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Add a throwException behavior when the StreamWriter inflight queue is full #1642

Merged
merged 9 commits into from May 6, 2022
Merged

Conversation

yirutang
Copy link
Contributor

@yirutang yirutang commented May 6, 2022

Add a throwException behavior when the StreamWriter inflight queue is full.

Fixes #1539 ☕️

If you write sample code, please follow the samples format.

@yirutang yirutang requested review from a team and GaoleMeng May 6, 2022 00:05
@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: bigquerystorage Issues related to the googleapis/java-bigquerystorage API. labels May 6, 2022
@yirutang yirutang changed the title fix:Add a throwException behavior when the StreamWriter inflight queue is full fix: Add a throwException behavior when the StreamWriter inflight queue is full May 6, 2022
@yirutang yirutang added the owlbot:run Add this label to trigger the Owlbot post processor. label May 6, 2022
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label May 6, 2022
@yirutang yirutang added the owlbot:run Add this label to trigger the Owlbot post processor. label May 6, 2022
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label May 6, 2022
@yirutang yirutang requested a review from a team as a code owner May 6, 2022 03:02
@yirutang yirutang merged commit 4dcf0d5 into googleapis:main May 6, 2022
Status.fromCode(Code.RESOURCE_EXHAUSTED)
.withDescription(
"Exceeds client side inflight buffer, consider add more buffer or open more connections."));
} else if (this.limitExceededBehavior == FlowController.LimitExceededBehavior.Ignore) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't be the ignore option just a break; at this point?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

L735, by default it is Block.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I meant was, that the actual meaning of LimitExceededBehavior.Ignore is, that this inflight quota counting is simply ignored. So the StreamWriter just continues sending. But this option is probably not that relevant for productive setups.

Suggested change
} else if (this.limitExceededBehavior == FlowController.LimitExceededBehavior.Ignore) {
} else if (this.limitExceededBehavior == FlowController.LimitExceededBehavior.Ignore) {
break;
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tend to think we shouldn't allow it...

gcf-merge-on-green bot pushed a commit that referenced this pull request May 24, 2022
🤖 I have created a release *beep* *boop*
---


## [2.14.0](v2.13.0...v2.14.0) (2022-05-19)


### Features

* add build scripts for native image testing in Java 17 ([#1440](#1440)) ([#1655](#1655)) ([ac2dfaf](ac2dfaf))


### Bug Fixes

* Add a throwException behavior when the StreamWriter inflight queue is full ([#1642](#1642)) ([4dcf0d5](4dcf0d5))
* add extra JsonWriterTest to show that the LimitBehavior addition is not breaking ([#1643](#1643)) ([320f5fc](320f5fc))
* ints/longs are numerics ([#1596](#1596)) ([d046c8d](d046c8d)), closes [#1516](#1516)


### Dependencies

* update arrow.version to v8 ([#1645](#1645)) ([06e3c34](06e3c34))
* update dependency com.google.cloud:google-cloud-shared-dependencies to v2.12.0 ([#1654](#1654)) ([ec4f60b](ec4f60b))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquerystorage Issues related to the googleapis/java-bigquerystorage API. size: m Pull request size is medium.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BigQuery Storage Write API: LimitExceededBehavior of FlowControlSettings is ignored in JsonStreamWriter
3 participants