Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Partitioned DML timeout was not always respected #203

Merged
merged 3 commits into from May 14, 2020
Merged

Conversation

olavloite
Copy link
Contributor

@olavloite olavloite commented May 13, 2020

Setting a timeout value for Partitioned DML would not be respected if the timeout value was higher than the timeout value set for the ExecuteSql RPC on the SpannerStub. Lower timeout values would be respected.

Fixes #199

@googlebot googlebot added the cla: yes label May 13, 2020
@olavloite olavloite requested a review from skuruppu May 13, 2020
.setInitialRpcTimeout(options.getPartitionedDmlTimeout())
.setMaxRpcTimeout(options.getPartitionedDmlTimeout())
Copy link
Contributor

@skuruppu skuruppu May 14, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are PDML requests not meant to be retried? I'm just curious why the initial and max timeouts are the same.

Copy link
Contributor Author

@olavloite olavloite May 14, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question.

I would say it's a a matter of probabilities and a little bit opinion:

  1. PDML is designed to accept long running update statements. That means that we should not automatically retry the statement if it takes longer than X time, as it would be impossible to find a good global value for X, as we don't know whether the statement is taking a long time because of a network problem or because the statement itself is taking a long time. Re-sending a long-running statement that is still running on the server would only hurt performance. My opinion is that for a PDML statement the chance that the statement is actually taking a long time is more probable than a network problem.
  2. The PDML documentation also clearly states that the update statement should be idempotent. This means that we should automatically retry it if the server is unavailable.

I think we should consider adding an additional method to the public API which would allow the users to specify a lower timeout for a specific statement, in addition to the current global setting. That would give the user more control over the timeout setting for statements that are known to take less time than the global setting.

Copy link
Contributor Author

@olavloite olavloite May 14, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added an extra test case to ensure that PDML is retried on UNAVAILABLE.

Copy link
Contributor

@skuruppu skuruppu left a comment

Thanks for the quick debugging and fix @olavloite.

olavloite added 3 commits May 14, 2020
Setting a timeout value for Partitioned DML would not be respected if the timeout
value was higher than the timeout value set for the ExecuteSql RPC on the SpannerStub.
Lower timeout values would be respected.

Fixes #199
@olavloite olavloite merged commit 13cb37e into master May 14, 2020
13 checks passed
@olavloite olavloite deleted the pdml-timeout branch May 14, 2020
gcf-merge-on-green bot pushed a commit that referenced this issue May 19, 2020
🤖 I have created a release \*beep\* \*boop\* 
---
## [1.55.0](https://www.github.com/googleapis/java-spanner/compare/v1.54.0...v1.55.0) (2020-05-19)


### Features

* mark when a Spanner client is closed ([#198](https://www.github.com/googleapis/java-spanner/issues/198)) ([50cb174](https://www.github.com/googleapis/java-spanner/commit/50cb1744e7ede611758d3ff63b3df77a1d3682eb))


### Bug Fixes

* make it possible to override backups methods ([#195](https://www.github.com/googleapis/java-spanner/issues/195)) ([2d19c25](https://www.github.com/googleapis/java-spanner/commit/2d19c25ba32847d116194565e67e1b1276fcb9f8))
* Partitioned DML timeout was not always respected ([#203](https://www.github.com/googleapis/java-spanner/issues/203)) ([13cb37e](https://www.github.com/googleapis/java-spanner/commit/13cb37e55ddfd1ff4ec22b1dcdc20c4832eee444)), closes [#199](https://www.github.com/googleapis/java-spanner/issues/199)
* partitionedDml stub was not closed ([#213](https://www.github.com/googleapis/java-spanner/issues/213)) ([a2d9a33](https://www.github.com/googleapis/java-spanner/commit/a2d9a33fa31f7467fc2bfbef5a29c4b3f5aea7c8))
* reuse clientId for invalidated databases ([#206](https://www.github.com/googleapis/java-spanner/issues/206)) ([7b4490d](https://www.github.com/googleapis/java-spanner/commit/7b4490dfb61fbc81b5bd6be6c9a663b36b5ce402))
* use nanos to prevent truncation errors ([#204](https://www.github.com/googleapis/java-spanner/issues/204)) ([a608460](https://www.github.com/googleapis/java-spanner/commit/a60846043dc0ca47e1970d8ab99380b6d725c7a9)), closes [#200](https://www.github.com/googleapis/java-spanner/issues/200)


### Dependencies

* update dependency com.google.cloud:google-cloud-shared-dependencies to v0.3.1 ([#190](https://www.github.com/googleapis/java-spanner/issues/190)) ([ad41a0d](https://www.github.com/googleapis/java-spanner/commit/ad41a0d4b0cc6a2c0ae0611c767652f64cfb2fb7))
---


This PR was generated with [Release Please](https://github.com/googleapis/release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants