Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bigtable.snippets.writes.writes_test: test_writes failed #3070

Closed
flaky-bot bot opened this issue Mar 10, 2020 · 3 comments · Fixed by #3494
Closed

bigtable.snippets.writes.writes_test: test_writes failed #3070

flaky-bot bot opened this issue Mar 10, 2020 · 3 comments · Fixed by #3494
Assignees
Labels
api: bigtable Issues related to the Bigtable API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@flaky-bot
Copy link

flaky-bot bot commented Mar 10, 2020

This test failed!

To configure my behavior, see the Build Cop Bot documentation.

If I'm commenting on this issue too often, add the buildcop: quiet label and
I will stop commenting.


buildID: fadb04c
buildURL: Build Status, Sponge
status: failed

@flaky-bot flaky-bot bot added buildcop: issue priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Mar 10, 2020
@flaky-bot
Copy link
Author

flaky-bot bot commented Mar 10, 2020

Test passed in build 63e99d6 (Build Status, Sponge)! Closing this issue.

@flaky-bot flaky-bot bot closed this as completed Mar 10, 2020
@gguuss gguuss assigned billyjacobson and unassigned gguuss Mar 11, 2020
@flaky-bot flaky-bot bot reopened this Apr 9, 2020
@flaky-bot
Copy link
Author

flaky-bot bot commented Apr 9, 2020

Looks like this issue is flaky. 😟

I'm going to leave this open and stop commenting.

A human should fix and close this.


commit: ae464b1
buildURL: Build Status, Sponge
status: failed

@tmatsuo tmatsuo added the api: bigtable Issues related to the Bigtable API. label Apr 9, 2020
@kolea2 kolea2 added priority: p2 Moderately-important priority. Fix may not be included in next release. and removed priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. labels Apr 13, 2020
@tmatsuo
Copy link
Contributor

tmatsuo commented Apr 23, 2020

From the log
row.commit() ended up with google.api_core.exceptions.DeadlineExceeded: 504 Deadline Exceeded

@billyjacobson Does it make sense to update the sample to handle this error and retry few times? If not, maybe we should wrap the each sample invocation with retries.

Because now test_writes has 4 consecutive sample invocation.

When the server is little bit flaky, the success rate of the test is significantly lowered. Let's say, the rpc success rate is 0.9. Then the success rate of test_writes will be 0.9^4 = 0.6561.

What if we retry this test 3 times? The success rate will be:
1 - 0.3439^3 = ~0.959

However, if we wrap these sample invocations with 3 times retry, the success rate of a single sample will be:
1-0.1^3 = 0.999

The success rate of the whole test will be:
0.999^4 = ~0.996

It's significantly higher.

@tmatsuo tmatsuo assigned tmatsuo and unassigned billyjacobson Apr 23, 2020
tmatsuo pushed a commit to tmatsuo/python-docs-samples that referenced this issue Apr 24, 2020
fix GoogleCloudPlatform#3070

Also added `BIGTABLE_INSTANCE` to testing/test-env.tmpl.sh
tmatsuo pushed a commit that referenced this issue Apr 24, 2020
fix #3070

Also added `BIGTABLE_INSTANCE` to testing/test-env.tmpl.sh
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigtable Issues related to the Bigtable API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants