feat: create Job retry for rate limit exceeded with status code 200 #1744

franklinWhaite · 2021-12-07T19:31:07Z

With these changes Create Job will retry for rate limit exceeded even if the status code is 200

…us code 200

stephaniewang526 · 2021-12-07T20:30:44Z

google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/BigQueryImpl.java

    BigQueryException createException;
    // NOTE(pongad): This double-try structure is admittedly odd.
    // translateAndThrow itself throws, and pretends to return an exception only
    // so users can pretend to throw.
    // This makes it difficult to translate without throwing.
    // Fixing this entails some work on BaseServiceException.translate.
    // Since that affects a bunch of APIs, we should fix this as a separate change.
+    final JobInfo jobInfoCopy = jobInfo;


Please add comment to explain the motivation for doing this.

Actually not needed, removed this variable

stephaniewang526 · 2021-12-07T20:34:04Z

google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/BigQueryImpl.java

                new Callable<com.google.api.services.bigquery.model.Job>() {
                  @Override
                  public com.google.api.services.bigquery.model.Job call() {
-                    return bigQueryRpc.create(jobPb, optionsMap);
+                    if(idRandom){
+                      // generate new random job


Maybe we can expand this comment to say "re-generate a new random job with the same jobInfo when jobId is not provided by the user"

expanded comment as recommended

stephaniewang526 · 2021-12-07T20:37:44Z

google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/BigQueryImpl.java

    BigQueryException createException;
    // NOTE(pongad): This double-try structure is admittedly odd.
    // translateAndThrow itself throws, and pretends to return an exception only
    // so users can pretend to throw.
    // This makes it difficult to translate without throwing.
    // Fixing this entails some work on BaseServiceException.translate.
    // Since that affects a bunch of APIs, we should fix this as a separate change.
+    final JobInfo jobInfoCopy = jobInfo;
+    final JobId[] finalJobId = new JobId[1];


Does this have to be an array? Can we just use jobId? since we are only storing a single, final jobId?

Tried several approaches and using an array is the only viable option i found. My understanding is that the variable needs to be a final in order to be accessed from within the inner class (BigQueryRetryHelper). If it's a final it can not be modified, therefore the new jobId values cannot be assigned to it

I would really appreciate any suggestions on how to avoid using an array here.

stephaniewang526 · 2021-12-07T20:40:39Z

google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/BigQueryImpl.java

-                    return bigQueryRpc.create(jobPb, optionsMap);
+                    if(idRandom){
+                      // generate new random job
+                      JobInfo newJobInfo = jobInfoCopy.toBuilder().setJobId(idProvider.get()).build();


Maybe rename this variable to recreatedJobInfo since everything but JobId is the same for this Job.

changed variable name as recommended

stephaniewang526 · 2021-12-07T20:42:07Z

google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/BigQueryRetryAlgorithm.java

+      e.g. status code 200 with a rate limit exceeded for job create
+      in these cases there is now previousThrowable so we need to check previousResponse
+       */
+      errorDesc = previousResponse.toString();


Please add a mock test in BigQueryImplTest to verify the correctness of the behavior; i.e. retrying with status code 200 on rateLimitExceeded error message.

added mock test to BigQueryImplTest test is called testCreateJobFailureShouldRetry

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

…1879) #1744 introduced retrying 200 responses, but the regex that we are using is too broad: [".*exceed.*rate.limit."] and in some cases it may catch a valid response. The issue is that the current logic is scanning the whole response. In this PR the retry logic is modified to specifically check the error message from the response. For this purpose parsing logic was developed to extract error messages from responses. This logic is specifically designed for the [jobs.insert method response](https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs/insert#response-body) (only case observed so far where a response with status code 200 might also return an error message).

fix(java): handle empty modules Fixes googleapis/synthtool#1743 Source-Link: googleapis/synthtool@482d649 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:bd5071596a47614d1fe15eb766c4255bae330f823b606e1196a3b0c8d2e96fd1

fix(java): handle empty modules Fixes googleapis/synthtool#1743 Source-Link: googleapis/synthtool@482d649 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:bd5071596a47614d1fe15eb766c4255bae330f823b606e1196a3b0c8d2e96fd1 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

franklinWhaite and others added 4 commits December 1, 2021 14:07

retrying job create with new job id each time

16c606c

Merge branch 'googleapis:main' into main

f4d9cb6

retrying for rate limit exceeded on create Job regardless of the stat…

af20e7e

…us code 200

removing flattened pom

124116b

franklinWhaite requested review from a team and loferris December 7, 2021 19:31

product-auto-label bot added the api: bigquery Issues related to the googleapis/java-bigquery API. label Dec 7, 2021

stephaniewang526 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Dec 7, 2021

yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Dec 7, 2021

stephaniewang526 reviewed Dec 7, 2021

View reviewed changes

franklinWhaite added 2 commits December 8, 2021 15:03

refactoring, removing unnecesary variables, adding comments

76cd092

linting code

84ae60f

stephaniewang526 added the owlbot:run Add this label to trigger the Owlbot post processor. label Dec 14, 2021

gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Dec 14, 2021

stephaniewang526 added kokoro:force-run Add this label to force Kokoro to re-run the tests. owlbot:run Add this label to trigger the Owlbot post processor. labels Dec 14, 2021

gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Dec 14, 2021

yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Dec 14, 2021

🦉 Updates from OwlBot

2482f78

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

franklinWhaite requested a review from a team as a code owner December 14, 2021 19:04

gcf-owl-bot bot and others added 3 commits December 14, 2021 19:05

🦉 Updates from OwlBot

514b57a

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Merge branch 'main' of https://github.com/franklinWhaite/java-bigquery

b30f7c2

refactoring, removing comments

ca2931c

stephaniewang526 changed the title ~~Create Job retry for rate limit exceeded with status code 200~~ feat: create Job retry for rate limit exceeded with status code 200 Dec 14, 2021

stephaniewang526 added kokoro:force-run Add this label to force Kokoro to re-run the tests. owlbot:run Add this label to trigger the Owlbot post processor. labels Dec 14, 2021

gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Dec 14, 2021

yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Dec 14, 2021

stephaniewang526 approved these changes Dec 14, 2021

View reviewed changes

stephaniewang526 merged commit 97a61dc into googleapis:main Dec 14, 2021

release-please bot mentioned this pull request Jan 4, 2022

chore(main): release java-bigquery 3.0.0 #1768

Closed

franklinWhaite mentioned this pull request Mar 2, 2022

Fix: adjusting retry logic to avoid retrying successful job creation #1879

Merged

release-please bot mentioned this pull request Jan 23, 2023

chore(main): release 2.21.0 #2478

Merged

This was referenced Jan 31, 2023

[BQ] January 30, 2023 kitta65/bq-extension-vscode#131

Closed

[BQ] January 30, 2023 kitta65/prettier-plugin-bq#136

Closed

[BQ] January 30, 2023 kitta65/bq2cst#144

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: create Job retry for rate limit exceeded with status code 200 #1744

feat: create Job retry for rate limit exceeded with status code 200 #1744

franklinWhaite commented Dec 7, 2021

stephaniewang526 Dec 7, 2021

franklinWhaite Dec 8, 2021

stephaniewang526 Dec 7, 2021

franklinWhaite Dec 8, 2021

stephaniewang526 Dec 7, 2021

franklinWhaite Dec 8, 2021

stephaniewang526 Dec 7, 2021

franklinWhaite Dec 8, 2021

stephaniewang526 Dec 7, 2021

franklinWhaite Dec 8, 2021

feat: create Job retry for rate limit exceeded with status code 200 #1744

feat: create Job retry for rate limit exceeded with status code 200 #1744

Conversation

franklinWhaite commented Dec 7, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment