Conversation
added retry feature if BigQuery error is 500 or 503 set the retry time for 5 seconds
Current coverage is 66.54% (diff: 100%)@@ master #1938 diff @@
==========================================
Files 135 135
Lines 10181 10189 +8
Methods 0 0
Messages 0 0
Branches 0 0
==========================================
+ Hits 6776 6780 +4
- Misses 3405 3409 +4
Partials 0 0
|
|
When testing, did you actually experience a 500 error, and the code worked properly? |
criccomini
left a comment
criccomini
left a comment
There was a problem hiding this comment.
Looks good to me, structurally. Few nits and style issues--left comments.
| @@ -473,18 +473,29 @@ def run_with_configuration(self, configuration): | |||
| job = jobs.get(projectId=self.project_id, jobId=job_id).execute() | |||
There was a problem hiding this comment.
Do you still need this if you are running it again at the beginning of the try?
There was a problem hiding this comment.
@criccomini you are right. I don't need it. copy/paste mistake from out internal git. Thanks for pointing out. I updated the code following your comments.
Please let me know if there are more comments. Thanks so much.
| job['status']['errorResult'], job | ||
| ) | ||
| ) | ||
| job_status_flag = True |
There was a problem hiding this comment.
Nit: rename to keep_polling_job
|
|
||
| except HttpError, err: | ||
| if err.code in [500, 503]: | ||
| logging.info('%s: Retryable error, waiting for job to complete: %s',err.code, job_id) |
There was a problem hiding this comment.
Nit: add space before err.code
| if err.code in [500, 503]: | ||
| logging.info('%s: Retryable error, waiting for job to complete: %s',err.code, job_id) | ||
| time.sleep(5) | ||
| else:raise Exception( |
fixed style errors and renamed variable
|
Thanks! |
Closes apache#1938 from krmettu/master
3 participants
Dear Airflow Maintainers,
Please accept this PR that addresses the following issues:
https://issues.apache.org/jira/browse/AIRFLOW-667
We have encountered this issue few times in our production environment. I added this fix based on google's documentation.
https://cloud.google.com/bigquery/troubleshooting-errors
"This error returns when there is a temporary server failure such as a network connection problem or a server overload. In general, wait a few seconds and try again."
Testing Done:
Tests are done connecting to BQ tables using the PR code.
Subject - Handle BigQuery 503 error
added retry feature if BigQuery error is 500 or 503
set the retry time for 5 seconds