[6X] Backdown and retry for "No route to host" in gpfdist ext table #11406
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Add GUC write_to_gpfdist_timeout write_to_gpfdist_timeout controls timeout value (in seconds) for writing data to gpfdist server. Default value is 300, valid scope is [1, 7200] Set CURLOPT_TIMEOUT as write_to_gpfdist_timeout For any error, retry with double interval time, returns SQL ERROR if write_to_gpfdist_timeout is reached Add regression test for GUC writable_external_table_timeout (cherry picked from commit ab73713)
…plum-db#11401) Reading from gpfdist external table can generate lots of HTTP traffic in very short time window. This may cause intermittent network issues resulting in "No route to host" error. Backdown and retry in such scenario. Reuse the backdown and retry logic for writable external table as gp_perform_backoff_and_check_response. Pass read/write specific part as function pointer(multi_perform_work/easy_perform_work). It's difficult to reproduce such scenario in an automatical way so we only manually tested this by setting IP_TTL option of socket to 1. This will immediately result in TTL exceeded packet that cause the "No route to host" error. (cherry picked from commit 7f1589a)
The GUC controls the time that gpdb waits before returning when it connects or writes to gpfdist. It handles retry times if getting an error from socket/network. So write_to_gpfdist_timeout is not an exact name and need to be renamed.
huiliang-liu
changed the title
huiliang-liu
changed the title
pf-qiu
approved these changes
Feb 3, 2021
5 tasks
This was referenced Mar 11, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reading from gpfdist external table can generate lots of HTTP traffic
in very short time window. This may cause intermittent network issues
resulting in "No route to host" error. Backdown and retry in such scenario.
Reuse the backdown and retry logic for writable external table as
gp_perform_backoff_and_check_response. Pass read/write specific
part as function pointer(multi_perform_work/easy_perform_work).
(cherry picked from commit 7f1589a)
This patch depends on gpfdist_retry_timeout GUC. So backport the commit of adding gpfdist_retry_timeout GUC too.
(cherry picked from commit ab73713)
Here are some reminders before you submit the pull request
make installcheck