external_storage: fix GCS download error, support GCS endpoints, and refactoring

[kennytm]

What problem does this PR solve?

Issue Number: close #7625

Problem Summary:

It was found that the GCS external_storage can only download 8 KB of data. This PR fixes the issue.

Additionally, tame_gcs hard-coded the request URLs, making the custom endpoint ineffective (EmbarkStudios/tame-gcs#21). This PR also workarounds the issue through URL rewriting.

What is changed and how it works?

What's Changed:

The problem is caused by using block_on_external_io recursively. This seems to break Tokio's executor and caused the download stream to end after a single call of poll_read (where the buffer size is 8 KB).

This PR fixes the problem by removing all nested block_on_external_io and replace them by async fn.

Related changes

• Need to cherry-pick to the release branch
  • need to cherry-pick to release-4.0.

Check List

Tests

• Manual test (add detailed scripts or steps below)
  • Restore through fake-gcs-server works.

Side effects

Release note

Restoring a backup archive from GCS now works.

[kennytm]

/release

[sre-bot]

download tikv at http://fileserver.pingcap.net/download/builds/pingcap/tikv/pr/2406eb8875ecda10badea3cebc3cbfd6b1f0b700/centos7/tikv-server.tar.gz

[kennytm]

cc @shuijing198799

[yiwu-arbug]

LGTM. Thanks for the fix.

[overvenus]

LGTM

[kennytm]

/merge

[sre-bot]

Your auto merge job has been accepted, waiting for:

• 7729

[sre-bot]

/run-all-tests

[sre-bot]

@kennytm merge failed.

[kennytm]

https://internal.pingcap.net/idc-jenkins/blue/organizations/jenkins/tikv_ghpr_integration_common_test/detail/tikv_ghpr_integration_common_test/13583/pipeline

udf_logic_xor failed

[2020-05-06T07:04:10.197Z] 2020/05/06 15:04:10.111 main.go:713: [fatal] run test [udf_logic_xor] err: sql:SELECT a, b, a xor b FROM (SELECT `pk` AS a, `col_decimal_6_3_signed` AS b FROM `table50_int_autoinc`) tmp WHERE (a xor b) = 1 ORDER BY LOWER(a), LOWER(b);: failed to run query 
[2020-05-06T07:04:10.197Z] "SELECT a, b, a xor b FROM (SELECT `pk` AS a, `col_decimal_6_3_signed` AS b FROM `table50_int_autoinc`) tmp WHERE (a xor b) = 1 ORDER BY LOWER(a), LOWER(b);" 
[2020-05-06T07:04:10.197Z]  around line 551, 
[2020-05-06T07:04:10.197Z] we need(297):
[2020-05-06T07:04:10.197Z] SELECT a, b, a xor b FROM (SELECT `pk` AS a, `col_decimal_6_3_signed` AS b FROM `table50_int_autoinc`) tmp WHERE (a xor b) = 1 ORDER BY LOWER(a), LOWER(b);
[2020-05-06T07:04:10.197Z] a	b	a xor b
[2020-05-06T07:04:10.197Z] 1	0.000	1
[2020-05-06T07:04:10.197Z] 10	0.000	1
[2020-05-06T07:04:10.198Z] 14	0.112	1
[2020-05-06T07:04:10.198Z] 16	0.000	1
[2020-05-06T07:04:10.198Z] 17	0.000	1
[2020-05-06T07:04:10.198Z] 24	0.000	1
[2020-05-06T07:04:10.198Z] 26	0.000	1
[2020-05-06T07:04:10.198Z] 29	0.000	1
[2020-05-06T07:04:10.198Z] 35	0.000	1
[2020-05-06T07:04:10.198Z] 38	0.000	1
[2020-05-06T07:04:10.198Z] 4	0.000	1
[2020-05-06T07:04:10.198Z] 41	0.112	1
[2020-05-06T07:04:10.198Z] but got(297):
[2020-05-06T07:04:10.198Z] SELECT a, b, a xor b FROM (SELECT `pk` AS a, `col_decimal_6_3_signed` AS b FROM `table50_int_autoinc`) tmp WHERE (a xor b) = 1 ORDER BY LOWER(a), LOWER(b);
[2020-05-06T07:04:10.198Z] a	b	a xor b
[2020-05-06T07:04:10.198Z] 1	0.000	1
[2020-05-06T07:04:10.198Z] 10	0.000	1
[2020-05-06T07:04:10.198Z] 16	0.000	1
[2020-05-06T07:04:10.198Z] 17	0.000	1
[2020-05-06T07:04:10.198Z] 24	0.000	1
[2020-05-06T07:04:10.198Z] 26	0.000	1
[2020-05-06T07:04:10.198Z] 29	0.000	1
[2020-05-06T07:04:10.198Z] 35	0.000	1
[2020-05-06T07:04:10.198Z] 38	0.000	1
[2020-05-06T07:04:10.198Z] 4	0.000	1
[2020-05-06T07:04:10.198Z] 45	0.000	1
[2020-05-06T07:04:10.198Z] 8	0.000	1

[kennytm]

/merge

[sre-bot]

Your auto merge job has been accepted, waiting for:

• 7729

[sre-bot]

/run-all-tests

[sre-bot]

cherry pick to release-4.0 in PR #7739

[overvenus]

/run-cherry-picker

[sre-bot]

cherry pick to release-3.1 in PR #7962