Integration test for Google GCS hosted table

[zhuansunxt]

• Integration test.
• Library version upgrade. The previous version has bug when reading the file content.

The python integration test does not work yet. Still looking.

[zsxwing]

• Could you also add a quick example in https://github.com/delta-io/delta-sharing/blob/main/server/src/universal/conf/delta-sharing-server.yaml.template#L14 ?
• For the test, I think we also need to write code to generate the file that GOOGLE_APPLICATION_CREDENTIALS points to.

[zhuansunxt]

• Could you also add a quick example in https://github.com/delta-io/delta-sharing/blob/main/server/src/universal/conf/delta-sharing-server.yaml.template#L14 ?

Added.

• For the test, I think we also need to write code to generate the file that GOOGLE_APPLICATION_CREDENTIALS points to.

Why can not we just setup the file path?

[zsxwing]

Why can not we just setup the file path?

GitHub Actions runs in a container, but we don't want to build a public docker image that includes the google key file. To be clear, I meant the following steps:

• Store the key file content in GitHub secrets.
• Add GOOGLE_APPLICATION_CREDENTIALS to point to a file path, such as /tmp/google_key.json.
• Read the key file content from env and write it into the file /tmp/google_key.json.

[zhuansunxt]

@zsxwing I added code to write temp file with key body loaded from github secret in integration test and it is verified to work (see https://github.com/delta-io/delta-sharing/tree/integration-test). However there is another issue of "Type[URL]" has no attribute "_PATH_REQUOTER" in the Github Action even if yarl is installed with desirable version (tried both 1.6.0 and 1.7.2).

[zsxwing]

However there is another issue of "Type[URL]" has no attribute "_PATH_REQUOTER" in the Github Action even if yarl is installed with desirable version (tried both 1.6.0 and 1.7.2).

We can ignore the mypy check error by adding a comment like this:

URL._PATH_REQUOTER = _Quoter(safe="@:", protected="/+=")  # type: ignore

[zsxwing]

Left some minor style comments. Otherwise LGTM

[zhuansunxt]

@zsxwing All comments addressed and integration tests are running correctly for server, spark and python client. Please take a final look.

[zsxwing]

LGTM