[SPARK-36961][PYTHON] Use PEP526 style variable type hints

[ueshin]

What changes were proposed in this pull request?

Uses PEP526 style variable type hints.

Why are the changes needed?

Now that we have started using newer Python syntax in the code base.
We should use PEP526 style variable type hints.

• https://www.python.org/dev/peps/pep-0526/

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Existing tests.

[ueshin]

cc @zero323 @xinrong-databricks @HyukjinKwon @itholic

[SparkQA]

Test build #144038 has finished for PR 34227 at commit 1068e7d.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
• new_class = type(NameTypeHolder.short_name, (NameTypeHolder,),

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48515/

[xinrong-meng]

Does this PR aim to change pandas API on Spark only?

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48515/

[zero323]

It is awesome @ueshin! Thanks for working on this.

[ueshin]

@xinrong-databricks

Does this PR aim to change pandas API on Spark only?

This also includes two files in sql module:

• python/pyspark/sql/pandas/conversion.py
• python/pyspark/sql/pandas/types.py

[SparkQA]

Test build #144101 has finished for PR 34227 at commit be288e2.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48579/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48579/

[SparkQA]

Test build #144105 has finished for PR 34227 at commit 126c89c.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48583/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48583/

[SparkQA]

Test build #144158 has finished for PR 34227 at commit 96c2a12.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48636/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48636/

[HyukjinKwon]

Merged to master.