[SPARK-36654][PYTHON] Drop type ignores from numpy imports

[zero323]

What changes were proposed in this pull request?

This PR removes type: ignore[import] annotations from numpy imports.

Additionally, minimum version of numpy required for the mypy tests is explicitly stated in the GitHub workflow files.

Why are the changes needed?

Since version 1.20 numpy is PEP 561 compatible so these ignores are no longer necessary.

Does this PR introduce any user-facing change?

This change targets primarily our development process and should be relatively transparent to the end users:

• If pyspark is installed as a package in the current environment both current master and this pr yield the same results (test_pr_package.sh.out.txt and test_master_package.sh.out.txt respectively)

• If pyspark is added manually to MYPYPATH both this pr and current master yield large number of errors (test_pr_mypypath.sh.out.txt and test_master_mypypath.sh.out.txt respectively)

  These errors are primarily, but not exclusively, related to pyspark.pandas hints (notable exception is cloudpickle ‒ to avoid this we could bring back stub file, that was removed during initial type hints migration).

  In this case, users can silence the errors,, by providing appropriate mypy.ini file.

This behavior was tested using simple test files

from pyspark import SparkContext

sc = SparkContext.getOrCreate()
reveal_type(sc.parallelize([1, 2, 3]))

and

from pyspark.ml.linalg import DenseVector

reveal_type(DenseVector([1, 2, 3]).toArray() + 1)

How was this patch tested?

Existing tests and manual verification of the behavior in isolated environments.

[SparkQA]

Test build #142941 has finished for PR 33900 at commit 5250191.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47442/

[SparkQA]

Test build #142978 has finished for PR 33900 at commit 46362c0.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47479/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47479/

[SparkQA]

Test build #144142 has finished for PR 33900 at commit b5833a3.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48619/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48619/

[SparkQA]

Test build #144193 has finished for PR 33900 at commit dc06c69.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48673/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48673/

[SparkQA]

Test build #144322 has finished for PR 33900 at commit ca11801.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48801/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48801/

[zero323]

Thanks @HyukjinKwon, @ueshin

Merged to master.