[SPARK-30205][PYSPARK] Import ABCs from collections.abc to remove deprecation warnings

[tirkarthi]

What changes were proposed in this pull request?

This PR aims to remove deprecation warnings by importing ABCs from collections.abc instead of collections.

• bpo-25988: Do not expose abstract collection classes in the collections module. python/cpython#10596

Why are the changes needed?

This will remove deprecation warnings in Python 3.7 and 3.8.

$ python -V
Python 3.7.5

$ python python/pyspark/resultiterable.py
python/pyspark/resultiterable.py:23: DeprecationWarning:
Using or importing the ABCs from 'collections' instead of from 'collections.abc'
is deprecated since Python 3.3,and in 3.9 it will stop working
  class ResultIterable(collections.Iterable):

Does this PR introduce any user-facing change?

No, this doesn't introduce user-facing change

How was this patch tested?

Manually because this is about deprecation warning messages.

[dongjoon-hyun]

ok to test

[SparkQA]

Test build #115107 has finished for PR 26835 at commit b8a5304.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
• class ResultIterable(Iterable):

[dongjoon-hyun]

Thank you for making your first PR, @tirkarthi .
However, Python 3.9 is not released yet. Can we revisit this as a single PR after Python 3.9 is officially released?

[tirkarthi]

The change was meant to go as part of 3.8 but was postponed to 3.9 as noted in upstream PR due to Python ecosystem being incompatible. The PR also fixes the deprecation warning for all other versions like 3.8 and 3.7. I feel this is good enough to merge but will leave it to maintainers though.

[dongjoon-hyun]

For the following reason, it looks worth to me, too. Thank you for the explanation.

The PR also fixes the deprecation warning for all other versions like 3.8 and 3.7.

Could you change the PR title something like Remove deprecation warnings explicitly?
Never mind.

[dongjoon-hyun]

I will review this again with the Python versions supported by Apache Spark. Thanks!

[tirkarthi]

Sure, thanks for the update.

[dongjoon-hyun]

Hi, @tirkarthi . Could you check if this is the only required instance?

[dongjoon-hyun]

I checked import collections and from collections a little. It looks okay.

[dongjoon-hyun]

+1, LGTM. Thank you for your first contribution, @tirkarthi .
Merged to master.

[dongjoon-hyun]

Thank you, @tirkarthi . (Karthikeyan Singaravelan). You are added to the Apache Spark contributor group and SPARK-30205 is assigned to you.

[tirkarthi]

Thank you :)

[HyukjinKwon]

I know this way is more preferred but can we do this by version-checking? That's consistent in the code base at least.

[tirkarthi]

Sorry, I was not aware of it.