[AIRFLOW-6311] Remove python2 from codebase

[xinbinhuang]

Make sure you have checked all steps below.

Jira

• My PR addresses the following Airflow Jira issues and references them in the PR title. For example, "[AIRFLOW-XXX] My Airflow PR"
  • https://issues.apache.org/jira/projects/AIRFLOW/issues/AIRFLOW-6311

Description

• Remove workaround for python2 newstr from settings.py
• Change default python interpreter for dataflow to python3, given that support is mostly finished. and they are also about to sunset python2 in 2020, ( thought the timeline has not been set yet)
  • Dataflow python3 support roadmap: https://issues.apache.org/jira/browse/BEAM-1251#comment-16979381]
  • Python2 sunset schedule: https://issues.apache.org/jira/browse/BEAM-8371
  • Apache-beam sign (without detailed timeline) on https://python3statement.org/
• It depends on when will this be merged in. We may also drop the py_interpreter all together

[potiuk]

Hey @xinbinhuang - I think that's going in the right direction but:

• some unit tests are failing
• it is backwards incompatible change so it requires note In UPDATING.md

[ashb]

This is mostly about changing the default interpreter in GCP operators more than anything (as we've already removed most of the Py2 specific code elsewhere) so this PR title would be better updated to reflect that (PR titles go in to the changelog)

[xinbinhuang]

some unit tests are failing

Sorry,I was making the changes on another PC that I thought have pre-commit hooks installed while did not . I will double-check that.

This is mostly about changing the default interpreter in GCP operators more than anything (as we've already removed most of the Py2 specific code elsewhere) so this PR title would be better updated to reflect that (PR titles go in to the changelog)

@potiuk @ashb I agree with your opinions. I will create another PR to just include the dataflow changes. while this remains simple as removing those 2lines in the settings.py

[xinbinhuang]

Rebased & Remove changes for Dataflow. Will create a separate PR later and link to this one

[codecov-io]

Codecov Report

Merging #6860 into master will decrease coverage by 0.28%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #6860      +/-   ##
==========================================
- Coverage   84.56%   84.28%   -0.29%     
==========================================
  Files         680      680              
  Lines       38454    38453       -1     
==========================================
- Hits        32520    32409     -111     
- Misses       5934     6044     +110

Impacted Files	Coverage Δ
airflow/settings.py	93.66% <ø> (-0.05%)	⬇️
airflow/kubernetes/volume_mount.py	44.44% <0%> (-55.56%)	⬇️
airflow/kubernetes/volume.py	52.94% <0%> (-47.06%)	⬇️
airflow/kubernetes/pod_launcher.py	45.25% <0%> (-46.72%)	⬇️
airflow/kubernetes/refresh_config.py	50.98% <0%> (-23.53%)	⬇️
...rflow/contrib/operators/kubernetes_pod_operator.py	78.75% <0%> (-20%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b7c11e4...5c76741. Read the comment docs.

[potiuk]

I merged and updated the commit subject slightly.

[xinbinhuang]

Thanks! @potiuk