ci: bump Python tests to 3.7 and add support for 3.8

[villebro]

SUMMARY

With the bump of pylint to version 2.5.3 per #10101, the Superset build env should now be fully 3.7+ compatible.

TEST PLAN

Run all python tests on 3.7 and ensure CI passes smoothly.

ADDITIONAL INFORMATION

• Has associated issue:
• Changes UI
• Requires DB Migration.
• Confirm DB Migration upgrade and downgrade tested.
• Introduces new feature or API
• Removes existing feature or API

[villebro]

I'm flagging this as a 0.37 dependency, as I feel adding official support for py38 is important as it's already in production use in some large deployments. If changing the test matrix is too much I can remove those for now, but I'd really like for us to consider py38 officially supported.

[ktmud]

Are we dropping support for Python 3.6 yet?

[villebro]

I propose we don't, but next release I think we should.

[villebro]

@mistercrunch how do you feel about moving tests to py37? If this is ok, who can change the required status checks to reference the 3.7 tests?

[etr2460]

After discussion today, folks were in favor of moving to python 3.7 as the supported version and removing official support for Python 3.6 after the 0.37 release

[bkyryliuk]

looking forward to it !

[mistercrunch]

I think it's ok for this project to be prescriptive of the python version (being an app, an not a library). It makes it easier on us as the build matrix can be focussed on a single version, the docker can be aligned with that version, and that everyone running Superset in production should align with the prescribed version for that release. So say if it's 3.7 after this version, it's "run on anything else at your own risk".

The alternative is to cover a version range of python, and to do that well we need to explode our build matrix. After the whole 2.7 to 3.6 fiasco I'd rather stay away from being straddling across python versions.

[etr2460]

lgtm with one comment

[etr2460]

Should we remove this line?

[villebro]

Let's keep it there a while longer and drop it in a separate PR where dataclasses and other backwards compatibility libs are also dropped.

[mistercrunch]

Close/reopened to re-trigger CI that was stuck

[willbarrett]

@villebro I'm thinking we need to ask Apache to update the required checks to 3.7, or temporarily also run against 3.6

[villebro]

Ok @willbarrett , let's temporarily enable 3.6 and 3.7 and disable 3.6 once Apache updates. I'll make the change and reach out to ASF infra.

Edit: decided against it to avoid slowing down CI.

[villebro]

ASF Infra ticket: https://issues.apache.org/jira/browse/INFRA-20643

[codecov-commenter]

Codecov Report

Merging #10110 into master will increase coverage by 0.03%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #10110      +/-   ##
==========================================
+ Coverage   63.62%   63.65%   +0.03%     
==========================================
  Files         764      764              
  Lines       36158    36158              
  Branches     3438     3438              
==========================================
+ Hits        23004    23017      +13     
+ Misses      13041    13028      -13     
  Partials      113      113              

Flag	Coverage Δ
#cypress	54.50% <ø> (ø)
#javascript	59.92% <ø> (ø)
#python	59.09% <ø> (+0.05%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
superset/connectors/sqla/models.py	89.54% <0.00%> (+0.14%)	⬆️
superset/views/core.py	74.33% <0.00%> (+0.25%)	⬆️
superset/models/core.py	87.22% <0.00%> (+0.83%)	⬆️
superset/db_engine_specs/mysql.py	91.48% <0.00%> (+12.76%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 57dc762...293bf6a. Read the comment docs.