Run test for Python-3.11 #1237

pankajastro · 2023-07-11T14:56:01Z

Run unit tests for Python3.11
exclude hive extra from it because of issue cloudera/python-sasl#30

pankajastro · 2023-07-11T15:22:07Z

CI failed while building sasl mostly CircleCI python env. I'll investigate it further.

pankajkoti

Looks like some checks are failing too (mypy and pre-commit check)

setup.cfg

pankajastro · 2023-07-14T12:45:30Z

~~CI failed while building sasl mostly CircleCI python env. I'll investigate it further.~~

This known issue cloudera/python-sasl#30

.circleci/scripts/pre_commit_readme_extra.py

setup.cfg

mdeshmu · 2023-07-14T14:59:55Z

@pankajastro

I made a couple of contributions to PyHive which were accepted and released in 0.7.1.dev0. You are requested to test with the dev version and report any bugs in the PyHive GitHub repository before 0.7.1 is released in a month or so.

PyHive also supports pure-sasl via additional extras 'pyhive[hive_pure_sasl]' which supports Python 3.11 in addition to previous Python versions. See Use pure-sasl in python 3.11 dropbox/PyHive#454
PyHive is now compatible with SQLAlchemy 2.0. See Adding compatibility with SQLAlchemy 2.0 dropbox/PyHive#457

pankajastro · 2023-07-14T17:07:19Z

@pankajastro

I made a couple of contributions to PyHive which were accepted and released in 0.7.1.dev0. You are requested to test with the dev version and report any bugs in the PyHive GitHub repository before 0.7.1 is released in a month or so.

PyHive also supports pure-sasl via additional extras 'pyhive[hive_pure_sasl]' which supports Python 3.11 in addition to previous Python versions. See Use pure-sasl in python 3.11 dropbox/PyHive#454

PyHive is now compatible with SQLAlchemy 2.0. See Adding compatibility with SQLAlchemy 2.0 dropbox/PyHive#457

Thanks @mdeshmu for letting us know. We are having this issue because we depend on the airflow apache hive provider that depends on sasl. Look like Jarek has already opened PR apache/airflow#32607 for airflow provider testing

codecov · 2023-07-25T16:58:32Z

Codecov Report

Patch and project coverage have no change.

Comparison is base (64dea53) 98.57% compared to head (bd32dc5) 98.57%.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1237   +/-   ##
=======================================
  Coverage   98.57%   98.57%           
=======================================
  Files          90       90           
  Lines        5393     5393           
=======================================
  Hits         5316     5316           
  Misses         77       77

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

pankajkoti

Mostly, looks good to me.

I am checking the results of Python 3.11 test job https://app.circleci.com/pipelines/github/astronomer/astronomer-providers/6082/workflows/fa610ca9-085d-4572-b550-4cefd117fba5/jobs/40521

Did a couple of tests fail and yet it reported a success? Can we please check this?

pankajkoti · 2023-08-01T09:59:46Z

.circleci/config.yml

+                          glob "tests/**/test_*.py" | sed '/tests\/apache\/hive/d' \
+                          | circleci tests split --split-by=timings \
+                        )
+            pytest --junit-xml=test-report/report.xml  $TEST_FILES


Can we add some comment (maybe at the top of the step?) on why we are ignoring to include the code coverage here?

I have added on at the top of this job PTAL

Aha, I missed the codecov step. I did not include it because except for hive all tests are running here and including it will not change coverage. But happy to include it if you think it makes sense

.circleci/scripts/pre_commit_readme_extra.py

.circleci/scripts/pre_commit_setup_cfg_python_3_11_all_extra.py

setup.cfg

pankajkoti · 2023-08-01T10:49:29Z

okay, looks good to me then when the above comments are addressed, great work 👏🏽

pankajastro · 2023-08-01T12:18:33Z

not sure, what happened I just did the force pushed and closed PR 🤔

pankajastro · 2023-08-01T12:19:00Z

re-opening

pankajkoti · 2023-08-01T12:39:55Z

It says PR closed, but PR merged and closed :D

pankajastro · 2023-08-01T12:45:40Z

@pankajkoti yes something I messed up :)

.circleci/scripts/pre_commit_setup_cfg_python_3_11_all_extra.py

Co-authored-by: Wei Lee <weilee.rx@gmail.com>

.circleci/scripts/pre_commit_setup_cfg_python_3_11_all_extra.py

pankajkoti

Looks good to me but can't seem to find codecov comment #1237 (comment)

pankajastro · 2023-08-02T09:58:49Z

Looks good to me but can't seem to find codecov comment #1237 (comment)

I was referring to this comment file .circleci/config.yml

# This runs unit tests for Python3.11 only and excludes the Apache Hive test since
# Currently, the Airflow Hive provider is excluded from Python3.11 in Airflow
# Check the issue https://github.com/cloudera/python-sasl/issues/30 for detail
# PR to bring back Airflow Apache Hive provider https://github.com/apache/airflow/pull/32607

.circleci/config.yml

* Bring back hive provider for Python3.11 * Revert #1237 * Bump hive provider verion * Update config

pankajastro force-pushed the py-3-11 branch 13 times, most recently from 3035a3a to 7313d30 Compare July 12, 2023 14:47

pankajkoti reviewed Jul 14, 2023

View reviewed changes

setup.cfg Show resolved Hide resolved

pankajastro force-pushed the py-3-11 branch 2 times, most recently from 5167ff2 to f181cd8 Compare July 14, 2023 12:43

pankajastro marked this pull request as ready for review July 14, 2023 12:45

pankajastro requested review from phanikumv, Lee-W, sunank200 and utkarsharma2 as code owners July 14, 2023 12:45

pankajastro commented Jul 14, 2023

View reviewed changes

.circleci/scripts/pre_commit_readme_extra.py Show resolved Hide resolved

pankajastro commented Jul 14, 2023

View reviewed changes

setup.cfg Show resolved Hide resolved

pankajastro requested a review from pankajkoti July 14, 2023 12:48

pankajkoti reviewed Aug 1, 2023

View reviewed changes

pankajastro closed this Aug 1, 2023

pankajastro force-pushed the py-3-11 branch from 4e472e8 to 64dea53 Compare August 1, 2023 12:16

pankajastro reopened this Aug 1, 2023

pankajastro force-pushed the py-3-11 branch from 4eb269b to 16d0908 Compare August 1, 2023 12:47

Run test for Python3.11

8395ebb

pankajastro force-pushed the py-3-11 branch from 6418001 to 8395ebb Compare August 1, 2023 15:07

pankajastro added 3 commits August 1, 2023 20:39

Enable action lint

ea17147

Update pre-commit

4f9950e

Update pre-commit

60665b9

pankajastro requested review from Lee-W and pankajkoti August 1, 2023 15:30

Lee-W reviewed Aug 2, 2023

View reviewed changes

.circleci/scripts/pre_commit_setup_cfg_python_3_11_all_extra.py Outdated Show resolved Hide resolved

pankajastro and others added 2 commits August 2, 2023 11:58

Apply review suggestion

ac598df

Apply review suggestion

a3667e5

Co-authored-by: Wei Lee <weilee.rx@gmail.com>

pankajastro commented Aug 2, 2023

View reviewed changes

.circleci/scripts/pre_commit_setup_cfg_python_3_11_all_extra.py Outdated Show resolved Hide resolved

Apply review suggestion

e6db42f

Lee-W approved these changes Aug 2, 2023

View reviewed changes

pankajkoti approved these changes Aug 2, 2023

View reviewed changes

pankajastro commented Aug 2, 2023

View reviewed changes

.circleci/config.yml Outdated Show resolved Hide resolved

Update .circleci/config.yml

bd32dc5

pankajastro merged commit 83957b8 into main Aug 2, 2023
11 checks passed

pankajastro deleted the py-3-11 branch August 2, 2023 16:51

pankajastro added a commit that referenced this pull request Aug 29, 2023

Revert #1237

960b7cc

pankajastro added a commit that referenced this pull request Aug 29, 2023

Bring back hive provider for Python3.11 (#1317)

923cb8c

* Bring back hive provider for Python3.11 * Revert #1237 * Bump hive provider verion * Update config

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Run test for Python-3.11 #1237

Run test for Python-3.11 #1237

pankajastro commented Jul 11, 2023 •

edited

pankajastro commented Jul 11, 2023

pankajkoti left a comment •

edited

pankajastro commented Jul 14, 2023

mdeshmu commented Jul 14, 2023

pankajastro commented Jul 14, 2023

codecov bot commented Jul 25, 2023 •

edited

pankajkoti left a comment

pankajkoti Aug 1, 2023

pankajastro Aug 1, 2023

pankajastro Aug 2, 2023

pankajkoti commented Aug 1, 2023

pankajastro commented Aug 1, 2023

pankajastro commented Aug 1, 2023

pankajkoti commented Aug 1, 2023

pankajastro commented Aug 1, 2023

pankajkoti left a comment

pankajastro commented Aug 2, 2023

Run test for Python-3.11 #1237

Run test for Python-3.11 #1237

Conversation

pankajastro commented Jul 11, 2023 • edited

pankajastro commented Jul 11, 2023

pankajkoti left a comment • edited

Choose a reason for hiding this comment

pankajastro commented Jul 14, 2023

mdeshmu commented Jul 14, 2023

pankajastro commented Jul 14, 2023

codecov bot commented Jul 25, 2023 • edited

Codecov Report

pankajkoti left a comment

Choose a reason for hiding this comment

pankajkoti Aug 1, 2023

Choose a reason for hiding this comment

pankajastro Aug 1, 2023

Choose a reason for hiding this comment

pankajastro Aug 2, 2023

Choose a reason for hiding this comment

pankajkoti commented Aug 1, 2023

pankajastro commented Aug 1, 2023

pankajastro commented Aug 1, 2023

pankajkoti commented Aug 1, 2023

pankajastro commented Aug 1, 2023

pankajkoti left a comment

Choose a reason for hiding this comment

pankajastro commented Aug 2, 2023

pankajastro commented Jul 11, 2023 •

edited

pankajkoti left a comment •

edited

codecov bot commented Jul 25, 2023 •

edited