Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test multiple python versions & revert python 3.10 not being supported #31

Merged
merged 3 commits into from
Aug 17, 2022

Conversation

dbaxa
Copy link
Contributor

@dbaxa dbaxa commented Aug 15, 2022

Hi, we are making use of this library and have for some time been using it in combination with newer versions of pandas (1.4.3) & numpy (1.23.1), and pyarrow (8.0.0 , however version 9.0.0 seems fine). While version 2.0.2 of this library works for us ... version 2.0.3 does not due to the changes to the dependency specification which break its usage in python 3.10.

This fixes #26 .

Signed-off-by: David Black <dblack@atlassian.com>
…n 3.10 & a specific version of numpy being pulled in by pyarrow.

Signed-off-by: David Black <dblack@atlassian.com>
@@ -11,8 +11,8 @@ include = ["CHANGELOG.md"]
[tool.poetry.dependencies]
python = "^3.7.1"
thrift = "^0.13.0"
pyarrow = "^5.0.0"
pandas = "^1.3.0"
Copy link
Contributor Author

@dbaxa dbaxa Aug 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO this library should drop support for python 3.7 & the update pandas minimum version to 1.4.3.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understandable. Because many Databricks users connect from older Pythons we prefer to maintain as much backwards compatibility as possible. For your use-case, is there any reason you cannot install a newer pandas? Or does databricks-sql-connector use pandas methods that were deprecated in newer versions?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No -^ it isn't an issue for us. It is rather that many libraries have dropped support for python 3.7 & you may wish to update pandas to a newer version at some point which will mean dropping support for python 3.7.

@dbaxa dbaxa changed the title Test multiple python versions Test multiple python versions & revert python 3.10 not being supported Aug 15, 2022
@dbaxa
Copy link
Contributor Author

dbaxa commented Aug 15, 2022

You can see test execution at https://github.com/dbaxa/databricks-sql-python/runs/7837469701.

@dbaxa
Copy link
Contributor Author

dbaxa commented Aug 15, 2022

cc @susodapop

Copy link
Contributor

@susodapop susodapop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent work. Thanks for adding the test matrix for different versions!

@susodapop
Copy link
Contributor

I'm running the e2e test suite against this change now.

@susodapop
Copy link
Contributor

All e2e tests pass in the following Python versions:

  • 3.7.9
  • 3.8.13
  • 3.9.13
  • 3.10.5

@susodapop susodapop merged commit cabc7f9 into databricks:main Aug 17, 2022
moderakh pushed a commit to moderakh/databricks-sql-python that referenced this pull request Aug 24, 2022
* Test with multiple python versions.
* Update pyarrow to version 9.0.0 to address issue in relation to python 3.10 & a specific version of numpy being pulled in by pyarrow.

Closes databricks#26

Signed-off-by: David Black <dblack@atlassian.com>
@susodapop susodapop mentioned this pull request Dec 14, 2022
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for Python 3.10
2 participants