Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump to quinn 1.0 #142

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open

Conversation

fpgmaas
Copy link

@fpgmaas fpgmaas commented Oct 19, 2023

This PR does the following:

  • Facilitates running unittests for multiple versions of Python and pyspark on each PR, see here
  • Drops support for Python 3.7 since it is EOL
  • Also drops support for Pyspark 2.x due to the dropped support for Python 3.7
  • Updates some of the dependencies in the docs dependency group.
  • Fixes a few linting issues.
  • Temporarily fixes a failing unittest

Let me know your thoughts :)

@SemyonSinchenko
Copy link
Collaborator

Python 3.7 may be EOL, but Spark 2 is still using in a lot of companies... Do we really need to drop the support of Spark 2? @MrPowers what do you think about it?

@fpgmaas
Copy link
Author

fpgmaas commented Oct 24, 2023

Python 3.7 may be EOL, but Spark 2 is still using in a lot of companies... Do we really need to drop the support of Spark 2? @MrPowers what do you think about it?

Valid point. I proposed to drop support for Spark 2 as a consequence of dropping support for Python 3.7, since that seems to be the latest version that supports Spark 2, see here.

I tried to run the tests with Spark 2 and Python 3.8, but that failed.

So I believe that if we want to keep supporting Spark 2 then we should also keep supporting Python 3.7. But this will prevent us from updating some dependencies and it requires us to keep using some workarounds such as this one.

If it's preferred to not drop support for Python 3.7 and Spark 2, let me know! I will try to update the PR accordingly.

@MrPowers
Copy link
Owner

Let's keep Python 3.7 & Spark 2 support for now.

It's super annoying, but I know lots of users are on legacy versions. But let's drop soon.

@MrPowers MrPowers changed the title Test against multiple versions of Python and drop support for Python 3.7 Bump to quinn 1.0 Mar 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants