Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SEDONA-129] Bump spark to 3.3.0 #636

Merged
merged 2 commits into from Jun 26, 2022
Merged

Conversation

Kimahriman
Copy link
Contributor

Did you read the Contributor Guide?

Is this PR related to a JIRA ticket?

What changes were proposed in this PR?

Bump Spark version to 3.3.0. Update GitHub workflows to test this as well.

How was this patch tested?

Updated workflows.

Did this PR include necessary documentation updates?

  • No, this PR does not affect any public API so no need to change the docs.

@Kimahriman
Copy link
Contributor Author

I included similar changes to #635 to get the builds to work. Took a slightly different approach opting for adapting the new behavior vs using more of the old behavior.

Follow on discussion: is it time to remove Spark 2.4 support? Would simplify workflows and code a bit, it's been EOL for quite a while now.

Also, anything else that needs to be done to get a Scala 2.13 release, or is that just a thing that would have to be done manually during the release process?

@jiayuasu
Copy link
Member

@Kimahriman Thanks for bring this up. I also want to drop the support of Spark 2.4 but since this is a major change, I will start a vote in Sedona community today.

@umartin
Copy link
Contributor

umartin commented Jun 19, 2022

I would be happy to withdraw my patch as this is more complete.

Good work!

@jiayuasu jiayuasu requested a review from Imbruced June 20, 2022 05:11
@jiayuasu jiayuasu merged commit 2d0c51f into apache:master Jun 26, 2022
@jiayuasu jiayuasu deleted the spark-3.3 branch June 26, 2022 06:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants