Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Scala 2.13 #38

Closed
benedeki opened this issue Apr 5, 2023 · 6 comments
Closed

Add support for Scala 2.13 #38

benedeki opened this issue Apr 5, 2023 · 6 comments
Labels
enhancement New feature or request

Comments

@benedeki
Copy link

benedeki commented Apr 5, 2023

Background

It has been requested to be able to use the library in a project built for on Scala 2.13

@benedeki benedeki added the enhancement New feature or request label Apr 5, 2023
@Zejnilovic
Copy link
Contributor

Zejnilovic commented Apr 5, 2023

AFAIK hats and AbsaOSS/spark-hofs#19 are for Spark 2.
Spark 2 does not support Scala 2.13.
Does it make sense to update these to use scala 2.13?

@yruslan
Copy link
Contributor

yruslan commented Apr 6, 2023

Hofs was created to add functionality missing in Spark 2. But Spark 3 has 'transform' etc. So hofs might become obsolete.

But hats adds nice interface on the top of standard functions for better array manipulation. And if it can be rewritten to use standard Spark's transform(), it makes sense to still have it in Spark 3.

@Zejnilovic
Copy link
Contributor

Hofs was created to add functionality missing in Spark 2. But Spark 3 has 'transform' etc. So hofs might become obsolete.

But hats adds nice interface on the top of standard functions for better array manipulation. And if it can be rewritten to use standard Spark's transform(), it makes sense to still have it in Spark 3.

Ideal to be done as part of #33

@benedeki
Copy link
Author

benedeki commented Apr 6, 2023

It would be cool, but would require the release to be split to version for Spark 2 and another one for Spark 3 (as spark-commons, and spark-partition-sizing have it)

@yruslan
Copy link
Contributor

yruslan commented Aug 1, 2023

I'm going to go for the same implementation for all Scala versions for now. Optimizing hofs away from Scala 2.12 and Scala 2.13 will be another issue and another PR.

I'll try to remove hofs dependency from Scala 2.12 and Scala 2.13 builds.

yruslan added a commit that referenced this issue Aug 1, 2023
yruslan added a commit that referenced this issue Aug 1, 2023
yruslan added a commit that referenced this issue Aug 1, 2023
yruslan added a commit that referenced this issue Aug 1, 2023
yruslan added a commit that referenced this issue Aug 1, 2023
yruslan added a commit that referenced this issue Aug 1, 2023
yruslan added a commit that referenced this issue Aug 1, 2023
yruslan added a commit that referenced this issue Aug 3, 2023
yruslan added a commit that referenced this issue Aug 3, 2023
yruslan added a commit that referenced this issue Aug 3, 2023
yruslan added a commit that referenced this issue Aug 3, 2023
yruslan added a commit that referenced this issue Aug 3, 2023
@yruslan
Copy link
Contributor

yruslan commented Aug 3, 2023

Fixed in #43

@yruslan yruslan closed this as completed Aug 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants