Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: Removing Spark 1.x, Scala 2.10 support in 0.24.0 release #1597

Closed
fnothaft opened this Issue Jul 10, 2017 · 4 comments

Comments

Projects
3 participants
@fnothaft
Copy link
Member

commented Jul 10, 2017

From the spark-dev list, it looks like Spark 2.3.0 will add support for Scala 2.12 and drop support for Scala 2.10. As such, I think that we should drop Spark 1.x and Scala 2.10 support in the first ADAM release after Spark 2.3.0, which for now, I am assuming to be ADAM 0.24.0. Thoughts?

@fnothaft fnothaft added the discussion label Jul 10, 2017

@fnothaft fnothaft added this to the 0.24.0 milestone Jul 10, 2017

@heuermh

This comment has been minimized.

Copy link
Member

commented Jul 11, 2017

+1

For me, the stragglers are cgcloud and dockerized Spark for Toil, both of which are still our responsibility to some extent. I haven't heard from any external users still wanting Spark 1.x.

@antonkulaga

This comment has been minimized.

Copy link
Contributor

commented Jul 13, 2017

+1

@antonkulaga

This comment has been minimized.

Copy link
Contributor

commented Jul 26, 2017

Maybe for 0.23 release, you can at least make Spark 2 and Scala 2.11 the default one so people will not need to run 2.11/spark2 bash scripts each time the clone adam?

@heuermh

This comment has been minimized.

Copy link
Member

commented Dec 5, 2017

Maybe for 0.23 release, you can at least make Spark 2 and Scala 2.11 the default one...

Sorry, misread this originally. We don't want to change the release scripts before the 0.23.0 release, so Spark 1.x/Scala 2.10 will remain the default.

@heuermh heuermh added this to Triage in Release 0.24.0 Jan 4, 2018

@heuermh heuermh referenced this issue Jan 5, 2018

Merged

[ADAM-1597] Move to Scala 2.11 and Spark 2.x. #1861

9 of 9 tasks complete

fnothaft added a commit that referenced this issue Jan 9, 2018

[ADAM-1597] Move to Scala 2.11 and Spark 2.x.
Resolves #1597:

* Remove Scala 2.10 and Spark 1.x from jenkins-test.
* Only build and release Scala 2.11 and Spark 2.x.
* Bump Scala dependency version to 2.11.12.
* Bump Spark dependency version to 2.2.1.
* Use Spark version 2.2.1 in jenkins-test.
* Try curl with Location redirect for Apache mirror script.
* Remove mention of Scala 2.10 and Spark 1.x from docs.
* Remove references to cgcloud.

@heuermh heuermh moved this from Triage to Completed in Release 0.24.0 Jan 9, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.