Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump Spark, Scala, Hadoop dependency versions #1221

Closed
heuermh opened this issue Oct 23, 2016 · 4 comments
Closed

Bump Spark, Scala, Hadoop dependency versions #1221

heuermh opened this issue Oct 23, 2016 · 4 comments
Labels
Milestone

Comments

@heuermh
Copy link
Member

@heuermh heuermh commented Oct 23, 2016

Apache Spark version 2.0.1 was released on 03 Oct 2016 and defaults to Scala version 2.11.8 and Hadoop 2.7 and later.

Apache Spark version 1.6.2 was released on 25 June 2016 and defaults to Scala version 2.10.5 and Hadoop 2.6. When bumping the Scala version to 2.10.5 downstream I see other libraries requiring 2.10.6.

@heuermh heuermh added the discussion label Oct 23, 2016
@DruidSmith
Copy link

@DruidSmith DruidSmith commented Oct 25, 2016

Is ADAM working on Spark 2.0? Has anyone looked at running ADAM atop AWS EMR?

@heuermh
Copy link
Member Author

@heuermh heuermh commented Oct 25, 2016

Is ADAM working on Spark 2.0?

Yes, as of version 0.20.0, released last week.

@heuermh
Copy link
Member Author

@heuermh heuermh commented Oct 27, 2016

@fnothaft fnothaft added this to the 0.21.0 milestone Nov 8, 2016
fnothaft added a commit to fnothaft/adam that referenced this issue Nov 8, 2016
fnothaft added a commit to fnothaft/adam that referenced this issue Nov 9, 2016
fnothaft added a commit to fnothaft/adam that referenced this issue Nov 9, 2016
heuermh added a commit that referenced this issue Nov 10, 2016
@fnothaft
Copy link
Member

@fnothaft fnothaft commented Nov 10, 2016

Sigh! Reopening this issue. See comment; TL;DR is that #1254 spuriously passed but the move_to_spark2.sh script is broken now on ToT. If no one else is interested, I'll hack on this in the AM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.