Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DS VM pySpark version and library mismatch #28

Closed
maxkazmsft opened this issue Oct 10, 2018 · 1 comment
Closed

DS VM pySpark version and library mismatch #28

maxkazmsft opened this issue Oct 10, 2018 · 1 comment
Labels
good first issue Good for newcomers
Milestone
Open Source Release

Comments

@maxkazmsft
Copy link
Collaborator

As DS VM upgraded to Spark 2.3 the supporting libraries and environment don't work with Spark 2.2 which is required for Airship (note Recommenders). The quick fix is to upgrade virtual env to Spark 2.3. Going forward, we should figure out if we want to keep upgrading Spark versions as DS VM upgrades them OR anchor to specific Spark version with standalone DS VM libs.
@yueguoguo yueguoguo added the good first issue Good for newcomers label Oct 10, 2018
@miguelgfierro
Copy link
Collaborator

hey @maxkazmsft, I added a troubleshooting point addressing this in the install.md, see #69, will this solve this issue?

@miguelgfierro miguelgfierro mentioned this issue Oct 18, 2018
Closed
@miguelgfierro miguelgfierro added this to the Early Access milestone Oct 25, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
good first issue Good for newcomers
Projects
None yet
Milestone
Open Source Release
Development

No branches or pull requests
4 participants
@yueguoguo @miguelgfierro @nikhilrj @maxkazmsft