Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Wrap Spark client API calls to detect version mismatches #3054

Open
arielshaqed opened this issue Mar 15, 2022 · 4 comments
Open

Wrap Spark client API calls to detect version mismatches #3054

arielshaqed opened this issue Mar 15, 2022 · 4 comments
Labels
team/ecosystem Team Ecosystem
Projects
lakeFS
Milestone
Spark 3.2

Comments

@arielshaqed
Copy link
Contributor

Typical bad installations of the Spark clients (both lakeFSFS and metadata) have mismatched versions that cause methods and classes not to be found. Wrap all calls with some layer that analyzes relevant exceptions, reports the version(s) with which this client is known to work, suggests using other versions.
@arielshaqed arielshaqed added this to the Spark 3.2 milestone Mar 15, 2022
@arielshaqed arielshaqed added the team/ecosystem Team Ecosystem label Mar 16, 2022
@talSofer talSofer added this to Backlog in lakeFS via automation Oct 2, 2022
@arielshaqed arielshaqed mentioned this issue Feb 9, 2023
Closed
@arielshaqed
Copy link
Contributor Author

Related to #2801, which could/might help here too.

@vaibhavk1992 vaibhavk1992 mentioned this issue Mar 16, 2023
Closed
@github-actions GitHub Actions
Copy link

github-actions bot commented Nov 1, 2023

This issue is now marked as stale after 90 days of inactivity, and will be closed soon. To keep it, mark it with the "no stale" label.

@github-actions github-actions bot added the stale label Nov 1, 2023
@github-actions GitHub Actions
Copy link

Closing this issue because it has been stale for 7 days with no activity.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Nov 11, 2023
@arielshaqed
Copy link
Contributor Author

Still needed, this is recurs.

@arielshaqed arielshaqed reopened this Nov 19, 2023
@arielshaqed arielshaqed added the no stale Using this label will prevent items from being marked as stale label Nov 19, 2023
@johnnyaug johnnyaug removed the stale label Nov 28, 2023
@github-actions github-actions bot added the stale label Mar 20, 2024
@arielshaqed arielshaqed removed no stale Using this label will prevent items from being marked as stale stale labels Mar 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
team/ecosystem Team Ecosystem
Projects
No open projects
lakeFS
  
Backlog
Milestone
Spark 3.2
Development

No branches or pull requests
2 participants
@johnnyaug @arielshaqed