Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sparknlp 868 make spark driver cores override local in start functions #13894

Conversation

maziyarpanahi
Copy link
Member

  • Users can use params to add spark.driver.cores, however, the hard-coded local[*] is overriding that
  • So we need to check if spark.driver.cores is set, both added to the conf and make sure local[N] is set where N is spark.driver.cores
  • There is a TypeError now checking if the params in Python side is an actual dict

@sonatype-lift
Copy link

sonatype-lift bot commented Jul 18, 2023

Sonatype Lift is retiring

Sonatype Lift will be retiring on Sep 12, 2023, with its analysis stopping on Aug 12, 2023. We understand that this news may come as a disappointment, and Sonatype is committed to helping you transition off it seamlessly. If you’d like to retain your data, please export your issues from the web console.
We are extremely grateful and thank you for your support over the years.

📖 Read about the impacts and timeline

@maziyarpanahi maziyarpanahi self-assigned this Jul 18, 2023
@maziyarpanahi maziyarpanahi changed the base branch from master to release/501-release-candidate July 18, 2023 11:18
@maziyarpanahi maziyarpanahi merged commit 893b693 into release/501-release-candidate Jul 18, 2023
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant