You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If we want to support Maggy on arbitrary Spark clusters, we need a reliable way of inferring the number of executors that are available on the Spark cluster.
Additionally, as a fallback the user should be able to specify a lower number of executors in case it's a shared cluster.
We have to look at the following properties:
spark.dynamicAllocation.maxExecutors
spark.executor.instances for static
Find equivalents on Databricks.
The text was updated successfully, but these errors were encountered:
If we want to support Maggy on arbitrary Spark clusters, we need a reliable way of inferring the number of executors that are available on the Spark cluster.
Additionally, as a fallback the user should be able to specify a lower number of executors in case it's a shared cluster.
We have to look at the following properties:
spark.dynamicAllocation.maxExecutors
spark.executor.instances for static
Find equivalents on Databricks.
The text was updated successfully, but these errors were encountered: