You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Below PR adds an API for data sources to indicate the advisory partition size for V2 writes. Even though this is applicable only when AQE is enabled and we don't have code where we support advisorySize, we may have to update the case class of GpuShuffleExchageExec for Spark-3.5+ since it's updated now
case class ShuffleExchangeExec(
override val outputPartitioning: Partitioning,
child: SparkPlan,
shuffleOrigin: ShuffleOrigin = ENSURE_REQUIREMENTS,
advisoryPartitionSize: Option[Long] = None)
Below PR adds an API for data sources to indicate the advisory partition size for V2 writes. Even though this is applicable only when AQE is enabled and we don't have code where we support
advisorySize
, we may have to update the case class of GpuShuffleExchageExec for Spark-3.5+ since it's updated nowApache Spark commit - apache/spark@59e0066dc7
Follow-on issue - apache/spark@2fece076a3
The text was updated successfully, but these errors were encountered: