29 Pull requests opened by 25 people

• [SPARK-50838][SQL] Add Cross Join as legal in recursion of Recursive CTE

  #50308 opened Mar 18, 2025

• [SPARK-51566][PYTHON] Python UDF traceback improvement

  #50313 opened Mar 18, 2025

• [SPARK-51187][SQL][SS] Introduce the migration logic of config removal from SPARK-49699

  #50314 opened Mar 19, 2025

• [SPARK-51548][SQL] Provides configuration to decide whether to copy objects before shuffle.

  #50318 opened Mar 19, 2025

• [SPARK-51552] [SQL] Disallow temporary variables in persisted views when under identifier

  #50325 opened Mar 19, 2025

• [SPARK-51568][SQL] Introduce isSupportedExtract to prevent happening unexpected behavior

  #50333 opened Mar 20, 2025

• [SPARK-51537][CONNECT][CORE] construct classpath using both global jars and session specific jars on executor

  #50334 opened Mar 20, 2025

• [WIP][SPARK-51423][SQL] Add the current_time() function for TIME datatype

  #50336 opened Mar 20, 2025

• [SPARK-51575][PYTHON] Combine Python Data Source pushdown & plan read workers

  #50340 opened Mar 20, 2025

• [SPARK-51573][SS] Fix Streaming State Checkpoint v2 checkpointInfo race condition

  #50344 opened Mar 21, 2025

• [SPARK-51580][SQL] Throw proper user facing error message when lambda function is out of place in `HigherOrderFunction`

  #50345 opened Mar 21, 2025

• [SPARK-51583] [SQL] Improve error message when to_timestamp function has arguments of wrong type

  #50347 opened Mar 21, 2025

• [SPARK-51586][SS] initialize input partitions independent of columnar support

  #50348 opened Mar 21, 2025

• [SPARK-51587][PYTHON][SS] Fix an issue where timestamp cannot be used in ListState when multiple state data is involved

  #50349 opened Mar 21, 2025

• [SPARK-51588][SQL][TESTS] Validate default values handling in micro-batch writes

  #50351 opened Mar 21, 2025

• [SPARK-51585][SQL] Oracle dialect supports pushdown datetime functions

  #50353 opened Mar 22, 2025

• [SPARK-51419][SQL] Get hour of TIME datatype

  #50355 opened Mar 23, 2025

• Canonicalized CTEs to avoid cached result not being used and recomputed

  #50360 opened Mar 24, 2025

• [SPARK-51062][PYTHON] Fix assertSchemaEqual to compare decimal precision and scale

  #50365 opened Mar 24, 2025

• [SPARK-51593][SQL] Refactor `QueryExecutionMetering` instantiation

  #50366 opened Mar 24, 2025

• [SPARK-51594][SQL] Use empty schema when saving a view which is not Hive compatible

  #50367 opened Mar 24, 2025

• [SPARK-51592][SQL] Avoid logging makes Driver OOM in AdaptiveSparkPlanExec

  #50368 opened Mar 24, 2025

• [SPARK-51597][PYTHON][3.5] Fix deprecated/invalid config in `setup.cfg`

  #50369 opened Mar 24, 2025

• docs: update commet for `nullIntolerant`

  #50370 opened Mar 24, 2025

• [SPARK-45994][PYTHON][3.5] Change `description-file` to `description_file`

  #50371 opened Mar 24, 2025

• [Pyspark 3.4][SPARK-45994][PYTHON] Change description-file to description_file

  #50372 opened Mar 24, 2025

• [WIP][SPARK-51439] Support SQL UDF with DEFAULT argument

  #50373 opened Mar 24, 2025

• [MINOR][DOCS]: corrected spellings and typos

  #50376 opened Mar 24, 2025

• [SPARK-46860][CORE]Enable basic authentication when downloading jars and other files from secured repos

  #50377 opened Mar 24, 2025

30 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• [SPARK-51358] [SS] Introduce snapshot upload lag detection through StateStoreCoordinator

  #50123 commented on Mar 25, 2025 • 73 new comments

• [SPARK-51372][SQL] Introduce TableInfo for table creations

  #50137 commented on Mar 24, 2025 • 35 new comments

• [SPARK-51350][SQL] Implement Show Procedures

  #50109 commented on Mar 24, 2025 • 20 new comments

• [SPARK-51414][SQL] Add the make_time() function

  #50269 commented on Mar 22, 2025 • 9 new comments

• [SPARK-XXXXX][SQL] Add maxRecordsPerOutputBatch to limit the number of record of Arrow output batch

  #50301 commented on Mar 24, 2025 • 6 new comments

• [SPARK-51505][SQL] Log empty partition number metrics in AQE coalesce

  #50273 commented on Mar 24, 2025 • 6 new comments

• [WIP][SPARK-51395][SQL] Refine handling of default values in procedures

  #50197 commented on Mar 21, 2025 • 5 new comments

• [SPARK-51441][SQL] Add DSv2 APIs for constraints

  #50253 commented on Mar 21, 2025 • 5 new comments

• [SPARK-51191][SQL] Validate default values handling in DELETE, UPDATE, MERGE

  #50271 commented on Mar 21, 2025 • 4 new comments

• [SPARK-44856][PYTHON] Improve Python UDTF arrow serializer performance

  #50099 commented on Mar 18, 2025 • 3 new comments

• [SPARK-51400] Replace ArrayContains nodes to InSet

  #50170 commented on Mar 21, 2025 • 3 new comments

• [SPARK-51513][SQL] Fix RewriteMergeIntoTable rule produces unresolved plan

  #50281 commented on Mar 25, 2025 • 2 new comments

• [SPARK-51338][INFRA] Add automated CI build for `connect-examples`

  #50187 commented on Mar 24, 2025 • 2 new comments

• [SPARK-51479][SQL] Nullable in Row Level Operation Column is not correct

  #50246 commented on Mar 20, 2025 • 2 new comments

• [SPARK-51270] Support nanosecond precision timestamp in Variant

  #50270 commented on Mar 24, 2025 • 1 new comment

• [SPARK-51272][CORE]. Fix for the race condition in Scheduler causing failure in retrying all partitions in case of indeterministic shuffle keys

  #50033 commented on Mar 18, 2025 • 1 new comment

• [SPARK-51534][Core] Improve parquet read performance by reducing unnecessary operations when converting parquet to row

  #50297 commented on Mar 21, 2025 • 0 new comments

• [SPARK-51574][PYTHON] Filter serialization for Python Data Source filter pushdown

  #50252 commented on Mar 20, 2025 • 0 new comments

• [SPARK-51503][SQL] Support Variant type in XML scan

  #50300 commented on Mar 19, 2025 • 0 new comments

• [SPARK-51430][PYTHON] Stop PySpark context logger from propagating logs to stdout

  #50198 commented on Mar 19, 2025 • 0 new comments

• [WIP][SPARK-51411][SS][DOCS] Add documentation for the transformWithState operator

  #50177 commented on Mar 21, 2025 • 0 new comments

• [SPARK-51332][SQL] DS V2 supports push down BIT_AND, BIT_OR, BIT_XOR, BIT_COUNT and BIT_GET

  #50097 commented on Mar 18, 2025 • 0 new comments

• [SPARK-51301][BUILD] Bump zstd-jni 1.5.7-1

  #50057 commented on Mar 24, 2025 • 0 new comments

• [SPARK-51298][SQL] Support variant in CSV scan

  #50052 commented on Mar 24, 2025 • 0 new comments

• [SPARK-51252] [SS] Add instance metrics for last uploaded snapshot version in HDFS State Stores

  #50030 commented on Mar 24, 2025 • 0 new comments

• [SPARK-51016][SQL] Stage.isIndeterminate gives wrong result in case the shuffle partitioner uses an inDeterministic attribute or expression.

  #50029 commented on Mar 20, 2025 • 0 new comments

• [SPARK-50992][SQL] OOMs and performance issues with AQE in large plans

  #49724 commented on Mar 23, 2025 • 0 new comments

• [SPARK-50806][SQL] Support InputRDDCodegen interruption on task cancellation

  #49501 commented on Mar 19, 2025 • 0 new comments

• [SPARK-50417] Make number of FallbackStorage sub-directories configurable

  #48960 commented on Mar 20, 2025 • 0 new comments

• [SPARK-37019][SQL] Add codegen support to array higher-order functions

  #34558 commented on Mar 19, 2025 • 0 new comments