32 Pull requests opened by 19 people

• [SPARK-52399][Kubernetes] Support local ephemeral-storage resource

  #51096 opened Jun 5, 2025

• pyspark: Improve SparkConf typing

  #51100 opened Jun 5, 2025

• [SPARK-52424] Add unit test to verify in Spark Connect, Column object defined outside of an UDF can be referenced properly.

  #51101 opened Jun 5, 2025

• [SPARK-52417][SQL] Simplify Table properties handling in View Schema Evolution Mode

  #51107 opened Jun 7, 2025

• [SPARK-37940][SQL]update error mechanism to fully SparkThrowable

  #51108 opened Jun 7, 2025

• [SPARK-37940][SQL]upgrade to new error class in QueryCompilationErrors.

  #51109 opened Jun 7, 2025

• [SPARK-42841][SQL]Assign a name to the error class _LEGACY_ERROR_TEMP_2005

  #51110 opened Jun 7, 2025

• [SPARK-42841][SQL]Assign a name to the error class _LEGACY_ERROR_TEMP_2003

  #51111 opened Jun 7, 2025

• [SPARK-42329][SQL]Assign name to _LEGACY_ERROR_TEMP_2256

  #51113 opened Jun 8, 2025

• [SPARK-42746][SQL][FIXUP] Fix optimizer failure for SortOrder in the LISTAGG function

  #51117 opened Jun 8, 2025

• [SPARK-42322][SQL]Assign name to_LEGACY_ERROR_TEMP_2235

  #51125 opened Jun 9, 2025

• Test Hadoop 3.4.2

  #51127 opened Jun 9, 2025

• [SPARK-52427][SQL] Normalize aggregate expression list covered by a Project

  #51128 opened Jun 9, 2025

• [SPARK-52426][CORE] Support redirecting stdout/stderr to logging system

  #51130 opened Jun 9, 2025

• [SPARK-52395][CORE] Fast fail when shuffle fetch failure happens

  #51131 opened Jun 9, 2025

• [SPARK-33538][SQL]Directly push IN predicates to the Hive Metastore

  #51132 opened Jun 9, 2025

• Quiet "Unable to load native-hadoop library for your platform" message

  #51136 opened Jun 9, 2025

• [SPARK-52433][PYTHON] Unify the string coercion in createDataFrame

  #51140 opened Jun 10, 2025

• Better do idle python workers cleanup

  #51143 opened Jun 10, 2025

• [SPARK-52439][SQL] Support creating check constraint with NULL

  #51146 opened Jun 10, 2025

• [WIP][INFRA] Retry SBT compilation when OOM

  #51149 opened Jun 11, 2025

• [WIP][INFRA] Upgrade to Python 3.11 in doc image

  #51150 opened Jun 11, 2025

• [BINDER] Upgrade pip in postBuild to fix ResolutionImpossible Fix: Binder build failure due to pip dependency resolution

  #51152 opened Jun 11, 2025

• [SPARK-49110][SQL] Fix reading metadata columns for tables with CHAR columns

  #51154 opened Jun 11, 2025

• [SPARK-52447][SQL] Move GetViewColumnByNameAndOrdinal normalization to SessionCatalogSuite

  #51155 opened Jun 11, 2025

• [WIP][SQL] Store internal TIME values as nanoseconds

  #51156 opened Jun 11, 2025

• [SPARK-52450][CONNECT] Improve performance of schema deepcopy

  #51157 opened Jun 11, 2025

• [SPARK-52422][PYTHON][FOLLOW-UP] Upgrade required pandas version in API doc gen

  #51158 opened Jun 11, 2025

• [SPARK-52453][BUILD] Automatically release and drop artifacts in Apache Nexus repository

  #51159 opened Jun 11, 2025

• [SPARK-52454][BUILD] Automatically remove old releases from the mirror

  #51160 opened Jun 11, 2025

• [SPARK-52455][SQL][TESTS] Test expression-based defaults in UPDATE and MERGE

  #51161 opened Jun 11, 2025

• [SPARK-52431][SDP] Finishing touches on Declarative Pipelines runner

  #51165 opened Jun 12, 2025

21 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• [SPARK-52388][SQL] Handle named and positional parameters under `PlanWithUnresolvedIdentifier`

  #51073 commented on Jun 6, 2025 • 6 new comments

• [SPARK-47547] BloomFilter fpp degradation

  #50933 commented on Jun 8, 2025 • 6 new comments

• [SPARK-52354] Add type coercion to UnionLoop

  #51063 commented on Jun 12, 2025 • 6 new comments

• [SPARK-52403][SQL] Add numTargetRowsCopied metric to MergeRowExec

  #51091 commented on Jun 7, 2025 • 5 new comments

• [SPARK-51554][SQL] Add the time_trunc() function for TIME datatype

  #50607 commented on Jun 11, 2025 • 5 new comments

• [SPARK-52381][Core] JsonProtocol: Only accept subclasses of SparkListenerEvent

  #51061 commented on Jun 6, 2025 • 5 new comments

• [SPARK-52402][PS] Fix divide-by-zero errors in Kendall and Pearson correlation under ANSI mode

  #51090 commented on Jun 8, 2025 • 2 new comments

• [SPARK-52065][SQL] Produce another plan tree with output columns (name, data type, nullability) in plan change logging

  #50852 commented on Jun 9, 2025 • 1 new comment

• [SPARK-52356][PS] Enable divide-by-zero for boolean floordiv with ANSI enabled

  #51079 commented on Jun 8, 2025 • 1 new comment

• [MINOR] Improve documentation of concat functions in pyspark

  #51086 commented on Jun 6, 2025 • 0 new comments

• Creating minimal CVE spark base image

  #51076 commented on Jun 9, 2025 • 0 new comments

• [SPARK-47404][SQL] Add configurable size limits for ANTLR DFA cache

  #51069 commented on Jun 12, 2025 • 0 new comments

• [SPARK-51831][SQL] Column pruning with existsJoin for Datasource V2

  #51046 commented on Jun 10, 2025 • 0 new comments

• [SPARK-52327][Core] Glob based provider for history server

  #51045 commented on Jun 9, 2025 • 0 new comments

• Increase report interval of spaming logs to 10 seconds

  #51012 commented on Jun 9, 2025 • 0 new comments

• [SPARK-52186][SQL] Rewrite EXISTS to add Scalar Project

  #50917 commented on Jun 6, 2025 • 0 new comments

• [Spark-50873][SQL] Prune column for ExistsSubquery in DSV2

  #50916 commented on Jun 6, 2025 • 0 new comments

• [SPARK-52104][CONNECT][SCALA] Validate column name eagerly in Spark Connect Scala Client

  #50873 commented on Jun 10, 2025 • 0 new comments

• [SPARK-51699][BUILD] Upgrade to Apache parent pom 34

  #50627 commented on Jun 9, 2025 • 0 new comments

• Enable -Xsource:3 compiler flag

  #50474 commented on Jun 8, 2025 • 0 new comments

• [SPARK-51256][SQL] Increase parallelism if joining with small bucket table

  #50004 commented on Jun 12, 2025 • 0 new comments