Repository navigation
Pista 2.6.0
Pista 2.6.0 updates Doris transaction handling and corrects JDBC Reader behavior. The release retains the documented Spark SQL runtime and target-specific delivery scope.
Changes
- Doris 2PC transaction-query failures are reported rather than treated as empty transaction results. Each connector write uses its own transaction label namespace for verification and failure cleanup.
- Explicit Doris 2PC append uses the Spark connector, including for small inputs. Incompatible forced-JDBC requests and conflicting destination settings fail before a target write.
- JDBC Readers apply configured predicates and column selection. ClickHouse JDBC Reader schema inference and configured authentication are corrected.
- Integration checks are split by domain, and the test runner verifies discovered tests against execution reports. The repository includes a pinned Maven Wrapper.
Upgrade notes
- When
spark.pista.doris.connector.enable2PC=true, configure the connector destination and useappendfor non-overwrite writes. An append cannot also setspark.pista.output.options.force.jdbc=true; qualify the output path asdb.tablewhen the configured database is notdefault. - With Doris metadata enabled, use the same task identity only for retries of the same business batch and a distinct label prefix for each new batch. If target publication succeeds but metadata recording fails, reconcile the target before retrying. Historical transactions are no longer automatically aborted before a new write.
- Direct callers or implementers of
TransactionCoordinatormust adapt to the per-writeDorisTransactionScopeparameters. The method signatures changed and callers compiled against the old signatures need rebuilding. - JDBC Reader predicates are now applied and requested columns are selected; workloads that previously relied on the ignored options may return fewer rows or columns.
Runtime and support scope
The build baseline is JDK 17, Scala 2.12.18 and Apache Spark 3.5.8. Target support remains limited to the documented and tested configurations. Structured Streaming, Kyuubi submission and unverified Iceberg REST or storage combinations remain Experimental. Doris 2PC does not provide cross-run exactly-once delivery. The Doris 2PC integration tests used Scala 2.12.17 in the test JVM; the candidate JAR Submitter tests did not directly exercise 2PC.
Security and SBOM limitations
The included SBOM does not enumerate all components shaded into Doris Spark Connector 25.2.0, including Netty 4.1.110.Final and Jackson 2.13.5. The candidate OSV scan used this incomplete SBOM, so its no-findings result does not cover the entire JAR. Additional version-based advisory matches for upstream-declared components have not all been assessed for presence or applicability.
Netty 4.1.110.Final is in the affected range of CVE-2025-24970, which involves SslHandler with a native SSLEngine receiving a specially crafted TLS packet. Static review identified a conditional path to TLS, but exploitability in Pista has not been established. Evaluate this limitation and the trust boundary of Doris endpoints before deployment.
Distribution and verification
The GitHub distribution consists of the Spark-submit JAR, source archive, runtime CycloneDX SBOM, release manifest and SHA-256 checksums. Verify the files against the manifest and checksums before use. Maven Central and PyPI artifacts are outside this release.
Full change comparison: v2.5.0...v2.6.0
License
Pista is licensed under Apache License 2.0. The distribution includes third-party components under their own terms, including MySQL Connector/J under GPLv2 with the Universal FOSS Exception and the Public Suffix List under MPL 2.0. See the included licenses, NOTICE, third-party attribution and SBOM.