Skip to content

[GLUTEN-12387][VL] Re-enable large deletion vector tests - #12866

Closed
felipepessoto wants to merge 1 commit into
apache:mainfrom
felipepessoto:remove-dv-force-fail-workaround
Closed

[GLUTEN-12387][VL] Re-enable large deletion vector tests#12866
felipepessoto wants to merge 1 commit into
apache:mainfrom
felipepessoto:remove-dv-force-fail-workaround

Conversation

@felipepessoto

Copy link
Copy Markdown
Contributor

What changes are proposed in this pull request?

The deletion-vector planning, filtering, and file-I/O improvements tracked by #12399 are now merged. Remove the workaround added for #12387 so Delta's two 2-billion-row DeletionVectorsSuite tests run normally again:

  • stop injecting immediate failures into both tests in setup-delta.sh
  • remove both tests from the known-failures baseline so CI enforces their results
  • update the stale workflow memory comment

Closes #12387.

How was this patch tested?

  • bash -n .github/workflows/util/delta-spark-ut/setup-delta.sh
  • git diff --check
  • The Delta Spark UT workflow will run the re-enabled tests on this PR.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: GitHub Copilot CLI 1.0.80

Remove the CI force-failure injection and expected-failure entries for the two 2-billion-row DeletionVectorsSuite tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings August 23, 2026 08:15
@github-actions github-actions Bot added the INFRA label Aug 23, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Re-enables Delta’s two large deletion-vector tests after the underlying fixes were merged.

Changes:

  • Removes forced test failures.
  • Removes obsolete known-failure entries.
  • Updates workflow memory documentation.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
.github/workflows/util/delta-spark-ut/setup-delta.sh Re-enables normal test execution.
.github/workflows/util/delta-spark-ut/known-failures.txt Removes the two obsolete failure entries.
.github/workflows/delta_spark_ut.yml Updates memory documentation.
Suppressed comments (1)

.github/workflows/util/delta-spark-ut/setup-delta.sh:175

  • Removing this block also removes its INJECTED == 2 assertion. DELTA_REF is workflow-dispatchable (see setup-delta.sh:66), and the aggregate gate only evaluates tests that emit reports, so a future Delta ref that renames or removes either test will silently pass without enforcing these regressions. Keep a read-only presence check for both test-name strings while dropping only the fail injection.
echo "::group::Disabling Delta scalastyle HeaderMatchesChecker"

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@felipepessoto
felipepessoto marked this pull request as draft August 23, 2026 10:33
@felipepessoto felipepessoto reopened this Aug 24, 2026
@felipepessoto

Copy link
Copy Markdown
Contributor Author

Shard 2 is hanging for 4h30m. Closing the PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[VL] Native memory OOMs the test/executor JVM when reading large (billions-of-rows) Delta tables with deletion vectors

2 participants