You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The targets are upstream-owned and remain present in current facebookincubator/velox main:
The Spark targets originate from commits 406f60a8e7f288057656461d964730d007c64d44 and 5c91e834eabdd86408f7821e1dffae284c6b78d6; their common cuDF-test registration was later refactored upstream.
The Iceberg targets originate from commit eedf68b315e284028faa19c273761f22b91eba88.
2026-08-04: PR source commit 0e0227837f00ae53651d91b1ce8ae1ce33a207a7 added the three bridge declarations and calls to CudfSplitReaderHelpers.cpp.
The strong provider was added in paired Spark-Gluten commit 8d1f589618fcbd84ba8513577c7046669ca9162f, in cpp/velox/jni/JniFileSystem.cc, and is compiled into libgluten rather than standalone Velox tests.
CudfSplitReaderHelpers.cpp is compiled into a Velox connector library that is linked by both production integrations and standalone Velox tests. The new CRT S3 path declares the three functions with C linkage and calls them directly. Those references are unconditional at the object-file level: even if runtime configuration would make glutenCrtS3RangeReaderAvailable() return false and avoid CRT S3 reads, the executable linker must still find definitions for every referenced symbol.
The only production provider lives in Spark-Gluten's JniFileSystem.cpp and enters the final libgluten binary. Standalone Velox test executables intentionally do not link libgluten, so pulling CudfSplitReaderHelpers.cpp from the connector archive leaves the three bridge symbols unresolved.
A fallback in one test Main.cpp does not establish a common test link boundary. The affected targets use different entry points:
SparkSQL tests use tests/sparksql/Main.cpp.
Iceberg read uses tests/iceberg/Main.cpp.
Iceberg deletion-vector supplies no Main.cpp and relies on gtest_main.
Attaching definitions to a particular main therefore fixes only targets that compile that exact file. The support must be added at the common velox_add_cudf_test construction point so every standalone cuDF executable receives the same fallback. Weak C-linkage definitions preserve the boundary: standalone tests resolve locally, while a strong test fake or Spark-Gluten's strong provider remains authoritative when present.
Impact
The four named standalone cuDF test executables cannot link unless they happen to receive an out-of-band provider from Spark-Gluten. This breaks Velox's standalone test boundary and prevents complete builds of the registered cuDF test inventory, even for tests that never exercise CRT S3 at runtime.
Proposed correction
Add one test-only standalone support source through the existing velox_add_cudf_test helper so every cuDF test executable receives weak C-linkage fallbacks:
availability returns false;
object size and range read return 0 and are unreachable through the normal unavailable path.
Strong test fakes override the weak definitions. Production does not compile the test-support source, so Spark-Gluten's strong definitions remain authoritative. Do not disable inherited targets and do not add Gluten as a standalone-test dependency.
A local candidate commit, ee9715f8ab34b36821ea86074639b75efed2f670, demonstrates this test-only boundary correction but is not published in this repository.
Acceptance criteria
All four named targets link.
Every registered cuDF test executable built through velox_add_cudf_test receives the standalone support.
velox_cudf_executor_prefetch_test continues linking with its strong bridge fake.
A production Gluten build still exports and uses the strong JniFileSystem definitions.
Run the four tests where supported; clearly separate compile/link proof from runtime results.
Problem
Four inherited standalone Velox cuDF targets fail to link on:
glutenCrtS3RangeReaderAvailableglutenCrtS3ObjectSizeglutenCrtS3ReadRangesAffected targets:
velox_cudf_iceberg_deletion_vector_testvelox_cudf_spark_filter_project_testvelox_cudf_spark_aggregates_testvelox_cudf_iceberg_read_testCategory
Spark-Flux integration-boundary regression affecting inherited standalone tests
History
The targets are upstream-owned and remain present in current
facebookincubator/veloxmain:406f60a8e7f288057656461d964730d007c64d44and5c91e834eabdd86408f7821e1dffae284c6b78d6; their common cuDF-test registration was later refactored upstream.eedf68b315e284028faa19c273761f22b91eba88.0e0227837f00ae53651d91b1ce8ae1ce33a207a7added the three bridge declarations and calls toCudfSplitReaderHelpers.cpp.0fb10f4ee2f9105e0d24e07d10f39a2e7181caf2.8d1f589618fcbd84ba8513577c7046669ca9162f, incpp/velox/jni/JniFileSystem.cc, and is compiled intolibglutenrather than standalone Velox tests.Root cause
CudfSplitReaderHelpers.cppis compiled into a Velox connector library that is linked by both production integrations and standalone Velox tests. The new CRT S3 path declares the three functions with C linkage and calls them directly. Those references are unconditional at the object-file level: even if runtime configuration would makeglutenCrtS3RangeReaderAvailable()returnfalseand avoid CRT S3 reads, the executable linker must still find definitions for every referenced symbol.The only production provider lives in Spark-Gluten's
JniFileSystem.cppand enters the finallibglutenbinary. Standalone Velox test executables intentionally do not linklibgluten, so pullingCudfSplitReaderHelpers.cppfrom the connector archive leaves the three bridge symbols unresolved.A fallback in one test
Main.cppdoes not establish a common test link boundary. The affected targets use different entry points:tests/sparksql/Main.cpp.tests/iceberg/Main.cpp.Main.cppand relies ongtest_main.Attaching definitions to a particular main therefore fixes only targets that compile that exact file. The support must be added at the common
velox_add_cudf_testconstruction point so every standalone cuDF executable receives the same fallback. Weak C-linkage definitions preserve the boundary: standalone tests resolve locally, while a strong test fake or Spark-Gluten's strong provider remains authoritative when present.Impact
The four named standalone cuDF test executables cannot link unless they happen to receive an out-of-band provider from Spark-Gluten. This breaks Velox's standalone test boundary and prevents complete builds of the registered cuDF test inventory, even for tests that never exercise CRT S3 at runtime.
Proposed correction
Add one test-only standalone support source through the existing
velox_add_cudf_testhelper so every cuDF test executable receives weak C-linkage fallbacks:false;0and are unreachable through the normal unavailable path.Strong test fakes override the weak definitions. Production does not compile the test-support source, so Spark-Gluten's strong definitions remain authoritative. Do not disable inherited targets and do not add Gluten as a standalone-test dependency.
A local candidate commit,
ee9715f8ab34b36821ea86074639b75efed2f670, demonstrates this test-only boundary correction but is not published in this repository.Acceptance criteria
velox_add_cudf_testreceives the standalone support.velox_cudf_executor_prefetch_testcontinues linking with its strong bridge fake.JniFileSystemdefinitions.