Resolve every Java-binding URL through the vortex-cloud registry, adding hf://

[robert3005]

Rationale for this change

Companion to #9265, completing hf:// coverage across the bindings. The Java binding built stores through its own scheme dispatch — bespoke S3, Azure and GCS builders plus an authority-keyed cache — and keyed every read by the full URL path. That assumption held for the schemes it served, but not for hf://: a Hugging Face store is rooted at a repository and revision, which occupy path segments, so a full-URL-path key would send the repository name to the Hub as part of the file path. And the authority-keyed cache cannot serve hf:// at all, since every Hub repository shares the datasets authority.

Rather than special-casing hf://, the whole dispatch is replaced with vortex_cloud::Registry — the same resolution the Python and DuckDB bindings use.

• Part of Vortex should support reading from the Hugging Face Datasets API #5379

What changes are included in this PR?

• vortex-cloud: Registry::with_vars (the previously test-only fixed-variable constructor) is now public, so bindings whose callers pass per-request configuration can layer it over the process environment.
• vortex-jni: make_object_store resolves through a per-property-set Registry and reports the path of the URL within the store; every caller (metadata reads, listFiles, deletes, globbed data sources, the writer) keys by that instead of deriving a path from the URL. The bespoke S3/Azure/GCS builders and the authority-keyed store cache are gone.
• Caller properties are object_store configuration keys already (aws_access_key_id, aws_endpoint, … — exactly what HadoopUtils/VortexS3Properties emit), so they pass straight through parse_url_opts. Distinct property sets get distinct registries, since a store built with one caller's credentials must not serve another's requests.
• The previously hardcoded S3 settings (generic endpoint, path-style, allow_http) and the 120s Azure timeout survive as defaults that both the environment and properties override. A default yields to any spelling of its key — endpoint and aws_endpoint are one configuration, and if both reached the store builder, whichever iterates last would win — so each default lists the spellings that suppress it, and a test pins that a suppressed default is fully absent.
• The crate's opendal feature flag is removed entirely: vortex-jni is an unpublished cdylib built exactly one way, no Rust consumer exists to opt out, and CI never exercised the off-combo, so the flag only added untested cfg branches. vortex-cloud/opendal is now an unconditional dependency and the shipped library serves cos:///oss:///goosefs:// out of the box. Those schemes keep a properties-native branch (their property names — secret_id, … — are the services' own, not environment names) with the authority-keyed cache that is sound for them.
• Unit tests pin: deep-mounted path reporting, cache isolation across stores sharing an authority, default/property layering with alias suppression, and per-property-set store isolation.

Verification

• cargo test -p vortex-jni --lib — 7 passed; clippy clean; cargo test -p vortex-cloud --all-features — 71 passed.
• ./gradlew :vortex-jni:test — BUILD SUCCESSFUL, 32 JVM tests against the rebuilt native library.
• Live Hub read through the Java API: DataSource.open(session, "hf://datasets/danking00/statpopgen-benchmark/…gnomad….vortex") returned the exact row count (10,000) and schema (1,190 columns) in ~6s.
• VortexDataSourceS3MockTest needs Docker (testcontainers), unavailable in this sandbox — left to CI, which is why this stays draft.

Limitations mirror the other bindings: the Hub serves no listing, so wildcard globs and listFiles fail with the store's listing error, while exact file paths resolve with head alone; writes and deletes are rejected by the Hub.

What APIs are changed? Are there any user-facing changes?

Java reads (DataSource.open, NativeFiles) now accept hf://datasets/<owner>/<name>[@<revision>]/<path> URLs for exact file paths, and the shipped library serves the OpenDAL-backed schemes unconditionally (the opendal feature flag no longer exists). vortex_cloud::Registry::with_vars is new public API. Two minor behavior shifts on the Rust side: environment variables can now override the previously hardcoded S3 defaults (properties still override everything), and memory:// URLs now resolve instead of erroring. No Java API changes.

---

🤖 Generated with Claude Code

https://claude.ai/code/session_01Lc5zw7Le2T3pakDEUdKTYd

[codspeed-hq]

Merging this PR will degrade performance by 11.21%

❌ 1 regressed benchmark
✅ 1933 untouched benchmarks
⏩ 51 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Simulation	compress_fsst[(1000, 64, 8)]	1 ms	1.2 ms	-11.21%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing claude/vortex-jni-hf (8043b00) with develop (39fde8c)}

Footnotes

1. 51 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩