Skip to content

chore: upgrade Scala Native to 0.5.12#867

Open
He-Pin wants to merge 5 commits into
databricks:masterfrom
He-Pin:chore/scala-native-0.5.12
Open

chore: upgrade Scala Native to 0.5.12#867
He-Pin wants to merge 5 commits into
databricks:masterfrom
He-Pin:chore/scala-native-0.5.12

Conversation

@He-Pin
Copy link
Copy Markdown
Contributor

@He-Pin He-Pin commented May 23, 2026

Motivation:
Upgrade the Scala Native baseline to 0.5.12 so follow-up native profiling, async-profiler work, and compatibility work use the latest released 0.5.x toolchain.

Key Design Decision:
Keep this PR focused on the toolchain migration and API cleanup enabled by that migration. The root sbt files were checked: build.sbt is JVM-only for this repository and project/plugins.sbt only adds JMH, so there is no Scala Native sbt plugin version to update here.

Modification:

  • Bump scalaNativeVersion from 0.5.11 to 0.5.12 in build.mill.
  • Use java.util.HexFormat in the Scala Native platform hash implementation now that Scala Native 0.5.12 provides it.
  • Remove the native-only hand-written byte-to-hex helper.
  • Allow fork PR CI to use Google Maven Central mirror fallback when no Databricks JFrog OIDC token is available. This is needed because newly released public artifacts, such as Scala Native 0.5.12 modules, may not be mirrored in JFrog yet, and it avoids the Maven Central repo1 TLS failure seen in CI by switching both coursier and the Mill launcher to maven-central.storage-download.googleapis.com.

Benchmark Results:
Not included for this dependency upgrade. This PR establishes the 0.5.12 baseline; follow-up optimization PRs should benchmark their own performance deltas on top of this version.

Analysis:
Scala Native 0.5.12 includes release GC reliability fixes, codegen/toolchain improvements, and Java 17 HexFormat support. The HexFormat change makes the native hash path match the JVM implementation more closely and reduces platform-specific code without changing Jsonnet semantics. The CI repository fallback keeps authenticated Databricks JFrog usage for trusted runs. For fork PRs without OIDC credentials, it uses the Google Maven Central mirror fallback and forces the Mill JVM launcher (MILL_VERSION=<version>-jvm) because the native Mill launcher failed Java TLS handshakes to HTTPS Maven Central mirrors before reaching project dependency resolution. In dummy-token mode it also injects --no-server.

References:

Result:

  • ./mill 'sjsonnet.jvm[3.3.7].test' passed.
  • ./mill 'sjsonnet.native[3.3.7].nativeLink' passed.
  • ./mill __.reformat && ./mill --no-server --ticker false --color false -j 1 __.test passed 444/444 before the HexFormat cleanup.
  • ./mill __.reformat && ./mill 'sjsonnet.native[3.3.7].test' passed 457/457 after the HexFormat cleanup.
  • bash -n .github/scripts/fix-build-config.sh passed.
  • The dummy-token CI path was smoke-tested locally and writes an sbt repository config with local + google-maven-central: https://maven-central.storage-download.googleapis.com/maven2/, exports COURSIER_REPOSITORIES=https://maven-central.storage-download.googleapis.com/maven2/, rewrites the Mill launcher URL to the same mirror, sets MILL_VERSION=1.1.5-jvm, injects --no-server, and appends Java TLS/IPv4 options for public mirror access.

@He-Pin He-Pin marked this pull request as ready for review May 23, 2026 10:38
@He-Pin He-Pin force-pushed the chore/scala-native-0.5.12 branch from 95b792a to 0a76b0c Compare May 23, 2026 10:46
@He-Pin He-Pin marked this pull request as draft May 23, 2026 10:46
@He-Pin He-Pin marked this pull request as ready for review May 23, 2026 10:47
@He-Pin He-Pin marked this pull request as draft May 23, 2026 11:04
@He-Pin He-Pin force-pushed the chore/scala-native-0.5.12 branch 7 times, most recently from 9b6b86f to 619d008 Compare May 23, 2026 12:12
Motivation:
Adopt the latest Scala Native 0.5.x release so future native profiling and benchmarking use the current toolchain baseline.

Modification:
Bump the sjsonnet native cross module from Scala Native 0.5.11 to 0.5.12, use java.util.HexFormat for native hash rendering now that Scala Native provides it, and allow fork PR CI to use Maven Central when no Databricks JFrog token is available.

Result:
JVM tests, native tests, native linking, formatting, and the full test suite pass locally. The kube-prometheus native output remains byte-identical against a 0.5.11 baseline.

References:
https://github.com/scala-native/scala-native/releases/tag/v0.5.12
@He-Pin He-Pin force-pushed the chore/scala-native-0.5.12 branch from 619d008 to 68bde92 Compare May 23, 2026 12:17
He-Pin added 4 commits May 24, 2026 01:51
Motivation:
Fork PR CI cannot use Databricks JFrog OIDC credentials, so databricks#867 needs a public Maven fallback for new Scala Native 0.5.12 artifacts. The previous fallback also forced the Mill JVM launcher, which bypassed restored Mill native-launcher caches and failed early while downloading the launcher from the Google Maven mirror.

Modification:
Use the canonical Maven Central endpoint for dummy-token fork builds and stop exporting MILL_VERSION with a -jvm suffix. Keep the no-server Mill wrapper patch and JVM TLS options so fork builds can reuse restored launcher caches while resolving new public dependencies from Maven Central.

Result:
The CI fallback targets the actual fork-build failure path without changing authenticated Databricks/JFrog builds or project source code.
Motivation:
Fork PR CI still failed before project compilation because the native Mill launcher path resolved mill-runner-daemon through Maven Central, and the hosted runner hit a Java TLS handshake failure.

Modification:
Keep the dummy-token Maven Central fallback, but explicitly use the JVM Mill launcher and patch the Mill bootstrap curl with retries. This avoids the runner-daemon resolution path and makes transient Maven Central connection failures less likely to fail the job immediately.

Result:
Fork PR builds can bootstrap Mill from the public mill-dist artifact path while authenticated Databricks/JFrog builds remain unchanged.
Motivation:
GitHub-hosted fork PR runners consistently fail TLS handshakes against repo.maven.apache.org and the storage-download Maven mirror before databricks#867 reaches project compilation.

Modification:
Point the dummy-token Maven fallback at the standard Google Maven Central storage bucket while retaining the JVM Mill launcher, no-server wrapper patch, and retry-hardened bootstrap curl.

Result:
The fork-only CI path avoids the failing Maven Central endpoints without changing authenticated Databricks/JFrog builds or sjsonnet source code.
Motivation:
The previous databricks#867 validation run reached green results for Graal, JS, WASM, and Native, but the JVM job failed during actions/checkout before project code or build scripts ran. This account cannot rerun failed jobs on databricks/sjsonnet directly.

Modification:
Add an empty commit to trigger a fresh pull_request validation run without changing repository contents.

Result:
CI can re-run from a clean checkout attempt while preserving the actual Scala Native 0.5.12 migration changes.
@He-Pin He-Pin marked this pull request as ready for review May 23, 2026 19:15
@He-Pin He-Pin closed this May 23, 2026
@He-Pin He-Pin reopened this May 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant