Skip to content

fix: Align stdlib 1:1 edge semantics#800

Open
He-Pin wants to merge 1 commit intodatabricks:masterfrom
He-Pin:fix-stdlib-1to1-audit
Open

fix: Align stdlib 1:1 edge semantics#800
He-Pin wants to merge 1 commit intodatabricks:masterfrom
He-Pin:fix-stdlib-1to1-audit

Conversation

@He-Pin
Copy link
Copy Markdown
Contributor

@He-Pin He-Pin commented Apr 26, 2026

Motivation

This follows a 1:1 stdlib audit against local google/jsonnet stdlib/std.jsonnet and Jsonnet 0.22.0. Several native stdlib implementations in sjsonnet diverged on edge semantics while still needing to stay efficient on hot paths.

Modification

  • Add official-style hidden comparison helpers and shared comparison logic for Jsonnet stdlib semantics.
  • Align minArray/maxArray default sentinels, explicit keyF=false / onEmpty=false, and non-comparable value errors.
  • Align array/string helpers including mapWithIndex, flattenArrays, flattenDeepArray, repeat, removeAt, isEmpty, string escaping helpers, and resolvePath.
  • Preserve explicit keyF=false behavior in set helpers without treating it as the default sentinel.
  • Add focused direction tests covering the audited compatibility cases and update the existing isEmpty golden.

Result

  • git diff --check
  • ./mill --no-server 'sjsonnet.jvm[3.3.7].checkFormat'
  • ./mill --no-server 'sjsonnet.jvm[3.3.7].test' (141/141 passed)

Follow-up Issues

Not mixed into this PR because they are higher-risk or need compatibility policy confirmation:

Base64 non-ASCII and substr fractional semantics remain tracked separately in #793 and #795.

Motivation:
Official Jsonnet std.jsonnet defines several edge-case stdlib behaviors that sjsonnet native implementations diverged from. The audit should keep hot paths native while making observable semantics match Jsonnet 0.22.0 where the behavior is clear and low risk.

Modification:
Add official-style comparison helpers, fix minArray/maxArray default sentinels and comparison errors, align mapWithIndex/flatten/removeAt/repeat/string escaping/isEmpty/resolvePath behavior, and preserve explicit keyF=false semantics in set helpers. Add directional compatibility coverage for the audited cases and update the existing isEmpty golden.

Result:
The new direction tests, full JVM test suite, format check, and whitespace check pass against this branch.
@He-Pin
Copy link
Copy Markdown
Contributor Author

He-Pin commented Apr 26, 2026

Additional audit update after adding source-built jrsonnet to the comparison matrix.

Environment:

  • official C++ Jsonnet: Jsonnet commandline interpreter v0.22.0
  • go-jsonnet: Jsonnet commandline interpreter (Go implementation) v0.22.0
  • jrsonnet: source-built jrsonnet 0.5.0-pre98, commit 80cd36abd868507312e2cc2c78cb0f55a684c620
  • sjsonnet: this PR's worktree assembly

Summary:

  • The cases fixed in this PR still look correct against official Jsonnet: removeAt fractional/negative indexes, minArray/maxArray non-comparable values, isEmpty over object/function, non-string escape helpers, keyF=false empty/singleton behavior, flatten*, repeat, mapWithIndex on strings, and filterMap laziness.
  • go-jsonnet and jrsonnet are useful cross-checks, but both diverge from official on several stdlib edge cases. We should not use either as the sole compatibility oracle.
  • Merge/stacking matters: this PR's worktree does not include all parallel stdlib fixes, so a final integration branch should include fix: Ignore hidden fields in std.mapWithKey #792 and fix: Align stdlib edge cases with Jsonnet #794 or rebase after them.

Notable remaining/parallel items:

item official behavior go-jsonnet jrsonnet sjsonnet PR #800 coverage
std.mapWithKey hidden fields ignores hidden fields matches official matches official still maps hidden fields in this worktree #792
std.substr("hello", 1.2, 2) "el" error error error #795
std.makeArray(2.5, function(i) i) [0, 1] error error error #796
std.slice([1,2,3], 1.2, 3, 1) error [2,3] error [2,3] #797
std.setUnion("ab", "bc") "[\"a\", \"b\"]c" matches official error ["a","b","c"] #798
std.parseInt("+1") error 1 error 1 covered by #794 parse rewrite
std.manifestToml({a:{b:"c"}}) "\n\n[a]\n b = \"c\"" matches official "[a]\n b = \"c\"" "[a]\nb = \"c\"" #799
std.base64("é") "6Q==" "w6k=" "w6k=" "w6k=" #793 policy/tracking

This PR can remain focused, but before final merge we should avoid a partial stdlib state where #800 lands without the earlier hidden-field/parse fixes or without clear follow-up issues for the deliberate policy differences.

@He-Pin
Copy link
Copy Markdown
Contributor Author

He-Pin commented Apr 26, 2026

Follow-up from the expanded stdlib audit pass (100 additional function-family cases across official C++ Jsonnet 0.22.0, go-jsonnet 0.22.0, source-built jrsonnet 0.5.0-pre98, and this PR's sjsonnet assembly):

New compatibility issues opened from that pass:

Already-covered differences observed in the same pass:

The broader pass did not change the assessment of this PR's core fixes: its included cases still match official behavior. The main integration concern remains merge/stack ordering so that #792/#794 and the follow-up issues are not lost from the overall stdlib alignment effort.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant