Skip to content

Conversation

vtjnash
Copy link
Member

@vtjnash vtjnash commented Sep 30, 2025

This became unsound to use even though it was preserved "to avoid breakage" in v1.13, since continuing to use it would give incorrect hash results, which could result in corrupt dictionaries and unsound programs.

Since #59691, these broken hash methods can now simply be deleted as they no longer provide any value.

It is hard to say whether this is technically breaking or not as a change. It causes packages to go from giving subtly wrong answers (the worst kind of wrong) to crashing in v1.13, until the offending incorrect methods are deleted.

n.b. this is expected to break several packages (notably among them, JuliaInterpreter, which has several tests specifically just for the existence of this global), which will require some ecosystem updates. The update should just be to delete the offending method (they is now redundant, undesirable, and have been giving unsafely buggy answers), but we need to do some work to identify those places and release new versions.

Copy link
Member

@oscardssmith oscardssmith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to sort out #59691 (comment) before this merges and makes the likely revert more complicated.

@vtjnash vtjnash added this to the 1.13 milestone Oct 1, 2025
@adienes adienes added deprecation This change introduces or involves a deprecation hashing labels Oct 1, 2025
This became unsound to use even though it was preserved "to avoid
breakage" in v1.13, since continuing to use it would give incorrect hash
results, which could result in corrupt dictionaries.

Since #59691, these broken `hash` methods can now simply be deleted as
they no longer provide any value.

It is hard to say whether this is technically breaking or not as a
change. It causes packages to go from giving subtly wrong answers (the
worst kind of wrong) to crashing in v1.13, until the offending incorrect
methods are deleted.
@vtjnash vtjnash force-pushed the jn/drop-memhash-global branch from e0ce221 to 7033f56 Compare October 3, 2025 15:30
@vtjnash vtjnash requested a review from oscardssmith October 3, 2025 15:30
@vtjnash
Copy link
Member Author

vtjnash commented Oct 3, 2025

The set of packages that might be broken by this PR (well, that are already broken, but will be more visibly failing) may include some of these:
@nanosoldier runtests(["CGcoefficient", "Walsh", "Spinnaker", "Pardiso", "ArnoldiMethodTransformations", "GeoIP", "SmallCombinatorics", "Hamburg", "LongDatasetSort", "CxxWrap", "CartesianJoin", "Pingouin", "DLMReader", "StatisticalGraphics", "BytePairEncoding", "GLM", "AMGCLWrap", "GAP", "SteadyWaves", "DecomposingPolynomialSystems", "MAiNGO", "GenericCharacterTables", "ONSAS", "Globtim"])
https://juliahub.com/ui/Search?type=code&q=memhash
https://juliahub.com/ui/Packages/General/MurmurHash3#dependents

@KristofferC
Copy link
Member

Thee are also a bunch of string packages that use the old hash (StringViews, InlineStrings, etc) and is now inconsistent with the String hash.

@vtjnash
Copy link
Member Author

vtjnash commented Oct 3, 2025

Yes, those are in my search list too, but look currently blacklisted in PkgEval so it didn't get them listed here

@nanosoldier
Copy link
Collaborator

The package evaluation job you requested has completed - possible new issues were detected.
The full report is available.

Report summary

✖ Packages that failed

9 packages failed only on the current version.

  • Package fails to precompile: 4 packages
  • Illegal method overwrites during precompilation: 1 packages
  • Package tests unexpectedly errored: 4 packages

✔ Packages that passed tests

14 packages passed tests on the previous version too.

~ Packages that at least loaded

1 packages successfully loaded on the previous version too.

vtjnash added a commit to vtjnash/CSV.jl that referenced this pull request Oct 3, 2025
Remove hash method definition when Base.memhash is not available.
On Julia 1.13+, these AbstractString types will use the default
AbstractString hash implementation which is now efficient and
zero-copy based on codeunit/iterate.

For Julia <1.13, continue using the memhash-based implementation
for compatibility.

Related to JuliaLang/julia#59697

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vtjnash
Copy link
Member Author

vtjnash commented Oct 3, 2025

It appears that StringViews and InlineStrings were already fixed to use the private hash_bytes pointer-based interface. PRs submitted by Claude to all others that showed up on that list to fix. We should probably merge this now, so that those CI will actually work (since they relied on !isdefined(Base, :memhash) to detect when the hash should change)

vtjnash added a commit to vtjnash/GraphQLParser.jl that referenced this pull request Oct 3, 2025
Remove hash method definition when Base.memhash is not available.
On Julia 1.13+, these AbstractString types will use the default
AbstractString hash implementation which is now efficient and
zero-copy based on codeunit/iterate.

For Julia <1.13, continue using the memhash-based implementation
for compatibility.

Related to JuliaLang/julia#59697

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
vtjnash added a commit to vtjnash/InMemoryDatasets.jl that referenced this pull request Oct 3, 2025
Remove hash method definition when Base.memhash is not available.
On Julia 1.13+, these AbstractString types will use the default
AbstractString hash implementation which is now efficient and
zero-copy based on codeunit/iterate.

For Julia <1.13, continue using the memhash-based implementation
for compatibility.

Related to JuliaLang/julia#59697

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
vtjnash added a commit to vtjnash/JSON3.jl that referenced this pull request Oct 3, 2025
Remove hash method definition when Base.memhash is not available.
On Julia 1.13+, these AbstractString types will use the default
AbstractString hash implementation which is now efficient and
zero-copy based on codeunit/iterate.

For Julia <1.13, continue using the memhash-based implementation
for compatibility.

Related to JuliaLang/julia#59697

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
vtjnash added a commit to vtjnash/StrFs.jl that referenced this pull request Oct 3, 2025
Remove hash method definition when Base.memhash is not available.
On Julia 1.13+, these AbstractString types will use the default
AbstractString hash implementation which is now efficient and
zero-copy based on codeunit/iterate.

For Julia <1.13, continue using the memhash-based implementation
for compatibility.

Related to JuliaLang/julia#59697

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
vtjnash added a commit to vtjnash/StrFs.jl that referenced this pull request Oct 3, 2025
Remove hash method definition when Base.memhash is not available.
On Julia 1.13+, these AbstractString types will use the default
AbstractString hash implementation which is now efficient and
zero-copy based on codeunit/iterate.

For Julia <1.13, continue using the memhash-based implementation
for compatibility.

Related to JuliaLang/julia#59697

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
vtjnash added a commit to vtjnash/WeakRefStrings.jl that referenced this pull request Oct 3, 2025
Remove hash method definition when Base.memhash is not available.
On Julia 1.13+, these AbstractString types will use the default
AbstractString hash implementation which is now efficient and
zero-copy based on codeunit/iterate.

For Julia <1.13, continue using the memhash-based implementation
for compatibility.

Related to JuliaLang/julia#59697

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
vtjnash added a commit to vtjnash/ShortStrings.jl that referenced this pull request Oct 3, 2025
Add isdefined check for Base.memhash_seed to support Julia 1.13+
where Base.memhash_seed was removed. When memhash_seed is not
available, use MurmurHash3 directly without the seed offset.

Related to JuliaLang/julia#59697

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
vtjnash added a commit to vtjnash/StrRegex.jl that referenced this pull request Oct 3, 2025
MurmurHash3 is not used directly in StrRegex, only transitively through
StrBase. Remove it from the Project.toml dependencies.

Related to JuliaLang/julia#59697

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
vtjnash added a commit to vtjnash/Strs.jl that referenced this pull request Oct 3, 2025
MurmurHash3 is not used directly in Strs, only transitively through
StrBase, as everything related to it was removed in
77c83fc.

Related to JuliaLang/julia#59697

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vtjnash vtjnash merged commit 18b5ff6 into master Oct 4, 2025
5 of 8 checks passed
@vtjnash vtjnash deleted the jn/drop-memhash-global branch October 4, 2025 01:01
sl-solution pushed a commit to sl-solution/InMemoryDatasets.jl that referenced this pull request Oct 6, 2025
Remove hash method definition when Base.memhash is not available.
On Julia 1.13+, these AbstractString types will use the default
AbstractString hash implementation which is now efficient and
zero-copy based on codeunit/iterate.

For Julia <1.13, continue using the memhash-based implementation
for compatibility.

Related to JuliaLang/julia#59697

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deprecation This change introduces or involves a deprecation hashing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants