Remove all @pure, does not actually fix #115.
#116
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## master #116 +/- ##
==========================================
- Coverage 84.26% 82.12% -2.15%
==========================================
Files 8 8
Lines 1506 1415 -91
==========================================
- Hits 1269 1162 -107
- Misses 237 253 +16
Continue to review full report at Codecov.
|
DilumAluthge
approved these changes
Feb 3, 2021
Tokazama
reviewed
Feb 3, 2021
|
Looking over these, I wonder if the issue was the use of |
|
I'm getting the problem again, despite no instances of julia> using VectorizationBase
[ Info: Precompiling VectorizationBase [3d5dd08c-fd9d-11e8-17fa-ed2836048c2f]
julia> !success(run(`grep -nr "pure" $(dirname(pathof(VectorizationBase)))`, wait=false))
true
julia> VectorizationBase.has_feature(::Val{:x86_64_avx512f}) = VectorizationBase.False()
julia> VectorizationBase.has_feature(::Val{:x86_64_avx2}) = VectorizationBase.False()
julia> using VectorizedRNG
[ Info: Precompiling VectorizedRNG [33b4df10-0173-11e9-2a0c-851a7edac40e]
julia> @code_typed VectorizedRNG.setrand64counter!(local_rng(), 0x01) # we have a constant offset of 130
CodeInfo(
1 ─ %1 = Base.getfield(rng, :ptr)::Ptr{UInt64}
│ %2 = Base.bitcast(Ptr{UInt8}, %1)::Ptr{UInt8}
│ %3 = VectorizationBase.llvmcall((" \n\n define void @entry(i64, i8, i64) alwaysinline {\n top:\n %ptr.0 = inttoptr i64 %0 to i8*\n%ptr.1 = getelementptr inbounds i8, i8* %ptr.0, i64 %2\nstore i8 %1, i8* %ptr.1, align 1\nret void\n }\n", "entry"), VectorizationBase.Cvoid, Tuple{Ptr{UInt8}, UInt8, Int64}, %2, v, 130)::Nothing
└── return %3
) => Nothing
julia> @code_llvm VectorizedRNG.setrand64counter!(local_rng(), 0x01) # we have a constant offset of 514
; @ /home/chriselrod/.julia/dev/VectorizedRNG/src/xoshiro.jl:97 within `setrand64counter!'
define void @"julia_setrand64counter!_1348"([1 x i64]* nocapture nonnull readonly align 8 dereferenceable(8) %0, i8 zeroext %1) {
top:
; @ /home/chriselrod/.julia/dev/VectorizedRNG/src/xoshiro.jl:98 within `setrand64counter!'
; ┌ @ essentials.jl:402 within `unsafe_convert'
; │┌ @ pointer.jl:30 within `convert'
%2 = bitcast [1 x i64]* %0 to i8**
%3 = load i8*, i8** %2, align 8
; └└
; ┌ @ /home/chriselrod/.julia/dev/VectorizationBase/src/llvm_intrin/memory_addr.jl:782 within `vstore!' @ /home/chriselrod/.julia/dev/VectorizationBase/src/llvm_intrin/memory_addr.jl:638
; │┌ @ /home/chriselrod/.julia/dev/VectorizationBase/src/llvm_intrin/memory_addr.jl:638 within `macro expansion'
%ptr.1.i = getelementptr inbounds i8, i8* %3, i64 514
store i8 %1, i8* %ptr.1.i, align 1
; └└
ret void
}
julia> using ArrayInterface
julia> !success(run(`grep -nr "pure" $(dirname(pathof(ArrayInterface)))`, wait=false))
true |
|
|
|
I can reproduce. julia> using VectorizationBase
julia> !success(run(`grep -nr "pure" $(dirname(pathof(VectorizationBase)))`, wait=false))
true
julia> VectorizationBase.has_feature(::Val{:x86_64_avx512f}) = VectorizationBase.False()
julia> VectorizationBase.has_feature(::Val{:x86_64_avx2}) = VectorizationBase.False()
julia> using VectorizedRNG
julia> @code_typed VectorizedRNG.setrand64counter!(local_rng(), 0x01)
CodeInfo(
1 ─ %1 = Base.getfield(rng, :ptr)::Ptr{UInt64}
│ %2 = Base.bitcast(Ptr{UInt8}, %1)::Ptr{UInt8}
│ %3 = VectorizationBase.llvmcall((" \n\n define void @entry(i64, i8, i64) alwaysinline {\n top:\n %ptr.0 = inttoptr i64 %0 to i8*\n%ptr.1 = getelementptr inbounds i8, i8* %ptr.0, i64 %2\nstore i8 %1, i8* %ptr.1, align 1\nret void\n }\n", "entry"), VectorizationBase.Cvoid, Tuple{Ptr{UInt8}, UInt8, Int64}, %2, v, 130)::Nothing
└── return %3
) => Nothing
julia> @code_llvm VectorizedRNG.setrand64counter!(local_rng(), 0x01)
; @ /Users/dilum/.julia/packages/VectorizedRNG/RVAIZ/src/xoshiro.jl:97 within `setrand64counter!'
define void @"julia_setrand64counter!_583"([1 x i64]* nocapture nonnull readonly align 8 dereferenceable(8) %0, i8 zeroext %1) {
top:
; @ /Users/dilum/.julia/packages/VectorizedRNG/RVAIZ/src/xoshiro.jl:98 within `setrand64counter!'
; ┌ @ essentials.jl:402 within `unsafe_convert'
; │┌ @ pointer.jl:30 within `convert'
%2 = bitcast [1 x i64]* %0 to i8**
%3 = load i8*, i8** %2, align 8
; └└
; ┌ @ /Users/dilum/.julia/packages/VectorizationBase/NMmk1/src/llvm_intrin/memory_addr.jl:782 within `vstore!' @ /Users/dilum/.julia/packages/VectorizationBase/NMmk1/src/llvm_intrin/memory_addr.jl:638
; │┌ @ /Users/dilum/.julia/packages/VectorizationBase/NMmk1/src/llvm_intrin/memory_addr.jl:638 within `macro expansion'
%ptr.1.i = getelementptr inbounds i8, i8* %3, i64 258
store i8 %1, i8* %ptr.1.i, align 1
; └└
ret void
}
julia> using ArrayInterface
julia> !success(run(`grep -nr "pure" $(dirname(pathof(ArrayInterface)))`, wait=false))
true |
|
I can repro even if I do the method redefinition after dilum:/Users/dilum/Downloads$ julia
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.7.0-DEV.400 (2021-01-28)
_/ |\__'_|_|_|\__'_| | Commit 9de107a63a (5 days old master)
|__/ |
julia> using VectorizationBase
julia> !success(run(`grep -nr "pure" $(dirname(pathof(VectorizationBase)))`, wait=false))
true
julia> using VectorizedRNG
julia> VectorizationBase.has_feature(::Val{:x86_64_avx512f}) = VectorizationBase.False()
julia> VectorizationBase.has_feature(::Val{:x86_64_avx2}) = VectorizationBase.False()
julia> @code_typed VectorizedRNG.setrand64counter!(local_rng(), 0x01)
CodeInfo(
1 ─ %1 = Base.getfield(rng, :ptr)::Ptr{UInt64}
│ %2 = Base.bitcast(Ptr{UInt8}, %1)::Ptr{UInt8}
│ %3 = VectorizationBase.llvmcall((" \n\n define void @entry(i64, i8, i64) alwaysinline {\n top:\n %ptr.0 = inttoptr i64 %0 to i8*\n%ptr.1 = getelementptr inbounds i8, i8* %ptr.0, i64 %2\nstore i8 %1, i8* %ptr.1, align 1\nret void\n }\n", "entry"), VectorizationBase.Cvoid, Tuple{Ptr{UInt8}, UInt8, Int64}, %2, v, 130)::Nothing
└── return %3
) => Nothing
julia> @code_llvm VectorizedRNG.setrand64counter!(local_rng(), 0x01)
; @ /Users/dilum/.julia/packages/VectorizedRNG/RVAIZ/src/xoshiro.jl:97 within `setrand64counter!'
define void @"julia_setrand64counter!_583"([1 x i64]* nocapture nonnull readonly align 8 dereferenceable(8) %0, i8 zeroext %1) {
top:
; @ /Users/dilum/.julia/packages/VectorizedRNG/RVAIZ/src/xoshiro.jl:98 within `setrand64counter!'
; ┌ @ essentials.jl:402 within `unsafe_convert'
; │┌ @ pointer.jl:30 within `convert'
%2 = bitcast [1 x i64]* %0 to i8**
%3 = load i8*, i8** %2, align 8
; └└
; ┌ @ /Users/dilum/.julia/packages/VectorizationBase/NMmk1/src/llvm_intrin/memory_addr.jl:782 within `vstore!' @ /Users/dilum/.julia/packages/VectorizationBase/NMmk1/src/llvm_intrin/memory_addr.jl:638
; │┌ @ /Users/dilum/.julia/packages/VectorizationBase/NMmk1/src/llvm_intrin/memory_addr.jl:638 within `macro expansion'
%ptr.1.i = getelementptr inbounds i8, i8* %3, i64 258
store i8 %1, i8* %ptr.1.i, align 1
; └└
ret void
}
julia> @code_llvm VectorizedRNG.setrand64counter!(local_rng(), 0x01)
; @ /Users/dilum/.julia/packages/VectorizedRNG/RVAIZ/src/xoshiro.jl:97 within `setrand64counter!'
define void @"julia_setrand64counter!_601"([1 x i64]* nocapture nonnull readonly align 8 dereferenceable(8) %0, i8 zeroext %1) {
top:
; @ /Users/dilum/.julia/packages/VectorizedRNG/RVAIZ/src/xoshiro.jl:98 within `setrand64counter!'
; ┌ @ essentials.jl:402 within `unsafe_convert'
; │┌ @ pointer.jl:30 within `convert'
%2 = bitcast [1 x i64]* %0 to i8**
%3 = load i8*, i8** %2, align 8
; └└
; ┌ @ /Users/dilum/.julia/packages/VectorizationBase/NMmk1/src/llvm_intrin/memory_addr.jl:782 within `vstore!' @ /Users/dilum/.julia/packages/VectorizationBase/NMmk1/src/llvm_intrin/memory_addr.jl:638
; │┌ @ /Users/dilum/.julia/packages/VectorizationBase/NMmk1/src/llvm_intrin/memory_addr.jl:638 within `macro expansion'
%ptr.1.i = getelementptr inbounds i8, i8* %3, i64 258
store i8 %1, i8* %ptr.1.i, align 1
; └└
ret void
}
julia> @code_llvm VectorizedRNG.setrand64counter!(local_rng(), 0x01)
; @ /Users/dilum/.julia/packages/VectorizedRNG/RVAIZ/src/xoshiro.jl:97 within `setrand64counter!'
define void @"julia_setrand64counter!_604"([1 x i64]* nocapture nonnull readonly align 8 dereferenceable(8) %0, i8 zeroext %1) {
top:
; @ /Users/dilum/.julia/packages/VectorizedRNG/RVAIZ/src/xoshiro.jl:98 within `setrand64counter!'
; ┌ @ essentials.jl:402 within `unsafe_convert'
; │┌ @ pointer.jl:30 within `convert'
%2 = bitcast [1 x i64]* %0 to i8**
%3 = load i8*, i8** %2, align 8
; └└
; ┌ @ /Users/dilum/.julia/packages/VectorizationBase/NMmk1/src/llvm_intrin/memory_addr.jl:782 within `vstore!' @ /Users/dilum/.julia/packages/VectorizationBase/NMmk1/src/llvm_intrin/memory_addr.jl:638
; │┌ @ /Users/dilum/.julia/packages/VectorizationBase/NMmk1/src/llvm_intrin/memory_addr.jl:638 within `macro expansion'
%ptr.1.i = getelementptr inbounds i8, i8* %3, i64 258
store i8 %1, i8* %ptr.1.i, align 1
; └└
ret void
}
julia> VectorizationBase.has_feature(::Val{:x86_64_avx2}) = VectorizationBase.False()
julia> @code_llvm VectorizedRNG.setrand64counter!(local_rng(), 0x01)
; @ /Users/dilum/.julia/packages/VectorizedRNG/RVAIZ/src/xoshiro.jl:97 within `setrand64counter!'
define void @"julia_setrand64counter!_613"([1 x i64]* nocapture nonnull readonly align 8 dereferenceable(8) %0, i8 zeroext %1) {
top:
; @ /Users/dilum/.julia/packages/VectorizedRNG/RVAIZ/src/xoshiro.jl:98 within `setrand64counter!'
; ┌ @ essentials.jl:402 within `unsafe_convert'
; │┌ @ pointer.jl:30 within `convert'
%2 = bitcast [1 x i64]* %0 to i8**
%3 = load i8*, i8** %2, align 8
; └└
; ┌ @ /Users/dilum/.julia/packages/VectorizationBase/NMmk1/src/llvm_intrin/memory_addr.jl:782 within `vstore!' @ /Users/dilum/.julia/packages/VectorizationBase/NMmk1/src/llvm_intrin/memory_addr.jl:638
; │┌ @ /Users/dilum/.julia/packages/VectorizationBase/NMmk1/src/llvm_intrin/memory_addr.jl:638 within `macro expansion'
%ptr.1.i = getelementptr inbounds i8, i8* %3, i64 130
store i8 %1, i8* %ptr.1.i, align 1
; └└
ret void
} |
|
However, I cannot reproduce if I start Julia with dilum:/Users/dilum/Downloads$ julia --compiled-modules=no
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.7.0-DEV.400 (2021-01-28)
_/ |\__'_|_|_|\__'_| | Commit 9de107a63a (5 days old master)
|__/ |
julia> using VectorizationBase
julia> !success(run(`grep -nr "pure" $(dirname(pathof(VectorizationBase)))`, wait=false))
true
julia> VectorizationBase.has_feature(::Val{:x86_64_avx512f}) = VectorizationBase.False()
julia> VectorizationBase.has_feature(::Val{:x86_64_avx2}) = VectorizationBase.False()
julia> using VectorizedRNG
julia> @code_typed VectorizedRNG.setrand64counter!(local_rng(), 0x01)
CodeInfo(
1 ─ %1 = Base.getfield(rng, :ptr)::Ptr{UInt64}
│ %2 = Base.bitcast(Ptr{UInt8}, %1)::Ptr{UInt8}
│ %3 = VectorizationBase.llvmcall((" \n\n define void @entry(i64, i8, i64) alwaysinline {\n top:\n %ptr.0 = inttoptr i64 %0 to i8*\n%ptr.1 = getelementptr inbounds i8, i8* %ptr.0, i64 %2\nstore i8 %1, i8* %ptr.1, align 1\nret void\n }\n", "entry"), VectorizationBase.Cvoid, Tuple{Ptr{UInt8}, UInt8, Int64}, %2, v, 130)::Nothing
└── return %3
) => Nothing
julia> @code_llvm VectorizedRNG.setrand64counter!(local_rng(), 0x01)
; @ /Users/dilum/.julia/packages/VectorizedRNG/RVAIZ/src/xoshiro.jl:97 within `setrand64counter!'
define void @"julia_setrand64counter!_1054"([1 x i64]* nocapture nonnull readonly align 8 dereferenceable(8) %0, i8 zeroext %1) {
top:
; @ /Users/dilum/.julia/packages/VectorizedRNG/RVAIZ/src/xoshiro.jl:98 within `setrand64counter!'
; ┌ @ essentials.jl:402 within `unsafe_convert'
; │┌ @ pointer.jl:30 within `convert'
%2 = bitcast [1 x i64]* %0 to i8**
%3 = load i8*, i8** %2, align 8
; └└
; ┌ @ /Users/dilum/.julia/packages/VectorizationBase/NMmk1/src/llvm_intrin/memory_addr.jl:782 within `vstore!' @ /Users/dilum/.julia/packages/VectorizationBase/NMmk1/src/llvm_intrin/memory_addr.jl:638
; │┌ @ /Users/dilum/.julia/packages/VectorizationBase/NMmk1/src/llvm_intrin/memory_addr.jl:638 within `macro expansion'
%ptr.1.i = getelementptr inbounds i8, i8* %3, i64 130
store i8 %1, i8* %ptr.1.i, align 1
; └└
ret void
} |
|
Great findings, I can reproduce both that redefining again after loading, or running with Seems like precompilation may be connected, where the old stuff isn't invalidated if it hasn't already been loaded? |
|
That would explain why As an awful hack, I could call the latter too, then at least they'd agree. lol |
|
If you can turn this into a script, we can do |
|
I.e. instead of just printing |
|
Here's the full bisect setup:
#!/bin/bash
rm -rf ~/.julia
make cleanall
make clean
make
rm -rf ~/.julia
./julia install-packages.jl
./julia test.jl
|
|
Hmmm. For me, I can still reproduce on 1.5.3. |
|
And I can reproduce on 1.5.0: |
|
On 1.5.0, I can't repro with $ julia --compiled-modules=no
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.5.0 (2020-08-01)
_/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |
julia> using VectorizationBase
julia> !success(run(`grep -nr "pure" $(dirname(pathof(VectorizationBase)))`, wait=false))
true
julia> VectorizationBase.has_feature(::Val{:x86_64_avx512f}) = VectorizationBase.False()
julia> VectorizationBase.has_feature(::Val{:x86_64_avx2}) = VectorizationBase.False()
julia> using VectorizedRNG
julia> @code_typed VectorizedRNG.setrand64counter!(local_rng(), 0x01)
CodeInfo(
1 ─ %1 = Base.getfield(rng, :ptr)::Ptr{UInt64}
│ %2 = Base.bitcast(Ptr{UInt8}, %1)::Ptr{UInt8}
│ %3 = VectorizationBase.llvmcall(("", "%ptr.0 = inttoptr i64 %0 to i8*\n%ptr.1 = getelementptr inbounds i8, i8* %ptr.0, i64 %2\nstore i8 %1, i8* %ptr.1, align 1\nret void"), VectorizationBase.Cvoid, Tuple{Ptr{UInt8},UInt8,Int64}, %2, v, 130)::Nothing
└── return %3
) => Nothing
julia> @code_llvm VectorizedRNG.setrand64counter!(local_rng(), 0x01)
; @ /Users/dilum/.julia/packages/VectorizedRNG/RVAIZ/src/xoshiro.jl:97 within `setrand64counter!'
define void @"julia_setrand64counter!_1664"([1 x i64]* nocapture nonnull readonly dereferenceable(8), i8) {
top:
; @ /Users/dilum/.julia/packages/VectorizedRNG/RVAIZ/src/xoshiro.jl:98 within `setrand64counter!'
; ┌ @ essentials.jl:392 within `unsafe_convert'
; │┌ @ pointer.jl:30 within `convert'
%2 = bitcast [1 x i64]* %0 to i8**
%3 = load i8*, i8** %2, align 8
; └└
; ┌ @ /Users/dilum/.julia/packages/VectorizationBase/maLbj/src/llvm_intrin/memory_addr.jl:782 within `vstore!' @ /Users/dilum/.julia/packages/VectorizationBase/maLbj/src/llvm_intrin/memory_addr.jl:638
; │┌ @ /Users/dilum/.julia/packages/VectorizationBase/maLbj/src/llvm_intrin/memory_addr.jl:638 within `macro expansion'
%ptr.1.i = getelementptr inbounds i8, i8* %3, i64 130
store i8 %1, i8* %ptr.1.i, align 1
; └└
ret void
}
julia> using ArrayInterface
julia> !success(run(`grep -nr "pure" $(dirname(pathof(ArrayInterface)))`, wait=false))
true
julia> versioninfo()
Julia Version 1.5.0
Commit 96786e22cc (2020-08-01 23:44 UTC)
Platform Info:
OS: macOS (x86_64-apple-darwin18.7.0)
CPU: Intel(R) Core(TM) i5-4278U CPU @ 2.60GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-9.0.1 (ORCJIT, haswell)
Environment:
ASDF_JULIA_VERSION = 1.5.0
JULIA_NUM_THREADS = auto
JULIA_PKG_SERVER =
JULIA_CUDA_VERBOSE = true |
|
Hmm, I can reproduce on 1.5 too now. Not sure how I wasn't earlier. |
|
Darn. Because we can repro on Julia 1.5.0, we won't be able to bisect. |
|
BTW, you should probably update your comment in the 265 issue to reflect that you can reproduce on 1.5.3. |
Tokazama
approved these changes
Feb 4, 2021
|
We should probably revisit some of this once aggressive const-prop arrives, but I think it looks good for now. |
We could use |
Sounds like a good idea. As for this PR, if it helps get people on board with solving the original issue that inspired this then it seems like a good idea. There's a possibility that the |
I'm honestly not really sure what to do here. This PR doesn't actually fix the issue, and there's at least a dozen things I'd rather spend my time on than VectorizationBase's relocatability problem at this point (which as of Julia 1.6 should just result in worse performance when it strikes; in Julia 1.5 it could lead to crashes) -- e.g., things that improve performance for everyone ;). This PR hasn't been thoroughly regression tested, so maybe we should just de- |
|
That would actually be my preference b/c I think there are some things here that definitely shouldn't be pure but some I copied from other packages verbatim where I know a lot of other people have approved of them (e.g., |
chriselrod
changed the title
@pure, fixes #115.@pure, does not actually fix #115.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The only remaining occurrence of
pureis in the README:I have JuliaLang/julia#265 (comment) problems on the master branch:
The function was not updating, it still says
%ptr.1.i = getelementptr inbounds i8, i8* %3, i64 514instead of%ptr.1.i = getelementptr inbounds i8, i8* %3, i64 130.But after this PR:
It updated as intended.