efficient sets using hash tables #1

StefanKarpinski · 2011-04-27T03:52:59Z

The current implementation was never meant to be more than a very temporary stop-gap and is frankly embarrassing.

Fix for SubDataFrame ref and change to col slices for DataFrame

Function calls: keyword arguments are collected and passed via a new `Keywords` object: julia> f(kw::Keywords, args...) = (args, kw) # method added to generic function f julia> f(1, a=3, 2, b=4) ((1,2),Keywords(a=3, b=4)) Method definitions: here, a keyword argument creates a new local variable whose value is overridden if called with a keyword of the same name: julia> g(kw::Keywords, args..., x=0) = (args, kw, x) # method added to generic function g julia> g(1, a=3, 2, b=4) ((1,2),Keywords(a=3, b=4),0) julia> g(1, a=3, 2, b=4, x=7) ((1,2),Keywords(a=3, b=4, x=7),7) Internally, function g(kw::Keywords, a, b, x=0) ... end get converted to function g(kw::Keywords, a, b) x = get(kw, :x, 0) ... end Methods lacking the `::Keywords` catch-all only accept their specific keywords: julia> h(args..., x=0) = (args, x) # method added to generic function h julia> h(1, 2, x=7) ((1,2),7) julia> h(1, a=3, 2, x=7) ERROR: unrecognized keyword a in h at no file

Added support for method(\t syntax for operators

@test

This commit fixes the first problem that was found while digging into #53613. It turns out that the post-domtree constructed from regular `IRCode` doesn't work for visiting conditional successors for post-opt analysis in cases like: ```julia julia> let code = Any[ # block 1 GotoIfNot(Argument(2), 3), # block 2 ReturnNode(Argument(3)), # block 3 (we should visit this block) Expr(:call, throw, "potential throw"), ReturnNode(), # unreachable ] ir = make_ircode(code; slottypes=Any[Any,Bool,Bool]) visited = BitSet() @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int push!(visited, succ) return false end @test 2 ∉ visited @test 3 ∈ visited end Test Failed at REPL[14]:16 Expression: 2 ∉ visited Evaluated: 2 ∉ BitSet([2]) ``` This might mean that we need to fix on the `postdominates` end, but for now, this commit tries to get around it by using the augmented post domtree in `visit_conditional_successors`, while also enforcing the augmented control flow graph (`construct_augmented_cfg`) to have a single exit node really. Since the augmented post domtree is now enforced to have a single return, we can keep using the current `postdominates` to fix the issue. However, this commit isn't enough to fix the NeuralNetworkReachability segfault as reported in #53613, and we need to tackle the second issue reported there too (#53613 (comment)).

@test

This commit fixes the first problem that was found while digging into #53613. It turns out that the post-domtree constructed from regular `IRCode` doesn't work for visiting conditional successors for post-opt analysis in cases like: ```julia julia> let code = Any[ # block 1 GotoIfNot(Argument(2), 3), # block 2 ReturnNode(Argument(3)), # block 3 (we should visit this block) Expr(:call, throw, "potential throw"), ReturnNode(), # unreachable ] ir = make_ircode(code; slottypes=Any[Any,Bool,Bool]) visited = BitSet() @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int push!(visited, succ) return false end @test 2 ∉ visited @test 3 ∈ visited end Test Failed at REPL[14]:16 Expression: 2 ∉ visited Evaluated: 2 ∉ BitSet([2]) ``` This might mean that we need to fix on the `postdominates` end, but for now, this commit tries to get around it by using the augmented post domtree in `visit_conditional_successors`, while also enforcing the augmented control flow graph (`construct_augmented_cfg`) to have a single exit node really. Since the augmented post domtree is now enforced to have a single return, we can keep using the current `postdominates` to fix the issue. However, this commit isn't enough to fix the NeuralNetworkReachability segfault as reported in #53613, and we need to tackle the second issue reported there too (#53613 (comment)).

This PR validates the input parameters to the Julia LAPACK wrappers, so that the error messages are more informative. On nightly ```julia julia> using LinearAlgebra julia> LAPACK.geev!('X', 'X', rand(2,2)) ** On entry to DGEEV parameter number 1 had an illegal value ERROR: ArgumentError: invalid argument #1 to LAPACK call ``` This PR ```julia julia> using LinearAlgebra julia> LAPACK.geev!('X', 'X', rand(2,2)) ERROR: ArgumentError: argument #1: jobvl must be one of ('N', 'V'), but 'X' was passed ``` Secondly, moved certain allocations (e.g. in `geevx`) below the validation checks, so that these only happen for valid parameter values. Thirdly, added `require_one_based_indexing` checks to functions where these were missing. (cherry picked from commit dcd1fb2)

@test

This commit fixes the first problem that was found while digging into #53613. It turns out that the post-domtree constructed from regular `IRCode` doesn't work for visiting conditional successors for post-opt analysis in cases like: ```julia julia> let code = Any[ # block 1 GotoIfNot(Argument(2), 3), # block 2 ReturnNode(Argument(3)), # block 3 (we should visit this block) Expr(:call, throw, "potential throw"), ReturnNode(), # unreachable ] ir = make_ircode(code; slottypes=Any[Any,Bool,Bool]) visited = BitSet() @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int push!(visited, succ) return false end @test 2 ∉ visited @test 3 ∈ visited end Test Failed at REPL[14]:16 Expression: 2 ∉ visited Evaluated: 2 ∉ BitSet([2]) ``` This might mean that we need to fix on the `postdominates` end, but for now, this commit tries to get around it by using the augmented post domtree in `visit_conditional_successors`, while also enforcing the augmented control flow graph (`construct_augmented_cfg`) to have a single exit node really. Since the augmented post domtree is now enforced to have a single return, we can keep using the current `postdominates` to fix the issue. However, this commit isn't enough to fix the NeuralNetworkReachability segfault as reported in #53613, and we need to tackle the second issue reported there too (#53613 (comment)).

@test

This is an alternative to #53642 The `dom_edges()` for an exit block in the CFG are empty when computing the PostDomTree so the loop below this may not actually run. In that case, the right semidominator is the ancestor from the DFSTree, which is the "virtual" -1 block. This resolves half of the issue in #53613: ```julia julia> let code = Any[ # block 1 GotoIfNot(Argument(2), 3), # block 2 ReturnNode(Argument(3)), # block 3 (we should visit this block) Expr(:call, throw, "potential throw"), ReturnNode(), # unreachable ] ir = make_ircode(code; slottypes=Any[Any,Bool,Bool]) visited = BitSet() @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int push!(visited, succ) return false end @test 2 ∈ visited @test 3 ∈ visited end Test Passed ``` This needs some tests (esp. since I don't think we have any DomTree tests at all right now), but otherwise should be good to go.

@test

This commit fixes the first problem that was found while digging into #53613. It turns out that the post-domtree constructed from regular `IRCode` doesn't work for visiting conditional successors for post-opt analysis in cases like: ```julia julia> let code = Any[ # block 1 GotoIfNot(Argument(2), 3), # block 2 ReturnNode(Argument(3)), # block 3 (we should visit this block) Expr(:call, throw, "potential throw"), ReturnNode(), # unreachable ] ir = make_ircode(code; slottypes=Any[Any,Bool,Bool]) visited = BitSet() @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int push!(visited, succ) return false end @test 2 ∉ visited @test 3 ∈ visited end Test Failed at REPL[14]:16 Expression: 2 ∉ visited Evaluated: 2 ∉ BitSet([2]) ``` This might mean that we need to fix on the `postdominates` end, but for now, this commit tries to get around it by using the augmented post domtree in `visit_conditional_successors`, while also enforcing the augmented control flow graph (`construct_augmented_cfg`) to have a single exit node really. Since the augmented post domtree is now enforced to have a single return, we can keep using the current `postdominates` to fix the issue. However, this commit isn't enough to fix the NeuralNetworkReachability segfault as reported in #53613, and we need to tackle the second issue reported there too (#53613 (comment)).

@test

This commit was initially crafted to fix an issue with post-opt analysis that came to light in #53613, planning to solve it by incorporating an augmented CFG directly into the post-opt analysis. But, with the issue now sorted out by #53739, this commit shifted focus to just adding test cases. === The original commit message === post-opt: use augmented post-domtree for `visit_conditional_successors` This commit fixes the first problem that was found while digging into #53613. It turns out that the post-domtree constructed from regular `IRCode` doesn't work for visiting conditional successors for post-opt analysis in cases like: ```julia julia> let code = Any[ # block 1 GotoIfNot(Argument(2), 3), # block 2 ReturnNode(Argument(3)), # block 3 (we should visit this block) Expr(:call, throw, "potential throw"), ReturnNode(), # unreachable ] ir = make_ircode(code; slottypes=Any[Any,Bool,Bool]) visited = BitSet() @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int push!(visited, succ) return false end @test 2 ∉ visited @test 3 ∈ visited end Test Failed at REPL[14]:16 Expression: 2 ∉ visited Evaluated: 2 ∉ BitSet([2]) ``` This might mean that we need to fix on the `postdominates` end, but for now, this commit tries to get around it by using the augmented post domtree in `visit_conditional_successors`, while also enforcing the augmented control flow graph (`construct_augmented_cfg`) to have a single exit node really. Since the augmented post domtree is now enforced to have a single return, we can keep using the current `postdominates` to fix the issue. However, this commit isn't enough to fix the NeuralNetworkReachability segfault as reported in #53613, and we need to tackle the second issue reported there too (#53613 (comment)).

@test

) This commit fixes the first problem that was found while digging into #53613. It turns out that the post-domtree constructed from regular `IRCode` doesn't work for visiting conditional successors for post-opt analysis in cases like: ```julia julia> let code = Any[ # block 1 GotoIfNot(Argument(2), 3), # block 2 ReturnNode(Argument(3)), # block 3 (we should visit this block) Expr(:call, throw, "potential throw"), ReturnNode(), # unreachable ] ir = make_ircode(code; slottypes=Any[Any,Bool,Bool]) visited = BitSet() @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int push!(visited, succ) return false end @test 2 ∉ visited @test 3 ∈ visited end Test Failed at REPL[14]:16 Expression: 2 ∉ visited Evaluated: 2 ∉ BitSet([2]) ``` This might mean that we need to fix on the `postdominates` end, but for now, this commit tries to get around it by using the augmented post domtree in `visit_conditional_successors`. Since the augmented post domtree is enforced to have a single return, we can keep using the current `postdominates` to fix the issue. However, this commit isn't enough to fix the NeuralNetworkReachability segfault as reported in #53613, and we need to tackle the second issue reported there too (#53613 (comment)).

@test

This is an alternative to #53642 The `dom_edges()` for an exit block in the CFG are empty when computing the PostDomTree so the loop below this may not actually run. In that case, the right semidominator is the ancestor from the DFSTree, which is the "virtual" -1 block. This resolves half of the issue in #53613: ```julia julia> let code = Any[ # block 1 GotoIfNot(Argument(2), 3), # block 2 ReturnNode(Argument(3)), # block 3 (we should visit this block) Expr(:call, throw, "potential throw"), ReturnNode(), # unreachable ] ir = make_ircode(code; slottypes=Any[Any,Bool,Bool]) visited = BitSet() @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int push!(visited, succ) return false end @test 2 ∈ visited @test 3 ∈ visited end Test Passed ``` This needs some tests (esp. since I don't think we have any DomTree tests at all right now), but otherwise should be good to go.

@test

) This commit fixes the first problem that was found while digging into #53613. It turns out that the post-domtree constructed from regular `IRCode` doesn't work for visiting conditional successors for post-opt analysis in cases like: ```julia julia> let code = Any[ # block 1 GotoIfNot(Argument(2), 3), # block 2 ReturnNode(Argument(3)), # block 3 (we should visit this block) Expr(:call, throw, "potential throw"), ReturnNode(), # unreachable ] ir = make_ircode(code; slottypes=Any[Any,Bool,Bool]) visited = BitSet() @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int push!(visited, succ) return false end @test 2 ∉ visited @test 3 ∈ visited end Test Failed at REPL[14]:16 Expression: 2 ∉ visited Evaluated: 2 ∉ BitSet([2]) ``` This might mean that we need to fix on the `postdominates` end, but for now, this commit tries to get around it by using the augmented post domtree in `visit_conditional_successors`. Since the augmented post domtree is enforced to have a single return, we can keep using the current `postdominates` to fix the issue. However, this commit isn't enough to fix the NeuralNetworkReachability segfault as reported in #53613, and we need to tackle the second issue reported there too (#53613 (comment)).

* Change to streaming out the heap snapshot data (#1) * Streaming the heap snapshot! This should prevent the engine from OOMing while recording the snapshot! Now we just need to sample the files, either online, before downloading, or offline after downloading :) If we're gonna do it offline, we'll want to gzip the files before downloading them. * Allow custom filename; use original API * Support legacy heap snapshot interface. Add reassembly function. * Add tests * Apply suggestions from code review * Update src/gc-heap-snapshot.cpp * Change to always save the parts in the same directory This way you can always recover from an OOM * Fix bug in reassembler: from_node and to_node were in the wrong order * Fix correctness mistake: The edges have to be reordered according to the node order. That's the whole reason this is tricky. But i'm not sure now whether the SoAs approach is actually an optimization.... It seems like we should probably prefer to inline the Edges right into the vector, rather than having to do another random lookup into the edges table? * Debugging messed up edge array idxs * Disable log message * Write the .nodes and .edges as binary data * Remove unnecessary logging * fix merge issues * attempt to add back the orphan node checking logic --------- Co-authored-by: Nathan Daly <nathan.daly@relational.ai> Co-authored-by: Nathan Daly <NHDaly@gmail.com> * attempt to fix the doc issue for assemble_snapshot remove unused k_node_number_of_fields from gc-heap-snapshot.cpp attempt to resolve the savepoint issue on serialize_node * remove println in take_heap_snapshot to avoid messing up console output in Julia REPL * rename alloc_type for array buffer in gc-heap-snapshot * streaming strings directly to avoid cache in memory dedupling strings for field paths * address PR comments --------- Co-authored-by: Nathan Daly <nathan.daly@relational.ai> Co-authored-by: Nathan Daly <NHDaly@gmail.com>

@isdefined

As an application of #55545, this commit avoids the insertion of `:throw_undef_if_not` nodes when the defined-ness of a slot is guaranteed by abstract interpretation. ```julia julia> function isdefined_nothrow(c, x) local val if c val = x end if @isdefined val return val end return zero(Int) end; julia> @code_typed isdefined_nothrow(true, 42) ``` ```diff diff --git a/old b/new index c4980a5c9c..3d1d6d30f0 100644 --- a/old +++ b/new @@ -4,7 +4,6 @@ CodeInfo( 3 ┄ %3 = φ (#2 => x, #1 => #undef)::Int64 │ %4 = φ (#2 => true, #1 => false)::Bool └── goto #5 if not %4 -4 ─ $(Expr(:throw_undef_if_not, :val, :(%4)))::Any -└── return %3 +4 ─ return %3 5 ─ return 0 ) => Int64 ```

@isdefined

As an application of #55545, this commit avoids the insertion of `:throw_undef_if_not` nodes when the defined-ness of a slot is guaranteed by abstract interpretation. ```julia julia> function isdefined_nothrow(c, x) local val if c val = x end if @isdefined val return val end return zero(Int) end; julia> @code_typed isdefined_nothrow(true, 42) ``` ```diff diff --git a/old b/new index c4980a5c9c..3d1d6d30f0 100644 --- a/old +++ b/new @@ -4,7 +4,6 @@ CodeInfo( 3 ┄ %3 = φ (#2 => x, #1 => #undef)::Int64 │ %4 = φ (#2 => true, #1 => false)::Bool └── goto #5 if not %4 -4 ─ $(Expr(:throw_undef_if_not, :val, :(%4)))::Any -└── return %3 +4 ─ return %3 5 ─ return 0 ) => Int64 ```

@isdefined

As an application of #55545, this commit avoids the insertion of `:throw_undef_if_not` nodes when the defined-ness of a slot is guaranteed by abstract interpretation. ```julia julia> function isdefined_nothrow(c, x) local val if c val = x end if @isdefined val return val end return zero(Int) end; julia> @code_typed isdefined_nothrow(true, 42) ``` ```diff diff --git a/old b/new index c4980a5c9c..3d1d6d30f0 100644 --- a/old +++ b/new @@ -4,7 +4,6 @@ CodeInfo( 3 ┄ %3 = φ (#2 => x, #1 => #undef)::Int64 │ %4 = φ (#2 => true, #1 => false)::Bool └── goto #5 if not %4 -4 ─ $(Expr(:throw_undef_if_not, :val, :(%4)))::Any -└── return %3 +4 ─ return %3 5 ─ return 0 ) => Int64 ```

Prior to this, especially on macOS, the gc-safepoint here would cause the process to segfault as we had already freed the current_task state. Rearrange this code so that the GC interactions (except for the atomic store to current_task) are all handled before entering GC safe, and then signaling the thread is deleted (via setting current_task = NULL, published by jl_unlock_profile_wr to other threads) is last. ``` ERROR: Exception handler triggered on unmanaged thread. Process 53827 stopped * thread #5, stop reason = EXC_BAD_ACCESS (code=2, address=0x100018008) frame #0: 0x0000000100b74344 libjulia-internal.1.12.0.dylib`jl_delete_thread [inlined] jl_gc_state_set(ptls=0x000000011f8b3200, state='\x02', old_state=<unavailable>) at julia_threads.h:272:9 [opt] 269 assert(old_state != JL_GC_CONCURRENT_COLLECTOR_THREAD); 270 jl_atomic_store_release(&ptls->gc_state, state); 271 if (state == JL_GC_STATE_UNSAFE || old_state == JL_GC_STATE_UNSAFE) -> 272 jl_gc_safepoint_(ptls); 273 return old_state; 274 } 275 STATIC_INLINE int8_t jl_gc_state_save_and_set(jl_ptls_t ptls, Target 0: (julia) stopped. (lldb) up frame #1: 0x0000000100b74320 libjulia-internal.1.12.0.dylib`jl_delete_thread [inlined] jl_gc_state_save_and_set(ptls=0x000000011f8b3200, state='\x02') at julia_threads.h:278:12 [opt] 275 STATIC_INLINE int8_t jl_gc_state_save_and_set(jl_ptls_t ptls, 276 int8_t state) 277 { -> 278 return jl_gc_state_set(ptls, state, jl_atomic_load_relaxed(&ptls->gc_state)); 279 } 280 #ifdef __clang_gcanalyzer__ 281 // these might not be a safepoint (if they are no-op safe=>safe transitions), but we have to assume it could be (statically) (lldb) frame #2: 0x0000000100b7431c libjulia-internal.1.12.0.dylib`jl_delete_thread(value=0x000000011f8b3200) at threading.c:537:11 [opt] 534 ptls->root_task = NULL; 535 jl_free_thread_gc_state(ptls); 536 // then park in safe-region -> 537 (void)jl_gc_safe_enter(ptls); 538 } ```

Prior to this, especially on macOS, the gc-safepoint here would cause the process to segfault as we had already freed the current_task state. Rearrange this code so that the GC interactions (except for the atomic store to current_task) are all handled before entering GC safe, and then signaling the thread is deleted (via setting current_task = NULL, published by jl_unlock_profile_wr to other threads) is last. ``` ERROR: Exception handler triggered on unmanaged thread. Process 53827 stopped * thread #5, stop reason = EXC_BAD_ACCESS (code=2, address=0x100018008) frame #0: 0x0000000100b74344 libjulia-internal.1.12.0.dylib`jl_delete_thread [inlined] jl_gc_state_set(ptls=0x000000011f8b3200, state='\x02', old_state=<unavailable>) at julia_threads.h:272:9 [opt] 269 assert(old_state != JL_GC_CONCURRENT_COLLECTOR_THREAD); 270 jl_atomic_store_release(&ptls->gc_state, state); 271 if (state == JL_GC_STATE_UNSAFE || old_state == JL_GC_STATE_UNSAFE) -> 272 jl_gc_safepoint_(ptls); 273 return old_state; 274 } 275 STATIC_INLINE int8_t jl_gc_state_save_and_set(jl_ptls_t ptls, Target 0: (julia) stopped. (lldb) up frame #1: 0x0000000100b74320 libjulia-internal.1.12.0.dylib`jl_delete_thread [inlined] jl_gc_state_save_and_set(ptls=0x000000011f8b3200, state='\x02') at julia_threads.h:278:12 [opt] 275 STATIC_INLINE int8_t jl_gc_state_save_and_set(jl_ptls_t ptls, 276 int8_t state) 277 { -> 278 return jl_gc_state_set(ptls, state, jl_atomic_load_relaxed(&ptls->gc_state)); 279 } 280 #ifdef __clang_gcanalyzer__ 281 // these might not be a safepoint (if they are no-op safe=>safe transitions), but we have to assume it could be (statically) (lldb) frame #2: 0x0000000100b7431c libjulia-internal.1.12.0.dylib`jl_delete_thread(value=0x000000011f8b3200) at threading.c:537:11 [opt] 534 ptls->root_task = NULL; 535 jl_free_thread_gc_state(ptls); 536 // then park in safe-region -> 537 (void)jl_gc_safe_enter(ptls); 538 } ``` (test incorporated into #55793)

Prior to this, especially on macOS, the gc-safepoint here would cause the process to segfault as we had already freed the current_task state. Rearrange this code so that the GC interactions (except for the atomic store to current_task) are all handled before entering GC safe, and then signaling the thread is deleted (via setting current_task = NULL, published by jl_unlock_profile_wr to other threads) is last. ``` ERROR: Exception handler triggered on unmanaged thread. Process 53827 stopped * thread JuliaLang#5, stop reason = EXC_BAD_ACCESS (code=2, address=0x100018008) frame #0: 0x0000000100b74344 libjulia-internal.1.12.0.dylib`jl_delete_thread [inlined] jl_gc_state_set(ptls=0x000000011f8b3200, state='\x02', old_state=<unavailable>) at julia_threads.h:272:9 [opt] 269 assert(old_state != JL_GC_CONCURRENT_COLLECTOR_THREAD); 270 jl_atomic_store_release(&ptls->gc_state, state); 271 if (state == JL_GC_STATE_UNSAFE || old_state == JL_GC_STATE_UNSAFE) -> 272 jl_gc_safepoint_(ptls); 273 return old_state; 274 } 275 STATIC_INLINE int8_t jl_gc_state_save_and_set(jl_ptls_t ptls, Target 0: (julia) stopped. (lldb) up frame JuliaLang#1: 0x0000000100b74320 libjulia-internal.1.12.0.dylib`jl_delete_thread [inlined] jl_gc_state_save_and_set(ptls=0x000000011f8b3200, state='\x02') at julia_threads.h:278:12 [opt] 275 STATIC_INLINE int8_t jl_gc_state_save_and_set(jl_ptls_t ptls, 276 int8_t state) 277 { -> 278 return jl_gc_state_set(ptls, state, jl_atomic_load_relaxed(&ptls->gc_state)); 279 } 280 #ifdef __clang_gcanalyzer__ 281 // these might not be a safepoint (if they are no-op safe=>safe transitions), but we have to assume it could be (statically) (lldb) frame JuliaLang#2: 0x0000000100b7431c libjulia-internal.1.12.0.dylib`jl_delete_thread(value=0x000000011f8b3200) at threading.c:537:11 [opt] 534 ptls->root_task = NULL; 535 jl_free_thread_gc_state(ptls); 536 // then park in safe-region -> 537 (void)jl_gc_safe_enter(ptls); 538 } ``` (test incorporated into JuliaLang#55793)

@alexfanqi

Rebase and extension of @alexfanqi's initial work on porting Julia to RISC-V. Requires LLVM 19. Tested on a VisionFive2, built with: ```make MARCH := rv64gc_zba_zbb MCPU := sifive-u74 USE_BINARYBUILDER:=0 DEPS_GIT = llvm override LLVM_VER=19.1.1 override LLVM_BRANCH=julia-release/19.x override LLVM_SHA1=julia-release/19.x ``` ```julia-repl ❯ ./julia _ _ _ _(_)_ | Documentation: https://docs.julialang.org (_) | (_) (_) | _ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help. | | | | | | |/ _` | | | | |_| | | | (_| | | Version 1.12.0-DEV.1374 (2024-10-14) _/ |\__'_|_|_|\__'_| | riscv/25092a3982* (fork: 1 commits, 0 days) |__/ | julia> versioninfo(; verbose=true) Julia Version 1.12.0-DEV.1374 Commit 25092a3* (2024-10-14 09:57 UTC) Platform Info: OS: Linux (riscv64-unknown-linux-gnu) uname: Linux 6.11.3-1-riscv64 #1 SMP Debian 6.11.3-1 (2024-10-10) riscv64 unknown CPU: unknown: speed user nice sys idle irq #1 1500 MHz 922 s 0 s 265 s 160953 s 0 s #2 1500 MHz 457 s 0 s 280 s 161521 s 0 s #3 1500 MHz 452 s 0 s 270 s 160911 s 0 s #4 1500 MHz 638 s 15 s 301 s 161340 s 0 s Memory: 7.760246276855469 GB (7474.08203125 MB free) Uptime: 16260.13 sec Load Avg: 0.25 0.23 0.1 WORD_SIZE: 64 LLVM: libLLVM-19.1.1 (ORCJIT, sifive-u74) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) Environment: HOME = /home/tim PATH = /home/tim/.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/games TERM = xterm-256color julia> ccall(:jl_dump_host_cpu, Nothing, ()) CPU: sifive-u74 Features: +zbb,+d,+i,+f,+c,+a,+zba,+m,-zvbc,-zksed,-zvfhmin,-zbkc,-zkne,-zksh,-zfh,-zfhmin,-zknh,-v,-zihintpause,-zicboz,-zbs,-zvknha,-zvksed,-zfa,-ztso,-zbc,-zvknhb,-zihintntl,-zknd,-zvbb,-zbkx,-zkt,-zvkt,-zicond,-zvksh,-zvfh,-zvkg,-zvkb,-zbkb,-zvkned julia> @code_native debuginfo=:none 1+2. .text .attribute 4, 16 .attribute 5, "rv64i2p1_m2p0_a2p1_f2p2_d2p2_c2p0_zicsr2p0_zifencei2p0_zmmul1p0_zba1p0_zbb1p0" .file "+" .globl "julia_+_3003" .p2align 1 .type "julia_+_3003",@function "julia_+_3003": addi sp, sp, -16 sd ra, 8(sp) sd s0, 0(sp) addi s0, sp, 16 fcvt.d.l fa5, a0 ld ra, 8(sp) ld s0, 0(sp) fadd.d fa0, fa5, fa0 addi sp, sp, 16 ret .Lfunc_end0: .size "julia_+_3003", .Lfunc_end0-"julia_+_3003" .type ".L+Core.Float64#3005",@object .section .data.rel.ro,"aw",@progbits .p2align 3, 0x0 ".L+Core.Float64#3005": .quad ".L+Core.Float64#3005.jit" .size ".L+Core.Float64#3005", 8 .set ".L+Core.Float64#3005.jit", 272467692544 .size ".L+Core.Float64#3005.jit", 8 .section ".note.GNU-stack","",@progbits ``` Lots of bugs guaranteed, but with this we at least have a functional build and REPL for further development by whoever is interested. Also requires Linux 6.4+, since the fallback processor detection used here relies on LLVM's `sys::getHostCPUFeatures`, which for RISC-V is implemented using hwprobe introduced in 6.4. We could probably add a fallback that parses `/proc/cpuinfo`, either by building a CPU database much like how we've done for AArch64, or by parsing the actual ISA string contained there. That would probably also be a good place to add support for profiles, which are supposedly the way forward to package RISC-V binaries. That can happen in follow-up PRs though. For now, on older kernels, use the `-C` arg to Julia to specify an ISA. Co-authored-by: Alex Fan <alex.fan.q@gmail.com>

…56300) The pipeline-prints test currently fails when running on an aarch64-macos device: ``` /Users/tim/Julia/src/julia/test/llvmpasses/pipeline-prints.ll:309:23: error: AFTERVECTORIZATION: expected string not found in input ; AFTERVECTORIZATION: vector.body ^ <stdin>:2:40: note: scanning from here ; *** IR Dump Before AfterVectorizationMarkerPass on julia_f_199 *** ^ <stdin>:47:27: note: possible intended match here ; *** IR Dump Before AfterVectorizationMarkerPass on jfptr_f_200 *** ^ Input file: <stdin> Check file: /Users/tim/Julia/src/julia/test/llvmpasses/pipeline-prints.ll -dump-input=help explains the following input dump. Input was: <<<<<< 1: opt: WARNING: failed to create target machine for 'x86_64-unknown-linux-gnu': unable to get target for 'x86_64-unknown-linux-gnu', see --version and --triple. 2: ; *** IR Dump Before AfterVectorizationMarkerPass on julia_f_199 *** check:309'0 X~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found 3: define i64 @julia_f_199(ptr addrspace(10) noundef nonnull align 16 dereferenceable(40) %0) #0 !dbg !4 { check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 4: top: check:309'0 ~~~~~ 5: %1 = call ptr @julia.get_pgcstack() check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 6: %ptls_field = getelementptr inbounds ptr, ptr %1, i64 2 check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 7: %ptls_load45 = load ptr, ptr %ptls_field, align 8, !tbaa !8 check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ . . . 42: check:309'0 ~ 43: L41: ; preds = %L41.loopexit, %L17, %top check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 44: %value_phi10 = phi i64 [ 0, %top ], [ %7, %L17 ], [ %.lcssa, %L41.loopexit ] check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 45: ret i64 %value_phi10, !dbg !52 check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 46: } check:309'0 ~~ 47: ; *** IR Dump Before AfterVectorizationMarkerPass on jfptr_f_200 *** check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ check:309'1 ? possible intended match 48: ; Function Attrs: noinline optnone check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 49: define nonnull ptr addrspace(10) @jfptr_f_200(ptr addrspace(10) %0, ptr noalias nocapture noundef readonly %1, i32 %2) #1 { check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 50: top: check:309'0 ~~~~~ 51: %3 = call ptr @julia.get_pgcstack() check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 52: %4 = getelementptr inbounds ptr addrspace(10), ptr %1, i32 0 check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ . . . >>>>>> -- ******************** Failed Tests (1): Julia :: pipeline-prints.ll ``` The problem is that these tests assume x86_64, which fails because the target isn't available, so it presumably uses the native target which has different vectorization characteristics: ``` ❯ ./usr/tools/opt --load-pass-plugin=libjulia-codegen.dylib -passes='julia' --print-before=AfterVectorization -o /dev/null ../../test/llvmpasses/pipeline-prints.ll ./usr/tools/opt: WARNING: failed to create target machine for 'x86_64-unknown-linux-gnu': unable to get target for 'x86_64-unknown-linux-gnu', see --version and --triple. ``` There's other tests that assume this (e.g. the `fma` cpufeatures one), but they don't fail, so I've left them as is.

@test

This is an alternative to #53642 The `dom_edges()` for an exit block in the CFG are empty when computing the PostDomTree so the loop below this may not actually run. In that case, the right semidominator is the ancestor from the DFSTree, which is the "virtual" -1 block. This resolves half of the issue in #53613: ```julia julia> let code = Any[ # block 1 GotoIfNot(Argument(2), 3), # block 2 ReturnNode(Argument(3)), # block 3 (we should visit this block) Expr(:call, throw, "potential throw"), ReturnNode(), # unreachable ] ir = make_ircode(code; slottypes=Any[Any,Bool,Bool]) visited = BitSet() @test !Core.Compiler.visit_conditional_successors(CC.LazyPostDomtree(ir), ir, #=bb=#1) do succ::Int push!(visited, succ) return false end @test 2 ∈ visited @test 3 ∈ visited end Test Passed ``` This needs some tests (esp. since I don't think we have any DomTree tests at all right now), but otherwise should be good to go.

ghost assigned StefanKarpinski Apr 27, 2011

StefanKarpinski closed this as completed in a555efb Jun 15, 2011

ViralBShah mentioned this issue Dec 25, 2011

Framework for symbolic optimizations #122

Closed

ajdecon mentioned this issue Feb 19, 2012

OpenBLAS tests fail during julia build #388

Closed

This was referenced Feb 27, 2012

wrong while install julia #473

Closed

make: *** [sys0.ji] Error 1 How can I solve it?? #474

Closed

slabanja mentioned this issue Mar 11, 2012

Segfault in "bare-mode" #567

Closed

ViralBShah mentioned this issue Mar 16, 2012

WIP: libuv rewrite and Windows port #521

Merged

vtjnash mentioned this issue Mar 17, 2012

build fails at sys.ji #600

Closed

ViralBShah mentioned this issue Mar 18, 2012

Building julia with clang: tests fail #602

Closed

Norgat mentioned this issue Mar 19, 2012

make fails #611

Closed

6e441f9c mentioned this issue Apr 8, 2012

sort segfaults #681

Closed

klmr mentioned this issue Apr 30, 2012

OpenBLAS build error (missing libquadmath.so.0) on x64 Linux #779

Closed

timholy mentioned this issue May 3, 2012

Dispatch method selection (RangeIndex... more specific than Integer?) #795

Closed

kk49 mentioned this issue May 9, 2012

finfer causes segmentation fault #816

Closed

forkandwait mentioned this issue May 12, 2012

Failed build on FreeBSD #829

Closed

6e441f9c mentioned this issue May 30, 2012

Julia segfaults while loading stage0.jl #901

Closed

cmey mentioned this issue Jun 21, 2012

julia fails to build with a segfault #960

Closed

tshort referenced this issue in tshort/julia Jun 23, 2012

Merge pull request #1 from tshort/DataPlaying

183dbfa

Fix for SubDataFrame ref and change to col slices for DataFrame

spott mentioned this issue Jul 16, 2012

problem bootstrapping the sys0.ji file #1052

Closed

dioptre mentioned this issue Jul 30, 2012

error during bootstrap on OS X #1089

Closed

timholy mentioned this issue Aug 4, 2012

Functions that produce results in pre-defined outputs #1115

Closed

staticfloat mentioned this issue Nov 1, 2012

Segfault when aborting matrix multiplication #1468

Closed

quesada mentioned this issue Dec 20, 2012

Failed compliation on ubuntu 12.10, libLLVMInstCombine.a(InstCombineAddSub.o) in archive is not an object #1796

Closed

fundamental mentioned this issue Apr 11, 2013

Regression in reload() Results in REPL Hanging #2829

Closed

railskipl mentioned this issue May 3, 2013

Tab-Expansion of Base in REPL causes SIGABRT #2064

Closed

brorson mentioned this issue Jun 7, 2013

Intermittant segfault when starting Julia on FC13... Race condition? #3323

Closed

JeffBezanson pushed a commit that referenced this issue Oct 5, 2013

Merge pull request #1 from ivarne/methods2

25becac

Added support for method(\t syntax for operators

staticfloat mentioned this issue Nov 19, 2013

Roadmap for 0.3 #4853

Closed

21 tasks

Aijunly mentioned this issue Mar 13, 2024

When update the pacages with erros. #53690

Open

cfborges mentioned this issue Mar 31, 2024

Correctly rounded variant of the hypot code #32345

Merged

d-netto mentioned this issue Apr 26, 2024

Canonicalize names of nested functions by keeping a more fine grained counter -- per (module, method name) pair #53719

Merged

Andrew-Milestone mentioned this issue Apr 28, 2024

Error adding NeuralOperators in Julia v1.10.2 #54292

Closed

gbaraldi mentioned this issue Jul 2, 2024

Bump LLVM to v18 #54848

Merged

CarolinaEstevez mentioned this issue Jul 17, 2024

Plots package doesn't work #55153

Open

AUK1939 mentioned this issue Sep 5, 2024

How to drill into this crash #55696

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

efficient sets using hash tables #1

efficient sets using hash tables #1

StefanKarpinski commented Apr 27, 2011

efficient sets using hash tables #1

efficient sets using hash tables #1

Comments

StefanKarpinski commented Apr 27, 2011