Port EscapeAnalysis to Core.Compiler with Optimization #3

TH3CHARLie · 2021-07-18T11:48:02Z

No description provided.

TH3CHARLie · 2021-07-26T12:11:31Z

A status report of this PR:

after disabling the isbitstype related feature (due to assertion failure in argextype) and disabling the global cache of invoke (therefore analyzing every statically resolved call when encountered), escape analysis pass is now turned on by default and can successfully pass the bootstrapping.

My goal is first to observe the IR_FLAG_NO_ESCAPE is successfully set and can be observed on C++ side because this is critical to decide on what LLVM we should special-case.

First we verify the result using EscapeAnalysis.jl, to make the results in sync with the porting version, I disabled some features in the package (see above for description):

julia> using EscapeAnalysis
[ Info: Precompiling EscapeAnalysis [c6a77b2f-5d97-4bf0-b260-35fa9cc0f3d0]

julia> function dummy_ref(s::String, b::Base.RefValue{String})
                  a = Ref(s)
                  a === b
              end
dummy_ref (generic function with 1 method)

julia> @analyze_escapes dummy_ref("haha", Ref("haha"))
typeof(dummy_ref)(_2::String ↑, _3::Base.RefValue{String} ↑, _4::Base.RefValue{String} ↓)
1 ↓ 1 ─ %1 = %new(Base.RefValue{String}, _2)::Base.RefValue{String}
2 ↑ │   %2 = (%1 === _3)::Bool
3 ◌ └──      return %2

As we expected, the %1 is marked as NoEscape, therefore the statement will be added IR_FLAG_NO_ESCAPE.

We use the following code to observe the flag on C++ side (see below), but it never triggers, notice here we don't need to use Revise, we just start the REPL compiled using this PR's code.

julia> function dummy_ref(s::String, b::Base.RefValue{String})
                  a = Ref(s)
                  a === b
              end
dummy_ref (generic function with 1 method)

julia> @code_typed optimize=true dummy_ref("haha", Ref("haha"))
CodeInfo(
1 ─ %1 = %new(Base.RefValue{String}, s)::Base.RefValue{String}
│   %2 = (%1 === b)::Bool
└──      return %2
) => Bool

src/codegen.cpp

TH3CHARLie · 2021-08-03T12:16:05Z

src/llvm-alloc-opt.cpp

+                printf("llvm-alloc-opt: find a metadata tag %p Function %p size %d\n", orig, &F, sz);
+            }
+        }
+        if (use_info.escaped && !has_metadata_tag) {


Main changes here!

TH3CHARLie · 2021-08-03T12:36:16Z

A status report of the latest progress:

I find the instruction will make the use_info.escaped and will therefore go to the optimizeTag call. So I changed the logic to make Instruction tagged with our metadata go to the moveToStack call even if it is marked as escaped in the alloc-opt pass. The main code change is shown in the comment above.

However, this leads to an abort error:

signal (6): Abort trap: 6
in expression starting at REPL[2]:1
__pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
Allocations: 1156428 (Pool: 1152371; Big: 4057); GC: 1
[1]    76587 abort      ./julia

cc @vtjnash

vtjnash · 2021-08-03T16:50:49Z

It is unfortunate that it seems not to have unwind info for abort yet (Apple may have a fix in the next release), but catching that happen in lldb will likely show you which assert is failing.

@time

So EscapeAnalysis.jl has a serious performance problem, which appears only in bootstrapping. I made some inspections using the following `print` debug: > within `find_escapes!` ```julia if length(argtypes) > 0 Core.println(argtypes[1], " ", length(argtypes), " ", nstmts) end ``` ... and it seems like the performance of `find_escapes!` will be really awful when there is a large number of statements, say, `nstmts > 100`. The worse case I confirmed so far happens when analyzing `construct_ssa!(::CodeInfo, ::IRCode, ::DomTree, ::Any, ::Vector{Any})`, where I got the print `Core.Const(val=Core.Compiler.construct_ssa!) 182 4262`, and it took more than 100 sec on my machine (!!!). Note that `find_escapes!` works on after-inlining IR, so we often such many statements. The interesting property is, this performance problem seems really specific to bootstrapping. I confirmed that `find_escapes!` runs in 0.34 sec when I enabled it later using Revise. ```julia julia> @time code_typed(Core.Compiler.construct_ssa!, (Core.Compiler.CodeInfo, Core.Compiler.IRCode, Core.Compiler.DomTree, Any, Vector{Any})); ... Core.Const(val=Core.Compiler.setindex!) 4 5 Core.Const(val=Core.Compiler.construct_ssa!) 182 4262 0.342626 seconds (723.68 k allocations: 177.828 MiB, 15.69% gc time, 99.94% compilation time) ```

fix performance problem in bootstrapping

…RLie/julia into th3/port-to-core-compiler

TH3CHARLie · 2021-08-15T17:13:11Z

Like this? @vchuravy

julia> @noinline f_no_escape(@nospecialize(x)) = typeof(x)
f_no_escape (generic function with 1 method)

julia> function g(x)
           aaa = Ref("foo")
           a = f_no_escape(aaa)
           nothing
       end
g (generic function with 1 method)

julia> @code_llvm g("haha")
;  @ REPL[5]:1 within `g`
; Function Attrs: sspstrong
define nonnull {}* @japi1_g_152({}* %0, {}** %1, i32 %2) #0 {
top:
  %3 = alloca {}**, align 8
  store volatile {}** %1, {}*** %3, align 8
;  @ REPL[5]:4 within `g`
  ret {}* inttoptr (i64 4557434888 to {}*)
}

vchuravy · 2021-08-15T17:17:56Z

What's the LLVM for no_escape? And does it run?

TH3CHARLie · 2021-08-15T17:20:49Z

the LLVM for f_no_escape

julia> @code_llvm f_no_escape(Ref("foo"))
;  @ REPL[4]:1 within `f_no_escape`
; Function Attrs: sspstrong
define nonnull {}* @japi1_f_no_escape_153({}* %0, {}** %1, i32 %2) #0 {
top:
  %3 = alloca {}**, align 8
  store volatile {}** %1, {}*** %3, align 8
  ret {}* inttoptr (i64 4772411712 to {}*)
}

Given this, I think it's inlined inside g, am I correct?

vchuravy · 2021-08-15T17:46:59Z

Yeah we knew what type it had and forwarded it... I was hoping to construe a scenario where we didn't know during codegen, but type inference would determine the value to be non-escaping. My worry is that we then would try to access the type tag (which is stored in front of the heap allocated object) but since we stack allocated it we would access invalid memory.

…

On Sun, Aug 15, 2021, 19:21 Xuanda Yang ***@***.***> wrote: the LLVM for f_no_escape julia> @code_llvm f_no_escape(Ref("foo")) ; @ REPL[4]:1 within `f_no_escape` ; Function Attrs: sspstrong define nonnull {}* @japi1_f_no_escape_153({}* %0, {}** %1, i32 %2) #0 { top: %3 = alloca {}**, align 8 store volatile {}** %1, {}*** %3, align 8 ret {}* inttoptr (i64 4772411712 to {}*) } Given this, I think it's inlined inside g, am I correct? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABDO2S4MC3KHZYDAMK5F6DT47ZPZANCNFSM5ASCBAMA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email> .

vchuravy · 2021-08-15T17:47:59Z

Can the escape analysis look through Base.inferencebarrier?

…

On Sun, Aug 15, 2021, 19:46 Valentin Churavy ***@***.***> wrote: Yeah we knew what type it had and forwarded it... I was hoping to construe a scenario where we didn't know during codegen, but type inference would determine the value to be non-escaping. My worry is that we then would try to access the type tag (which is stored in front of the heap allocated object) but since we stack allocated it we would access invalid memory. On Sun, Aug 15, 2021, 19:21 Xuanda Yang ***@***.***> wrote: > the LLVM for f_no_escape > > julia> @code_llvm f_no_escape(Ref("foo")) > ; @ REPL[4]:1 within `f_no_escape` > ; Function Attrs: sspstrong > define nonnull {}* @japi1_f_no_escape_153({}* %0, {}** %1, i32 %2) #0 { > top: > %3 = alloca {}**, align 8 > store volatile {}** %1, {}*** %3, align 8 > ret {}* inttoptr (i64 4772411712 to {}*) > } > > Given this, I think it's inlined inside g, am I correct? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#3 (comment)>, or > unsubscribe > <https://github.com/notifications/unsubscribe-auth/AABDO2S4MC3KHZYDAMK5F6DT47ZPZANCNFSM5ASCBAMA> > . > Triage notifications on the go with GitHub Mobile for iOS > <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> > or Android > <https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email> > . >

TH3CHARLie · 2021-08-16T13:55:18Z

With the below example, I think the analysis pass can look through the Base.inferencebarrier call

julia> @noinline f_no_escape(x) = Base.inferencebarrier(nothing)
f_no_escape (generic function with 1 method)

julia> @analyze_escapes f_no_escape(Ref("hah"))
typeof(f_no_escape)(_2::Base.RefValue{String} ↑)
1 ◌ 1 ─     return nothing


julia> @noinline f_no_escape(x) = (Base.inferencebarrier(nothing); x)
f_no_escape (generic function with 1 method)

julia> @analyze_escapes f_no_escape(Ref("hah"))
typeof(f_no_escape)(_2::Base.RefValue{String} ↑)
1 ◌ 1 ─     return _2

vtjnash · 2021-08-16T15:16:51Z

@vchuravy is suggesting something like this:

julia> @noinline f(x) = typeof(x.contents)

julia> @code_llvm f(Core.Box(3))
define nonnull {}* @japi1_f_366({}* %0, {}** %1, i32 %2) #0 {
top:
  %3 = alloca {}**, align 8
  store volatile {}** %1, {}*** %3, align 8
  %4 = bitcast {}** %1 to {}***
  %5 = load {}**, {}*** %4, align 8
; ┌ @ Base.jl:42 within `getproperty`
   %6 = load atomic {}*, {}** %5 unordered, align 8
   %.not = icmp eq {}* %6, null
   br i1 %.not, label %fail, label %pass

fail:                                             ; preds = %top
   call void @jl_throw({}* inttoptr (i64 140632140761280 to {}*))
   unreachable

pass:                                             ; preds = %top
; └
  %7 = bitcast {}* %6 to i64*
  %8 = getelementptr inbounds i64, i64* %7, i64 -1
  %9 = load atomic i64, i64* %8 unordered, align 8
  %10 = and i64 %9, -16
  %11 = inttoptr i64 %10 to {}*
  ret {}* %11
}

TH3CHARLie · 2021-08-16T15:20:13Z

Then in this case, x of the function f will be marked as Escape after the analysis pass.

vtjnash · 2021-08-16T15:43:25Z

Where does it escape from? It can't be the typeof call, since the compiler / runtime sometimes injects though just for convenience

TH3CHARLie · 2021-08-16T15:47:05Z

In your example, contents is accessed by a getfield, and it's here that the argument get marked as escaped.

On current master of EscapeAnalysis.jl

julia> @analyze_escapes f_no_escape(Core.Box(3))
typeof(f_no_escape)(_2::Core.Box *)
1 ◌ 1 ─ %1 = Base.getfield(_2, :contents)::Any
2 ↑ │   %2 = Main.typeof(%1)::DataType
3 ◌ └──      return %2

TH3CHARLie · 2021-08-16T15:52:23Z

Yeah I've noticed the difference and have updated the result in the comment

vtjnash · 2021-08-16T15:54:39Z

I suppose because it could throw? Does the Some{Any}().value type work for this example?

TH3CHARLie · 2021-08-16T15:56:48Z

I think anything involving in a getfield is conservatively handled because we cannot prove its effect-freeness now: https://github.com/aviatesk/EscapeAnalysis.jl/blob/34449a763b14a90f0eebe0ea842f88f248e524c6/test/runtests.jl#L129-L131

aviatesk · 2021-08-17T07:00:37Z

The throwness check depends on minimum number of initialized fields, and that's why Jameson asked whether Some{Any}().value works or not (and further, this is why we use MutableSome in our test suite, etc.).
Follow the link in the comment and check the definitions of those structs to understand the differences.

TH3CHARLie · 2021-08-17T07:56:22Z

I am still pretty confused about this. By "depends on minimum number of initialized fields" do you mean the check will produce different results given the different numbers of fields inside the struct (for example, MutableSome only has one)?

For jameson's comment, are we looking for something like this:

julia> using EscapeAnalysis

julia> @noinline f_no_escape(x::Some{Any}) = typeof(x.value)
f_no_escape (generic function with 1 method)

julia> @analyze_escapes f_no_escape(Some(1))
ERROR: TypeError: in typeassert, expected Core.Compiler.IRCode, got a value of type Nothing
Stacktrace:
 [1] #analyze_escapes#7
   @ ~/dev/EscapeAnalysis.jl/src/EscapeAnalysis.jl:637 [inlined]
 [2] analyze_escapes(f::Any, types::Any)
   @ EscapeAnalysis ~/dev/EscapeAnalysis.jl/src/EscapeAnalysis.jl:635
 [3] top-level scope
   @ REPL[3]:1

aviatesk · 2021-08-17T08:00:58Z

check the type of Some(1)

TH3CHARLie · 2021-08-17T08:06:18Z

Oh yeah I added a signature to it now it becomes:

julia> @analyze_escapes f_no_escape(Some{Any}(1))
typeof(f_no_escape)(_2::Some{Any} ↑)
1 ◌ 1 ─ %1 = Base.getfield(_2, :value)::Any
2 ↑ │   %2 = Main.typeof(%1)::DataType
3 ◌ └──      return %2

vchuravy · 2021-08-20T08:34:37Z

Sorry, only have sporadic internet here. What about: ``` @noinline function ***@***.***(x)) y = Base.inferencebarrier(x) T = typeof(y) @show T return nothing end ``` So if escape analysis can look at this function see that x is non-escaping, that would be great! Important here is that the inferred type of y is Any, but the runtime type is of course something concrete. Now codegen runs and on the caller side we optimize `x`, and on the caller side we generate code that might assume that x is heap allocated, and has a type tag stored in front of it. The way to fix this would be for IPO based heap2stack to also store the typetag in front.

…

On Mon, Aug 16, 2021, 15:55 Xuanda Yang ***@***.***> wrote: With the below example, I think the analysis pass can look through the Base.inferencebarrier call julia> @noinline f_no_escape(x) = Base.inferencebarrier(nothing) f_no_escape (generic function with 1 method) julia> @analyze_escapes f_no_escape(Ref("hah")) typeof(f_no_escape)(_2::Base.RefValue{String} ↑) 1 ◌ 1 ─ return nothing julia> @noinline f_no_escape(x) = (Base.inferencebarrier(nothing); x) f_no_escape (generic function with 1 method) julia> @analyze_escapes f_no_escape(Ref("hah")) typeof(f_no_escape)(_2::Base.RefValue{String} ↑) 1 ◌ 1 ─ return _2 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABDO2RC326O44E4XK3L4NLT5EKFBANCNFSM5ASCBAMA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email> .

fix boostrapping issues

@eval

…tional` (JuliaLang#42434) Otherwise, in rare cases, we may see this sort of weird behavior: ```julia julia> @eval edgecase(_) = $(Core.Compiler.InterConditional(2, Int, Any)) edgecase (generic function with 1 method) julia> code_typed((Any,)) do x edgecase(x) ? x : nothing end 1-element Vector{Any}: CodeInfo( 1 ─ goto #3 if not $(QuoteNode(Core.InterConditional(2, Int64, Any))) 2 ─ return x 3 ─ return Main.nothing ) => Any julia> code_typed((Any,)) do x edgecase(x) ? x : nothing end 1-element Vector{Any}: CodeInfo( 1 ─ goto #3 if not $(QuoteNode(Core.InterConditional(2, Int64, Any))) 2 ─ %2 = π (x, Int64) └── return %2 3 ─ return Main.nothing ) => Union{Nothing, Int64} ```

LilithHafner · 2022-11-02T14:18:25Z

Escape analysis is super useful for a variety of reasons. I came here from JuliaLang#7721, where better escape analysis means better automatic closing of resources.

Has this work made its way into JuliaLang/julia yet?

aviatesk · 2022-11-09T06:07:31Z

It's already merge into the Julia base. We aren't enabling it (nor using it for any optimizations) at this moment because of a latency problem though.

xref: JuliaLang#43800

aviatesk · 2022-11-09T06:07:56Z

Also https://docs.julialang.org/en/v1/devdocs/EscapeAnalysis/.

TH3CHARLie added 8 commits July 17, 2021 12:41

initial porting

aa43434

Merge branch 'master' into th3/port-to-core-compiler

107ecbe

minor comment update

9387063

Merge branch 'master' into th3/port-to-core-compiler

4e918fc

Merge branch 'master' into th3/port-to-core-compiler

f6c8ef6

update to EscapeAnalysis master

5c31701

hack to pass bootstrapping

d54cf77

Merge branch 'master' into th3/port-to-core-compiler

3e33746

TH3CHARLie commented Jul 26, 2021

View reviewed changes

src/codegen.cpp Outdated Show resolved Hide resolved

TH3CHARLie added 4 commits July 27, 2021 18:17

fix foreigncall failure

e039418

Merge branch 'master' into th3/port-to-core-compiler

c16f431

fix nargs

bc4c99b

fix comparison, now codegen log is correctly displayed

54735a6

TH3CHARLie marked this pull request as draft July 27, 2021 14:32

TH3CHARLie added 7 commits July 28, 2021 01:30

tag metadata

7b04df9

Merge branch 'master' into th3/port-to-core-compiler

8cb5841

capture metadata

58f039b

Merge branch 'master' into th3/port-to-core-compiler

41144a3

Merge branch 'master' into th3/port-to-core-compiler

f9418c2

Merge branch 'master' into th3/port-to-core-compiler

b308d03

attempt to move heap alloc to stack

17d1ccc

TH3CHARLie commented Aug 3, 2021

View reviewed changes

TH3CHARLie and others added 4 commits August 4, 2021 16:16

Merge branch 'master' into th3/port-to-core-compiler

26b3733

Merge pull request #4 from aviatesk/avi/port

3521979

fix performance problem in bootstrapping

Merge branch 'th3/port-to-core-compiler' of https://github.com/TH3CHA…

28f2662

…RLie/julia into th3/port-to-core-compiler

This comment has been minimized.

Sign in to view

TH3CHARLie marked this pull request as ready for review August 16, 2021 13:09

This comment has been minimized.

Sign in to view

TH3CHARLie changed the title ~~Port EscapeAnalysis to Core.Compiler~~ Port EscapeAnalysis to Core.Compiler with Optimization Aug 22, 2021

TH3CHARLie and others added 5 commits August 27, 2021 15:14

prepare for merge

3301dda

Merge branch 'master' into th3/port-to-core-compiler

4b4ac4c

try port latest EscapeAnalysis.jl

08fd8d2

fix boostrapping issues

bfdd250

Merge pull request #5 from aviatesk/avi/port

2f97772

fix boostrapping issues

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Port EscapeAnalysis to Core.Compiler with Optimization #3

Port EscapeAnalysis to Core.Compiler with Optimization #3

TH3CHARLie commented Jul 18, 2021

TH3CHARLie commented Jul 26, 2021 •

edited

TH3CHARLie Aug 3, 2021

TH3CHARLie commented Aug 3, 2021 •

edited

vtjnash commented Aug 3, 2021

This comment has been minimized.

TH3CHARLie commented Aug 15, 2021

vchuravy commented Aug 15, 2021

TH3CHARLie commented Aug 15, 2021

vchuravy commented Aug 15, 2021 via email

vchuravy commented Aug 15, 2021 via email

TH3CHARLie commented Aug 16, 2021

vtjnash commented Aug 16, 2021

TH3CHARLie commented Aug 16, 2021

vtjnash commented Aug 16, 2021

TH3CHARLie commented Aug 16, 2021 •

edited

This comment has been minimized.

TH3CHARLie commented Aug 16, 2021

vtjnash commented Aug 16, 2021

TH3CHARLie commented Aug 16, 2021

aviatesk commented Aug 17, 2021

TH3CHARLie commented Aug 17, 2021

aviatesk commented Aug 17, 2021

TH3CHARLie commented Aug 17, 2021

vchuravy commented Aug 20, 2021 via email

LilithHafner commented Nov 2, 2022

aviatesk commented Nov 9, 2022

aviatesk commented Nov 9, 2022

Port EscapeAnalysis to Core.Compiler with Optimization #3

Are you sure you want to change the base?

Port EscapeAnalysis to Core.Compiler with Optimization #3

Conversation

TH3CHARLie commented Jul 18, 2021

TH3CHARLie commented Jul 26, 2021 • edited

TH3CHARLie Aug 3, 2021

Choose a reason for hiding this comment

TH3CHARLie commented Aug 3, 2021 • edited

vtjnash commented Aug 3, 2021

This comment has been minimized.

TH3CHARLie commented Aug 15, 2021

vchuravy commented Aug 15, 2021

TH3CHARLie commented Aug 15, 2021

vchuravy commented Aug 15, 2021 via email

vchuravy commented Aug 15, 2021 via email

TH3CHARLie commented Aug 16, 2021

vtjnash commented Aug 16, 2021

TH3CHARLie commented Aug 16, 2021

vtjnash commented Aug 16, 2021

TH3CHARLie commented Aug 16, 2021 • edited

This comment has been minimized.

TH3CHARLie commented Aug 16, 2021

vtjnash commented Aug 16, 2021

TH3CHARLie commented Aug 16, 2021

aviatesk commented Aug 17, 2021

TH3CHARLie commented Aug 17, 2021

aviatesk commented Aug 17, 2021

TH3CHARLie commented Aug 17, 2021

vchuravy commented Aug 20, 2021 via email

LilithHafner commented Nov 2, 2022

aviatesk commented Nov 9, 2022

aviatesk commented Nov 9, 2022

TH3CHARLie commented Jul 26, 2021 •

edited

TH3CHARLie commented Aug 3, 2021 •

edited

TH3CHARLie commented Aug 16, 2021 •

edited