-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fusion of nested f.(args) calls into a single broadcast call #17300
Conversation
Note that this is still an early preview. I literally committed the instant |
There are also some potential optimizations that I haven't implemented. For example, if you do Are there any other arguments besides numeric literals than can be "inlined" in this way? |
Performance is looking good: f(x) = x + 1
f1(x) = broadcast(f, broadcast(f, broadcast(f, x)))
f2(x) = [f(f(f(x))) for x in x]
f3(x) = f.(f.(f.(x)))
@time f1(x)
@time f2(x)
@time f3(x) gives (after the usual repetitions):
i.e. it is doing about as well as manual loop fusion. |
Wait... If you add that now, what feature are we going to introduce next to justify another release after 0.5? ;-) |
It's not clear that we should do this now. 0.5 is |
@Keno, that's true. On the other hand, this feature is relatively non-disruptive because the |
Will need a rebase and some fixes after #17307. |
...and it will change the result when applied to non-pure functions. So either merge this PR for 0.5, or put a big warning in the docs. |
Calls to |
Not meaning to ruin anyones day, but with the julia> log.(log.(log.([1000]))) # if this
1-element Array{Float64,1}:
0.658889
julia> broadcast(x -> log(log(log(x))), [1000]) # is rewritten to this
ERROR: DomainError:
in nan_dom_err at ./math.jl:0 [inlined]
in log(::Float64) at ./math.jl:140
in (::##1#2)(::Int64) at ./REPL[15]:1
in promote_op at ./number.jl:0 [inlined]
in promote_eltype_op at ./abstractarray.jl:0 [inlined]
in broadcast(::Function, ::Array{Int64,1}) at ./broadcast.jl:197
in eval(::Module, ::Any) at ./boot.jl:234
in macro expansion at ./REPL.jl:92 [inlined]
in (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at ./event.jl:46 This is not meant as an argument against this PR though, but rather as motivation to rethink that fallback... |
Damn fallback. We really need to use type inference instead of actually calling the function on |
+100 for this PR
@stevengj I'm not sure of their current status (I only checked a couple weeks ago), but can we at minimum have a default (fallback) definition of
@Keno - it may be a performance question, but this change is totally kick-ass. I was disappointed when I first switched to Julia reading the improvements over MATLAB in speed to find out that the same memory mistakes MATLAB made for the kind of work I was doing at the time (large, memory-limited linear algebra problems) also existed in Julia. Of course, there was Devectorize.jl and other tricks, but for a newbie it was difficult to know what to do. My personal opinion was always that reducing the memory overhead for Julia's core, ex-MATLAB audience should be one of the priorities. On the other hand, you definitely make valid points about rushed features! (In fact, I think changing the parsing of |
@martinholters Maybe a reason not to throw so much julia> import NaNMath
julia> broadcast(x -> NaNMath.log(NaNMath.log(NaNMath.log(x))), [1000])
1-element Array{Float64,1}:
0.658889 |
@andreasnoack, please continue that discussion at #17314, since it is orthogonal to this PR. |
Moreover, *nested* ``f.(args...)`` calls are *fused* into a single ``broadcast`` | ||
loop. For example, ``sin.(cos.(X))`` is equivalent to ``broadcast(x -> sin(cos(x)), X)``, | ||
similar to ``[sin(cos(x)) for x in X]``: there is only a single loop over ``X``, | ||
and a single array is allocated for the result. [In contrast, ``sin(cos(X))`` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I hope brackets don't have special meaning in rst
I don't think we use square brackets for parenthetical comments elsewhere in the docs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The usual typographical convention is to use square brackets for parenthetical comments that have nested parentheses.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Huh, I haven't come across that convention. Do you have a citation for it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In math, the usual style is to nest parens as {[(...)]}
. In text, it is common to recommend square inside round parens if you must nest, but that isn't possible here because the nested parens are code and hence are constrained by Julia syntax.
http://blog.apastyle.org/apastyle/2013/05/punctuation-junction-parentheses-and-brackets.html
http://www.chicagomanualofstyle.org/16/ch12/ch12_sec026.html
http://www.chicagomanualofstyle.org/16/ch06/ch06_sec099.html
Another subtlety: If julia> Number.(sin.([0 pi/2]))
1×2 Array{Number,2}:
0.0 1.0
julia> broadcast(x -> Number(sin(x)), [0.0 pi/2])
1×2 Array{Float64,2}:
0.0 1.0 One may argue whether the |
As @JeffBezanson remarked, |
67f8239
to
03ea7e0
Compare
(Test failure should be fixed by #17318.) |
Yes, well, but without any dependencies on the compiler's mood, so more predictable across Julia versions. But if we could get rid of |
I would mention that this is not just a performance issue: if we release |
So if this can be made ready this week, I think we should consider doing it now. |
…f they are the last argument in the fusion)
Okay, dot calls with keyword arguments are now handled. (They are not handled correctly on If the tests pass, this should be ready to merge as far as I can tell right now. |
Some impressive sexpr-slinging in here. :) |
It's always a questionable sign if you find yourself wishing that |
Thanks for finishing this on time, very, very cool that this made it for 0.5!!! |
Does it makes sense that: x = rand(10000)
@time sin(cos(x)); # 0.000243 seconds (12 allocations: 156.625 KB)
@time sin.(cos.(x)); # 0.012385 seconds (6.85 k allocations: 386.261 KB)
@time broadcast(xval->sin(cos(xval)), x); # 0.012997 seconds (6.77 k allocations: 382.771 KB)
@time sin.(cos.(x) .+ 0) # 0.000325 seconds (50 allocations: 235.953 KB) seems to be a strange speed regression on my machine using the dotted syntax if the fusion kicks in (what I imagine is caused by this, hence the last example trying to use the Is there a way to see what the lowered code looks like from the REPL? |
The |
Note that each line that uses the dot function call syntax generates a new anonymous function. You need to wrap it in a function to avoid benchmarking compiler |
Timings look good when wrapped in a function: x = rand(10^4)
f1(x) = sin(cos(x))
f2(x) = sin.(cos.(x))
@time f1(x); @time f2(x);
...repeat a few times...
@time f1(x); @time f2(x);
0.000407 seconds (12 allocations: 156.625 KB)
0.000370 seconds (15 allocations: 78.500 KB) In this case, the |
Nice! I have to say I'm impressed by all of this work 👍 Out of curiosity, will |
No, I think dot operators will be left as-is for 0.5. (In consequence, this loop fusion probably won't have a big practical impact until 0.6. But it is good to get at least a preview in 0.5.) |
Ohhhh I see. It is the anonymous function call in the global scope, versus having Thanks so much for the clarification. I am so excited for this feature (the |
Yeah, it would be nice if the compiler were smart enough to re-use identical anonymous functions. |
That leads me to predict you'll see some packages using code like It will be a super-nice syntax change, whenever it lands. :) |
@andyferris as @StefanKarpinski has mentioned, in another issue, it would be really great if we could find a way for Julia to slap you if you have a tendency to write too much code that way... I just hope this is all that happens and they don't also use 0-based arrays on top of such abominations. |
lol :) |
As discussed in #16285 and suggested by @yuyichao, this PR implements fusion of nested
f.(args)
calls into a singlebroadcast
loop. For example,sin.(cos.(x))
is transformed by the parser intobroadcast(x -> sin(cos(x)), x)
.Because this is a syntactic guarantee, there is no need to worry about side effects or function purity, so it is very different from loop fusion viewed as a compiler optimization.
To do:
func.(literals...)
.Note that operators like
.+
are still handled separately, but the long-term plan in #16285 is to treat dot operators as.(
calls.