New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

at-dot macro for adding dots to function calls #20321

Merged
merged 11 commits into from Feb 2, 2017

Conversation

Projects
None yet
6 participants
@stevengj
Member

stevengj commented Jan 30, 2017

The PR adds a new macro @. (and parser support to convert this to @__DOTS__ @__DOT__ @__dot__) that adds dots to every function call, operator, and assignment in an expression. For example,

@. x += 3x^2 - sqrt(5x)

is equivalent to x .+= 3 .* x.^2 .- sqrt.(5 .* x).

You can opt-out of the dots for specific function calls by splicing with $, e.g. @. sqrt($sort(abs(x))) is equivalent to sqrt.(sort(abs.(x))). (This also works for function objects spliced directly into expressions by other means, e.g. it allows @. to work in conjunction with @views, and it should similarly allow some interoperability with other macros.)

cc @StefanKarpinski, since @. was (I think?) his suggestion.

@stevengj stevengj added the broadcast label Jan 30, 2017

@stevengj stevengj requested a review from StefanKarpinski Jan 30, 2017

@nalimilan

This comment has been minimized.

Show comment
Hide comment
@nalimilan

nalimilan Jan 30, 2017

Contributor

The feature is really cool, but I wonder whether a more descriptive name wouldn't be in order. @broadcast?

Contributor

nalimilan commented Jan 30, 2017

The feature is really cool, but I wonder whether a more descriptive name wouldn't be in order. @broadcast?

Show outdated Hide outdated doc/src/stdlib/arrays.md
```@docs
Base.broadcast
Base.Broadcast.broadcast!
Base.__DOTS__

This comment has been minimized.

@tkelman

tkelman Jan 30, 2017

Contributor

we want to document the macro, not the helper function, right?

@tkelman

tkelman Jan 30, 2017

Contributor

we want to document the macro, not the helper function, right?

This comment has been minimized.

@stevengj

stevengj Jan 31, 2017

Member

Right, whoops.

@stevengj

stevengj Jan 31, 2017

Member

Right, whoops.

Show outdated Hide outdated base/exports.jl
@@ -1382,6 +1382,7 @@ export
@polly,
@assert,
@__DOTS__,

This comment has been minimized.

@tkelman

tkelman Jan 30, 2017

Contributor

why export this if only lowering should see it under this name?

@tkelman

tkelman Jan 30, 2017

Contributor

why export this if only lowering should see it under this name?

@stevengj

This comment has been minimized.

Show comment
Hide comment
@stevengj

stevengj Jan 30, 2017

Member

I'm of two minds about the name.

  • If it were an ordinary macro name, I would prefer @dots (to emphasize that it turns things into fusing dot calls, not just broadcast calls). This has the advantage of not requiring any parser sugar.

  • @. is shorter and thus more lightweight and tempting to insert into code. And the specialness of the macro name is maybe reflective of the specialness of the dot-call sugar?

@StefanKarpinski, you suggested the @. syntax, do you have any further thoughts on why you preferred that over an ordinary macro name?

Member

stevengj commented Jan 30, 2017

I'm of two minds about the name.

  • If it were an ordinary macro name, I would prefer @dots (to emphasize that it turns things into fusing dot calls, not just broadcast calls). This has the advantage of not requiring any parser sugar.

  • @. is shorter and thus more lightweight and tempting to insert into code. And the specialness of the macro name is maybe reflective of the specialness of the dot-call sugar?

@StefanKarpinski, you suggested the @. syntax, do you have any further thoughts on why you preferred that over an ordinary macro name?

@stevengj

This comment has been minimized.

Show comment
Hide comment
@stevengj

stevengj Jan 30, 2017

Member

(The other thing to keep in mind with @dots vs. @broadcast is that, going forward, I expect that only a minority of Julia users will call broadcast directly. More people will be familiar with the dot syntax — indeed, more people already are for operators, thanks to Matlab.)

Member

stevengj commented Jan 30, 2017

(The other thing to keep in mind with @dots vs. @broadcast is that, going forward, I expect that only a minority of Julia users will call broadcast directly. More people will be familiar with the dot syntax — indeed, more people already are for operators, thanks to Matlab.)

@StefanKarpinski

This comment has been minimized.

Show comment
Hide comment
@StefanKarpinski

StefanKarpinski Jan 30, 2017

Member

My thinking was that @. will be the easiest to remember: "I have all these dots all over the place, gee, isn't there a way to avoid those... oh yeah just write @. at the beginning. That was easy!" Otherwise people need to remember what name replaces all the dots. Keep in mind that many users will use f.() syntax and .+ operators without ever knowing or caring that they're implemented as higher order calls to broadcast. To them it will just be mysterious why one uses @broadcast to put dots on everything. There's also the advantage that @. can't possibly collide with any macro name.

Member

StefanKarpinski commented Jan 30, 2017

My thinking was that @. will be the easiest to remember: "I have all these dots all over the place, gee, isn't there a way to avoid those... oh yeah just write @. at the beginning. That was easy!" Otherwise people need to remember what name replaces all the dots. Keep in mind that many users will use f.() syntax and .+ operators without ever knowing or caring that they're implemented as higher order calls to broadcast. To them it will just be mysterious why one uses @broadcast to put dots on everything. There's also the advantage that @. can't possibly collide with any macro name.

@StefanKarpinski

This comment has been minimized.

Show comment
Hide comment
@StefanKarpinski

StefanKarpinski Jan 30, 2017

Member

I see that you just said effectively the same thing, @stevengj :)

Member

StefanKarpinski commented Jan 30, 2017

I see that you just said effectively the same thing, @stevengj :)

@StefanKarpinski

This comment has been minimized.

Show comment
Hide comment
@StefanKarpinski

StefanKarpinski Jan 30, 2017

Member

Wouldn't it be cleaner to just allow the parser to recognize @. as a valid macro and internalize it as calling the $(Symbol("@.")) expander function?

Member

StefanKarpinski commented Jan 30, 2017

Wouldn't it be cleaner to just allow the parser to recognize @. as a valid macro and internalize it as calling the $(Symbol("@.")) expander function?

@stevengj

This comment has been minimized.

Show comment
Hide comment
@stevengj

stevengj Jan 31, 2017

Member

@StefanKarpinski, there is a lot of code that assumes a certain form for identifiers. e.g. the documentation system broke when I tried to attach a docstring to @. ...I could get it to work, but I was worried that I'd have to insert special-case code all over the place. The macro seems a lot easier to work with if it is just sugar for a normal Julia identifier.

@tkelman, the reason I export @__DOTS__ and have @. expand to the unqualified @__DOTS__ (rather than to Base.@__DOTS__) is that this way you can override @. inside a module by defining your own __DOTS__ macro, which seemed to add flexibility. (That's also why the __DOTS__ name is documented in the @. docstring.)

Member

stevengj commented Jan 31, 2017

@StefanKarpinski, there is a lot of code that assumes a certain form for identifiers. e.g. the documentation system broke when I tried to attach a docstring to @. ...I could get it to work, but I was worried that I'd have to insert special-case code all over the place. The macro seems a lot easier to work with if it is just sugar for a normal Julia identifier.

@tkelman, the reason I export @__DOTS__ and have @. expand to the unqualified @__DOTS__ (rather than to Base.@__DOTS__) is that this way you can override @. inside a module by defining your own __DOTS__ macro, which seemed to add flexibility. (That's also why the __DOTS__ name is documented in the @. docstring.)

@StefanKarpinski

This comment has been minimized.

Show comment
Hide comment
@StefanKarpinski

StefanKarpinski Jan 31, 2017

Member

Makes sense. It's definitely a nit, but should this be called @__dot__ instead? Singular because it corresponds to the syntax @. which has only a single dot. This happens to make it correspond, but default, to a behavior which puts many dots on things, but that might not be what other people define it to do. In generally, I've found that syntactic features with special names, like getindex and setindex! end up being less confusing if the special names correspond to the syntax rather than the default meaning of that syntax. The lowercase part is because we've usually used special names like __init__ have been lowercase. Otoh, we have @__FILE__ and @__DIR__, so that could be argued either way – I just thought we should be cognizant of the choice.

Member

StefanKarpinski commented Jan 31, 2017

Makes sense. It's definitely a nit, but should this be called @__dot__ instead? Singular because it corresponds to the syntax @. which has only a single dot. This happens to make it correspond, but default, to a behavior which puts many dots on things, but that might not be what other people define it to do. In generally, I've found that syntactic features with special names, like getindex and setindex! end up being less confusing if the special names correspond to the syntax rather than the default meaning of that syntax. The lowercase part is because we've usually used special names like __init__ have been lowercase. Otoh, we have @__FILE__ and @__DIR__, so that could be argued either way – I just thought we should be cognizant of the choice.

@stevengj

This comment has been minimized.

Show comment
Hide comment
@stevengj

stevengj Jan 31, 2017

Member

I'm fine with @__dot__ or @__DOT__. I made it uppercase to correspond to the other "special" macros, but I don't have any strong preference.

Member

stevengj commented Jan 31, 2017

I'm fine with @__dot__ or @__DOT__. I made it uppercase to correspond to the other "special" macros, but I don't have any strong preference.

@tkelman

This comment has been minimized.

Show comment
Hide comment
@tkelman

tkelman Jan 31, 2017

Contributor

Lowercase would be preferable to me as it behaves more like a function modifier, and less like something from the C preprocessor which many of the other dunder-caps macros behave like.

Contributor

tkelman commented Jan 31, 2017

Lowercase would be preferable to me as it behaves more like a function modifier, and less like something from the C preprocessor which many of the other dunder-caps macros behave like.

stevengj added some commits Jan 31, 2017

@StefanKarpinski

Bravo. That's a lovely PR.

Expr(x.head, dotargs...)
end
end
end

This comment has been minimized.

@StefanKarpinski

StefanKarpinski Jan 31, 2017

Member

I should remember in the future that this is a perfect example of why you cannot do serious metaprogramming with strings and why you need data structures for it – a complex, recursive syntax transformation.

@StefanKarpinski

StefanKarpinski Jan 31, 2017

Member

I should remember in the future that this is a perfect example of why you cannot do serious metaprogramming with strings and why you need data structures for it – a complex, recursive syntax transformation.

Show outdated Hide outdated doc/src/stdlib/arrays.md
All mathematical operations and functions are supported for arrays
See also the [dot syntax for vectorizing functions](@ref man-vectorized);
for example, `f.(args...)` implicitly calls `broadcast(f, args...)`.
Rather than relying on "vectorized" methods of function like `sin`

This comment has been minimized.

@Sacha0

Sacha0 Jan 31, 2017

Member

Should "of function like sin" be "of functions like sin"?

@Sacha0

Sacha0 Jan 31, 2017

Member

Should "of function like sin" be "of functions like sin"?

@@ -244,6 +247,28 @@ let x = [1:4;]
@test sin.(f17300kw.(x, y=1)) == sin.(f17300kw.(x; y=1)) == sin.(x .+ 1)
end
# splice escaping of @.
let x = [4, -9, 1, -16]
@test [2, 3, 4, 5] == @.(1 + sqrt($sort(abs(x))))

This comment has been minimized.

@Sacha0

Sacha0 Jan 31, 2017

Member

Inconsequential, but missing a space between the @. and following expression?

@Sacha0

Sacha0 Jan 31, 2017

Member

Inconsequential, but missing a space between the @. and following expression?

This comment has been minimized.

@ararslan

ararslan Jan 31, 2017

Member

If I'm not mistaken, I think in this case the macro is just being called with the parenthesized form, in which case there shouldn't be a space after @..

@ararslan

ararslan Jan 31, 2017

Member

If I'm not mistaken, I think in this case the macro is just being called with the parenthesized form, in which case there shouldn't be a space after @..

This comment has been minimized.

@Sacha0

Sacha0 Jan 31, 2017

Member

Ah, of course! Thanks for straightening me out :).

@Sacha0

Sacha0 Jan 31, 2017

Member

Ah, of course! Thanks for straightening me out :).

Show outdated Hide outdated test/broadcast.jl
@test y === x == [7,6,5,4]
x[1:2] .= 1
@test y === x == [1,1,5,4]
x[1:2] .+= [2,3]
@. x[1:2] .+= [2,3]

This comment has been minimized.

@Sacha0

Sacha0 Jan 31, 2017

Member

Did you leave the dots in place here and just below to exercise @. when dots already exist, or did you mean to remove the dots here and just below? (A comment or two on these tests explaining your intention might help those who come after?)

@Sacha0

Sacha0 Jan 31, 2017

Member

Did you leave the dots in place here and just below to exercise @. when dots already exist, or did you mean to remove the dots here and just below? (A comment or two on these tests explaining your intention might help those who come after?)

Show outdated Hide outdated test/subarray.jl
@@ -479,7 +479,7 @@ Y = 4:-1:1
@test isa(@view(X[1:3]), SubArray)
@test X[1:end] == @view X[1:end]
@test X[1:end] == @. @view X[1:end]

This comment has been minimized.

@Sacha0

Sacha0 Jan 31, 2017

Member

Was your intention here to exercise the interaction of @. with @view? If so, perhaps add a comment to that effect?

@Sacha0

Sacha0 Jan 31, 2017

Member

Was your intention here to exercise the interaction of @. with @view? If so, perhaps add a comment to that effect?

This comment has been minimized.

@ararslan

ararslan Jan 31, 2017

Member

Or even better, put it in a testset. 🙂

@ararslan

ararslan Jan 31, 2017

Member

Or even better, put it in a testset. 🙂

@Sacha0

Sacha0 approved these changes Jan 31, 2017

This looks great! (Review regrettably brief.)

stevengj added some commits Jan 31, 2017

@stevengj

This comment has been minimized.

Show comment
Hide comment
@stevengj

stevengj Feb 2, 2017

Member

AppVeyor failure seems like an unrelated network glitch: Cannot clone ColorTypes from https://github.com/JuliaGraphics/ColorTypes.jl.git. Failed to receive response: The server returned an invalid or unrecognized response

Member

stevengj commented Feb 2, 2017

AppVeyor failure seems like an unrelated network glitch: Cannot clone ColorTypes from https://github.com/JuliaGraphics/ColorTypes.jl.git. Failed to receive response: The server returned an invalid or unrecognized response

@stevengj

This comment has been minimized.

Show comment
Hide comment
@stevengj

stevengj Feb 2, 2017

Member

Should be good to merge?

Member

stevengj commented Feb 2, 2017

Should be good to merge?

@StefanKarpinski StefanKarpinski merged commit 2e0290a into JuliaLang:master Feb 2, 2017

1 of 2 checks passed

continuous-integration/appveyor/pr AppVeyor build failed
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@tkelman

This comment has been minimized.

Show comment
Hide comment
@tkelman

tkelman Feb 2, 2017

Contributor

nevermind, dunno what I was looking at

Contributor

tkelman commented Feb 2, 2017

nevermind, dunno what I was looking at

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment