Backward looking macros: Feature Request 2.0

[pdeffebach]

This has been discussed multiple times on Slack and Discord, but I want to file an issue here so it can be formally tracked.

Something I would really like in Julia is "backwards looking macros", broadly similar to R's %>% operator magic. But don't worry, this is not yet another piping thread.

I really like Chain.jl and use it all the time. ie. @chain x f1(a) f2(b) becomes f2(f1(x, a), b). The only problem is that it's hard to work with interactively. Often times I write x first and then only later realize I need to do something to it. In R, a frequent pattern I have is

R> f(a, b, c, df %>% filter(x == 1))

where the last filter is added because I realized I only wanted to consider observations where x == 1 in the operation. But I realized this after I had already been typing.

I recognize there is InfixFunctions.jl , but it lacks the same flexibility as metaprogramming.

I do not know how complicated this would be to implement. But my preferred syntax would be something along the lines of

backwards macro xx(args...)
    ...
end

which is exactly like a normal macro, but it captures the preceding element. as well. In this way

y @_xx_backwards a b c

behaves exactly the same as

@xx(y, a, b, c)

Thank you!

[aplavin]

This is indeed a common usecase, but is already possible with the current macro system without introducing extra syntax with potentially complex rules.
For example, with DataPipes:

julia> using DataPipes
julia> tbl = [(a=1, b=2), (a=3, b=4)]

# regular pipe - macro in front
julia> @p tbl |> filter(_.a > 2) |> map(_.b)
1-element Vector{Int64}:
 4

# wrote tbl and later realized that extra steps are needed - macro after tbl before those steps
julia> tbl |> @f filter(_.a > 2) |> map(_.b)
1-element Vector{Int64}:
 4

I often use this syntax when working with tables interactively, feels pretty convenient so far.

Underscores.jl also has a similar feature.

---

And as for filter specifically, in the next julia release it will be possible to write f(a, b, c, tbl |> filter(r -> r.x == 1)) without any macro whatsoever.

[pdeffebach]

This is not exactly the same though. Here @f creates an anonymous function, which has a small overhead in interactive work. It also requires more notation, |> and @f, so I still think this would be a valuable feature to have.

[uniment]

Other than providing a chaining syntax à la magrittr, what benefits would such a feature bring?

[pdeffebach]

Binary operators are nice to type in general I suppose. I'm glad we aren't lisp having to write +(a, b, -(c, d))). It's a nice thing to type.

[uniment]

I agree that infix operators are nice. They also necessitate the complexity of operator precedence rules.

I can see how infix macro support is useful for magrittr, but that's only because I'm familiar with the chaining problem. I'm too unimaginative to consider how it could be useful in other contexts though. Do you have any examples?

[uniment]

issue #11608 is worth reading btw. There was a short window of time when ~ was a special infix macro; the question arose, whether to generalize it or remove it. The latter was chosen.

[mbauman]

Surely I must be missing something here — is your preferred calling syntax alternative to @xx(a, b, c) really a @_xx_backwards b c?

[aplavin]

This is not exactly the same though.
Here @f creates an anonymous function, which has a small overhead in interactive work.

Overhead is indeed small, and only when you type new code - running it again is instant.

It also requires more notation, |> and @f, so I still think this would be a valuable feature to have.

|> is already the standard Julia syntax for piping, hard to think of a better fit to your example.

Wrt @f, compare

f(a, b, c, @p tbl |> filter(_.x == 1))

with macro in front vs

f(a, b, c, tbl |> @f filter(_.x == 1))

with macro in the middle. That's exactly the same amount of notation. Further, @p and @f can be made the same macro with some compromise, see eg Underscores.jl.

[pdeffebach]

Surely I must be missing something here — is your preferred calling syntax alternative to @xx(a, b, c) really a @_xx_backwards b c?

No, definitely not!

I don't have a concrete syntax proposal. But something like x |mymacro| expr might be nice.

w.r.t. Piping that automatically creates functions. I have a strong preference towards a feature which only transforms expressions, without necessarily creating a new scope, since it allows a lot more flexibility, importantly the ability to create variables inside a macro block which are visible outside of it.

[aplavin]

I have a strong preference towards a feature which only transforms expressions

Sure, I also prefer that. But the minor inconvenience of requiring the macro writer to create a function vs a feature that makes code reading nonlinear and more complicated?.. I would definitely choose the former!

since it allows a lot more flexibility, importantly the ability to create variables inside a macro block which are visible outside of it.

Well, it's possible with the existing piping syntax:

# the begin-end expression presumably generated by macro
1:5 |> begin
	y = nothing
	function (x)
		y = filter(>=(3), x)
	end
end

# y == [3, 4, 5]
# unless run from toplevel REPL - arguments can be made to make it more consistent

But why? The presented usecase is specifically for short pipelines where it's too much hassle interactively to move the cursor in front. Maybe, best to keep such pipes at least side effect-free?