@threads in array comprehension

[samo-lin]

This is a feature suggestion. It would be nice it the @threads macro would work also inside array comprehensions as:

[i @threads for i=1:4]

I.e. that example should produce the same result as:

julia> [i for i=1:4]
4-element Array{Int64,1}:
 1
 2
 3
 4

Yet, currently it is not supported:

julia> [i @threads for i=1:4]
ERROR: syntax: invalid comprehension syntax

See the corresponding topic in Julia discourse.

[JeffBezanson]

Yes, this could be implemented even now by expanding to a @threads for loop. Though the syntax will need to be @threads [i for i=1:4]. Macros can only see expressions that are passed to them, and the for i=1:4 part is not sufficient to emit the needed code (for i=1:4 also can't be parsed by itself, so there wouldn't be anything to pass to the macro, but that's a detail).

It's difficult to support every feature of comprehensions though. Initially we could only do known-length iterators and cases where the result type is easy to determine.

[samo-lin]

It surely would be a great feature even if, initially, it could only be done for simple cases. Array comprehensions are just so beautifully compact, and yet readable. Maybe then @threads could take a keyword argument e.g. something like

@threads indices=(i, j) [i for i=1:4, j=1:3, k=1:5]

?

[samo-lin]

In the Julia discourse thread someone brought up the topic of broadcast. I believe this could maybe be analogically done like this

@threads dims=(1,2) B .= sin.(A) .+ cos.(A);

assuming A and B are both arrays with 2 or more dimensions.

[stevengj]

See #19777 for broadcast.

[stakaz]

Also maybe stuff like this would be very handy:

y = sum(@threads f(x) for x in X)

or at least the same syntax as for @distributed would be great:

y = @threads (+) for x in X
    f(x)
end

Generally the syntax is not that important at all, the functionality is what matters here, I would say.

[tkf]

FYI, with ThreadsX.jl you can do ThreadsX.collect(f(x) for x in xs) and ThreadsX.sum(f(x) for x in xs) (no macros!). See also FLoops.jl for parallel for loop with very flexible reduction syntax.

[stillyslalom]

For multidimensional array comprehensions, ThreadsX.map/map! works with Iterators.product:

julia> ThreadsX.map(Iterators.product(1:3, 10:10:30)) do (a, b)
           (a + b, Threads.threadid())
       end
3×3 Array{Tuple{Int64,Int64},2}:
 (11, 1)  (21, 8)  (31, 7)
 (12, 1)  (22, 2)  (32, 6)
 (13, 3)  (23, 8)  (33, 4)

[User-764Q]

Is this still an issue? I see. a number of options discussed in the comments but no activity for around a year.

Thanks.