byrow problem (no multithreading?)

[minhuangr]

I have tried the example below on Win10 and Unbuntu 18 under WSL2 with Julia 1.7.1. Results are similar.

julia> Threads.nthreads()
8

julia> using InMemoryDatasets, BenchmarkTools

julia> ds = Dataset(rand(10^5, 100), :auto);

julia> m = Matrix(ds);

julia> @Btime byrow(ds, sum, 1:100);
22.097 ms (171 allocations: 889.34 KiB)

julia> @Btime sum(m, dims = 2);
15.696 ms (6 allocations: 879.06 KiB)

julia> @Btime byrow(ds, sum, 1:100, threads = true);
22.181 ms (168 allocations: 889.17 KiB)

[sl-solution]

Based on allocations, it seems that no parallel computation is happening. Would you please run the following code and share the timing for n=8?

julia> function f1(n)
           for i in 1:n
               sleep(1)
           end
       end
f1 (generic function with 1 method)

julia> function f2(n)
           Threads.@threads for i in 1:n
               sleep(1)
           end
       end
f2 (generic function with 1 method)

[minhuangr]

It seems to be fine for this.

julia> @Btime f2(8)
1.007 s (81 allocations: 5.98 KiB)

julia> @Btime f1(8)
8.072 s (40 allocations: 1.12 KiB)

[sl-solution]

It seems to be fine for this.

julia> @Btime f2(8) 1.007 s (81 allocations: 5.98 KiB)

julia> @Btime f1(8) 8.072 s (40 allocations: 1.12 KiB)

Is IMD loaded when you were benchmarking? If not, would you please load and rerun?

[minhuangr]

I used the REPL running your byrow example (I did not close it). I have also closed the REPL to rerun it with IMD loaded. After restarting the machine, I tried again. Unfortunately, the results are the same. I think Ubuntu under WSL2 should behave like a linux system. So the problem may not be peculiar to WIN10.

[minhuangr]

julia> Threads.nthreads()
8

julia> using InMemoryDatasets, BenchmarkTools

julia> ds = Dataset(rand(10^5, 100), :auto);

julia> m = Matrix(ds);

julia> @btime byrow(ds, sum, 1:100);
  22.274 ms (171 allocations: 889.34 KiB)

julia> @btime sum(m, dims = 2);
  15.808 ms (6 allocations: 879.06 KiB)

julia>

julia> function f1(n)
           for i in 1:n
               sleep(1)
           end
       end
f1 (generic function with 1 method)

julia>

julia> function f2(n)
           Threads.@threads for i in 1:n
               sleep(1)
           end
       end
f2 (generic function with 1 method)

julia>

julia> @btime f2(8)
  1.004 s (81 allocations: 5.98 KiB)

julia> @btime f1(8)
  8.063 s (40 allocations: 1.12 KiB)

julia> Threads.nthreads()
8

[sl-solution]

The last thing you should check is the value of IMD.__NCORES, since the byrow behind the scene uses Threads.@threads and as you checked there is no problem with Threads.@threads in your system.

One thing you may check to be sure that@btime byrow(ds, sum, 1:100); is not parallelised is by looking at system monitor.

[minhuangr]

I have used htop to find that there is only one process working there.

Yep, IMD.__NCORES is 1. Is there a way to fix it?

[sl-solution]

It is defined in the following address:

InMemoryDatasets.jl/src/byrow/row_functions.jl

Line 19 in 209c72e

const __NCORES = Threads.nthreads()

As you can see it is simply equals to Threads.nthreads() - So if Threads.nthreads() shows 8 in your session it also should be set to 8 in IMD, unless you run a code which doesn't allow IMD to set it as 8.

[minhuangr]

Emm, this is quite strange. I activate a new environment and only install InMemoryDatasets but IMD.__NCORES is still 1 though Threads.nthreads() is 8.

[sl-solution]

Another thing which I am not sure and I should check. Maybe the first time you run IMD, Julia started with 1 threads and Julia precompiled __NCORES as 1 and stuck with it. If this is the case then it is a bug and I should fix it.

to check this, start Julia with multiple cores and create a activate a temp environment, install IMD fresh in there and recheck byrow

] activate temp
] add InMemoryDatasets

[sl-solution]

I created a debug branch, which I replaced every occurrence of __NCORES with Threads.nthreads(). You may check this branch and see if every thing is ok?

[minhuangr]

Maybe you need to modify byrow.jl as well.

[minhuangr]

I have replaced __NCORES in byrow.jl and now everthing works.

[sl-solution]

Great!

So this is a bug. I will fix it