Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Example in Multiple GPUs doc fails #282

Closed
jeffcbecker opened this issue Jul 9, 2020 · 2 comments
Closed

Example in Multiple GPUs doc fails #282

jeffcbecker opened this issue Jul 9, 2020 · 2 comments
Labels
bug Something isn't working

Comments

@jeffcbecker
Copy link

jeffcbecker commented Jul 9, 2020

Describe the bug

A clear and concise description of what the bug is.

Tried to run first example in https://juliagpu.gitlab.io/CUDA.jl/usage/multigpu/ and it fails
To reproduce

julia> using Distributed, CUDA

julia> addprocs(length(devices()))
4-element Array{Int64,1}:
 2
 3
 4
 5

julia> asyncmap((zip(workers(), devices()))) do (p, d)
           remotecall_wait(p) do
               @info "Worker $p uses $d"
               device!(d)
           end
       end
ERROR: On worker 2:
KeyError: key CUDA [052768ef-5323-5732-b1bb-66c8b64840ba] not found
getindex at ./dict.jl:477 [inlined]
root_module at ./loading.jl:967 [inlined]
deserialize_module at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:912
handle_deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:812
deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:735
deserialize_datatype at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:1210
handle_deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:788
deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:735
deserialize_datatype at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:1235
handle_deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:790
deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:735
handle_deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:795
deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Serialization/src/Serialization.jl:735 [inlined]
deserialize_msg at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Distributed/src/messages.jl:99
#invokelatest#1 at ./essentials.jl:712 [inlined]
invokelatest at ./essentials.jl:711 [inlined]
message_handler_loop at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Distributed/src/process_messages.jl:185
process_tcp_streams at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Distributed/src/process_messages.jl:142
#97 at ./task.jl:358
Stacktrace:
 [1] (::Base.var"#726#728")(::Task) at ./asyncmap.jl:178
 [2] foreach(::Base.var"#726#728", ::Array{Any,1}) at ./abstractarray.jl:1919
 [3] maptwice(::Function, ::Channel{Any}, ::Array{Any,1}, ::Base.Iterators.Zip{Tuple{Array{Int64,1},CUDA.DeviceSet}}) at ./asyncmap.jl:178
 [4] wrap_n_exec_twice at ./asyncmap.jl:154 [inlined]
 [5] #async_usemap#711 at ./asyncmap.jl:103 [inlined]
 [6] #asyncmap#710 at ./asyncmap.jl:81 [inlined]
 [7] asyncmap(::Function, ::Base.Iterators.Zip{Tuple{Array{Int64,1},CUDA.DeviceSet}}) at ./asyncmap.jl:81
 [8] top-level scope at REPL[17]:1

julia> 

The Minimal Working Example (MWE) for this bug:

<details><summary>Manifest.toml</summary>
<p>

Paste your Manifest.toml here, or accurately describe which version of CUDA.jl and its dependencies (GPUArrays.jl, GPUCompiler.jl, LLVM.jl) you are using.

(@v1.4) pkg> status
Status `~/.julia/environments/v1.4/Project.toml`
  [6e4b80f9] BenchmarkTools v0.5.0
  [052768ef] CUDA v0.1.0
  [c5f51814] CUDAdrv v6.3.0
  [be33ccc6] CUDAnative v3.0.4
  [5ae59095] Colors v0.12.3
  [3a865a2d] CuArrays v2.2.2
  [5789e2e9] FileIO v1.3.0
  [0c68f7d7] GPUArrays v3.4.1
  [4e3cecfd] ImageShow v0.2.3
  [d0351b0e] InspectDR v0.3.7
  [da04e1cc] MPI v0.15.0
  [3b7a836e] PGFPlots v3.2.1
  [8314cec4] PGFPlotsX v1.2.8
  [91a5bcdd] Plots v1.5.3
  [d330b81b] PyPlot v2.9.0
  [b8865327] UnicodePlots v1.2.0

</p>
</details>

Expected behavior

A clear and concise description of what you expected to happen.

example should run, not give stack trace

Version info

Details on Julia:

please post the output of:

versioninfo()

Julia Version 1.4.2
Commit 44fa15b150* (2020-05-23 18:35 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Xeon(R) Gold 6154 CPU @ 3.00GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, skylake)

Details on CUDA:

please post the output of:

CUDA.versioninfo()

v"10.2.0"

Additional context

Add any other context about the problem here.

@jeffcbecker jeffcbecker added the bug Something isn't working label Jul 9, 2020
@kshyatt
Copy link
Contributor

kshyatt commented Jul 10, 2020

Fixed the backticks so code should be a little more readable

@maleadt
Copy link
Member

maleadt commented Jul 10, 2020

Should include @everywhere using CUDA

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants