Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some test failures with A770 #302

Closed
fredlarochelle opened this issue Mar 29, 2023 · 6 comments
Closed

Some test failures with A770 #302

fredlarochelle opened this issue Mar 29, 2023 · 6 comments
Labels
bug Something isn't working

Comments

@fredlarochelle
Copy link

fredlarochelle commented Mar 29, 2023

Running the oneAPI.jl release v1.0.2 with Julia 1.8.5 on Fedora 37 server (kernel 6.2.8), I get a couple of test failures when running pkg> test oneAPI like in the README. The CPU is an Intel Xeon E5-2695 V3 with 128gb of ram and the GPU is an Intel Arc A770 16gb. Also, can the test suite use more than 1 core?

(@v1.8) pkg> test oneAPI
     Testing oneAPI
      Status `/tmp/jl_mnmtha/Project.toml`
  [79e6a3ab] Adapt v3.6.1
  [7a1cc6ca] FFTW v1.6.0
  [0c68f7d7] GPUArrays v8.6.5
  [90137ffa] StaticArrays v1.5.19
  [8f75cd03] oneAPI v1.0.2
  [b049733a] oneAPI_Support_jll v0.2.1+0
  [ade2ca70] Dates `@stdlib/Dates`
  [8ba89e20] Distributed `@stdlib/Distributed`
  [37e2e46d] LinearAlgebra `@stdlib/LinearAlgebra`
  [de0858da] Printf `@stdlib/Printf`
  [3fa0cd96] REPL `@stdlib/REPL`
  [9a3f8284] Random `@stdlib/Random`
  [10745b16] Statistics `@stdlib/Statistics`
  [8dfed614] Test `@stdlib/Test`
      Status `/tmp/jl_mnmtha/Manifest.toml`
  [621f4979] AbstractFFTs v1.3.1
  [79e6a3ab] Adapt v3.6.1
  [fa961155] CEnum v0.4.2
  [d360d2e6] ChainRulesCore v1.15.7
  [9e997f8a] ChangesOfVariables v0.1.6
  [34da2185] Compat v4.6.1
  [ffbed154] DocStringExtensions v0.9.3
  [e2ba6199] ExprTools v0.1.9
  [7a1cc6ca] FFTW v1.6.0
  [0c68f7d7] GPUArrays v8.6.5
  [46192b85] GPUArraysCore v0.1.4
⌅ [61eb1bfa] GPUCompiler v0.17.3
  [3587e190] InverseFunctions v0.1.8
  [92d709cd] IrrationalConstants v0.2.2
  [692b3bcd] JLLWrappers v1.4.1
⌅ [929cbde3] LLVM v4.17.1
  [2ab3a3ac] LogExpFunctions v0.3.23
  [21216c6a] Preferences v1.3.0
  [189a3867] Reexport v1.2.2
  [ae029012] Requires v1.3.0
  [276daf66] SpecialFunctions v2.2.0
  [90137ffa] StaticArrays v1.5.19
  [1e83bf80] StaticArraysCore v1.4.0
  [a759f4b9] TimerOutputs v0.5.22
  [8f75cd03] oneAPI v1.0.2
  [f5851436] FFTW_jll v3.3.10+0
  [1d5cc7b8] IntelOpenMP_jll v2018.0.3+2
⌅ [dad2f222] LLVMExtra_jll v0.0.18+0
  [856f044c] MKL_jll v2022.2.0+0
  [700fe977] NEO_jll v22.53.25242+0
  [efe28fd5] OpenSpecFun_jll v0.5.5+0
  [85f0d8ed] SPIRV_LLVM_Translator_unified_jll v0.2.0+0
  [6ac6d60f] SPIRV_Tools_jll v2022.1.0+0
  [09858cae] gmmlib_jll v22.3.0+0
  [94295238] libigc_jll v1.0.12812+0
  [f4bc562b] oneAPI_Level_Zero_Headers_jll v1.4.8+0
  [13eca655] oneAPI_Level_Zero_Loader_jll v1.8.12+0
  [b049733a] oneAPI_Support_jll v0.2.1+0
  [0dad84c5] ArgTools v1.1.1 `@stdlib/ArgTools`
  [56f22d72] Artifacts `@stdlib/Artifacts`
  [2a0f44e3] Base64 `@stdlib/Base64`
  [ade2ca70] Dates `@stdlib/Dates`
  [8ba89e20] Distributed `@stdlib/Distributed`
  [f43a241f] Downloads v1.6.0 `@stdlib/Downloads`
  [7b1f6079] FileWatching `@stdlib/FileWatching`
  [b77e0a4c] InteractiveUtils `@stdlib/InteractiveUtils`
  [4af54fe1] LazyArtifacts `@stdlib/LazyArtifacts`
  [b27032c2] LibCURL v0.6.3 `@stdlib/LibCURL`
  [76f85450] LibGit2 `@stdlib/LibGit2`
  [8f399da3] Libdl `@stdlib/Libdl`
  [37e2e46d] LinearAlgebra `@stdlib/LinearAlgebra`
  [56ddb016] Logging `@stdlib/Logging`
  [d6f4376e] Markdown `@stdlib/Markdown`
  [ca575930] NetworkOptions v1.2.0 `@stdlib/NetworkOptions`
  [44cfe95a] Pkg v1.8.0 `@stdlib/Pkg`
  [de0858da] Printf `@stdlib/Printf`
  [3fa0cd96] REPL `@stdlib/REPL`
  [9a3f8284] Random `@stdlib/Random`
  [ea8e919c] SHA v0.7.0 `@stdlib/SHA`
  [9e88b42a] Serialization `@stdlib/Serialization`
  [6462fe0b] Sockets `@stdlib/Sockets`
  [2f01184e] SparseArrays `@stdlib/SparseArrays`
  [10745b16] Statistics `@stdlib/Statistics`
  [fa267f1f] TOML v1.0.0 `@stdlib/TOML`
  [a4e569a6] Tar v1.10.1 `@stdlib/Tar`
  [8dfed614] Test `@stdlib/Test`
  [cf7118a7] UUIDs `@stdlib/UUIDs`
  [4ec0a83e] Unicode `@stdlib/Unicode`
  [e66e0078] CompilerSupportLibraries_jll v1.0.1+0 `@stdlib/CompilerSupportLibraries_jll`
  [deac9b47] LibCURL_jll v7.84.0+0 `@stdlib/LibCURL_jll`
  [29816b5a] LibSSH2_jll v1.10.2+0 `@stdlib/LibSSH2_jll`
  [c8ffd9c3] MbedTLS_jll v2.28.0+0 `@stdlib/MbedTLS_jll`
  [14a3606d] MozillaCACerts_jll v2022.2.1 `@stdlib/MozillaCACerts_jll`
  [4536629a] OpenBLAS_jll v0.3.20+0 `@stdlib/OpenBLAS_jll`
  [05823500] OpenLibm_jll v0.8.1+0 `@stdlib/OpenLibm_jll`
  [83775a58] Zlib_jll v1.2.12+3 `@stdlib/Zlib_jll`
  [8e850b90] libblastrampoline_jll v5.1.1+0 `@stdlib/libblastrampoline_jll`
  [8e850ede] nghttp2_jll v1.48.0+0 `@stdlib/nghttp2_jll`
  [3f19e933] p7zip_jll v17.4.0+0 `@stdlib/p7zip_jll`
        Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading.
Precompiling project...
  7 dependencies successfully precompiled in 6 seconds. 38 already precompiled.
     Testing Running tests...
┌ Info: System information:
│ Binary dependencies:
│ - NEO_jll: 22.53.25242+0
│ - libigc_jll: 1.0.12812+0
│ - gmmlib_jll: 22.3.0+0
│ - SPIRV_LLVM_Translator_unified_jll: 0.2.0+0
│ - SPIRV_Tools_jll: 2022.1.0+0
│ 
│ Toolchain:
│ - Julia: 1.8.5
│ - LLVM: 13.0.1
│ 
│ 1 driver:
│ - 00000000-0000-0000-1750-ba480103629a (v1.3.25242, API v1.3.0)
│ 
│ 1 device:
└ - Intel(R) Arc(TM) A770 Graphics [0x56a0]
[ Info: Using oneAPI support library at /home/fred/.julia/artifacts/1a12a90e346d09e5f8af8799a9494f594009af18/lib/liboneapi_support.so
                                                  |          | ---------------- CPU ---------------- |
Test                                     (Worker) | Time (s) | GC (s) | GC % | Alloc (MB) | RSS (MB) |
array                                         (2) |     2.56 |   0.06 |  2.2 |     264.96 |  1769.74 |
examples                                      (2) |    38.14 |   0.00 |  0.0 |      11.34 |  1769.74 |
      From worker 2:	WARNING: Method definition #3018#kernel(Any) in module Main at /home/fred/.julia/packages/oneAPI/Ne5Nd/test/execution.jl:293 overwritten at /home/fred/.julia/packages/oneAPI/Ne5Nd/test/execution.jl:301.
execution                                     (2) |    63.13 |   1.97 |  3.1 |    5867.74 |  1769.74 |
level-zero                                    (2) |     4.44 |   0.09 |  2.1 |     308.53 |  1769.74 |
onemkl                                        (2) |    63.10 |   1.27 |  2.0 |    3360.05 |  1769.74 |
pointer                                       (2) |     0.28 |   0.00 |  0.0 |      10.89 |  1769.74 |
sycl                                          (2) |     0.02 |   0.00 |  0.0 |       1.25 |  1769.74 |
      From worker 2:	
      From worker 2:	signal (11): Segmentation fault
      From worker 2:	in expression starting at /home/fred/.julia/packages/oneAPI/Ne5Nd/test/device/intrinsics.jl:47
      From worker 2:	_ZN3NEO14PrintFormatter11printStringEPKcRKSt8functionIFvPcEE at /home/fred/.julia/artifacts/3aff79d9a3ea33417dd6e38594e1d122a32fc8ed/lib/libze_intel_gpu.so.1 (unknown line)
      From worker 2:	_ZN3NEO14PrintFormatter17printKernelOutputERKSt8functionIFvPcEE at /home/fred/.julia/artifacts/3aff79d9a3ea33417dd6e38594e1d122a32fc8ed/lib/libze_intel_gpu.so.1 (unknown line)
      From worker 2:	_ZN2L013PrintfHandler11printOutputEPKNS_19KernelImmutableDataEPN3NEO18GraphicsAllocationEPNS_6DeviceE at /home/fred/.julia/artifacts/3aff79d9a3ea33417dd6e38594e1d122a32fc8ed/lib/libze_intel_gpu.so.1 (unknown line)
      From worker 2:	_ZN2L015CommandQueueImp24printKernelsPrintfOutputEv at /home/fred/.julia/artifacts/3aff79d9a3ea33417dd6e38594e1d122a32fc8ed/lib/libze_intel_gpu.so.1 (unknown line)
      From worker 2:	_ZN2L015CommandQueueImp18postSyncOperationsEv at /home/fred/.julia/artifacts/3aff79d9a3ea33417dd6e38594e1d122a32fc8ed/lib/libze_intel_gpu.so.1 (unknown line)
      From worker 2:	_ZN2L015CommandQueueImp32synchronizeByPollingForTaskCountEm at /home/fred/.julia/artifacts/3aff79d9a3ea33417dd6e38594e1d122a32fc8ed/lib/libze_intel_gpu.so.1 (unknown line)
      From worker 2:	macro expansion at /home/fred/.julia/packages/oneAPI/Ne5Nd/lib/level-zero/libze.jl:1556 [inlined]
      From worker 2:	macro expansion at /home/fred/.julia/packages/oneAPI/Ne5Nd/lib/level-zero/utils.jl:5 [inlined]
      From worker 2:	macro expansion at /home/fred/.julia/packages/oneAPI/Ne5Nd/lib/level-zero/libze.jl:13 [inlined]
      From worker 2:	zeCommandQueueSynchronize at /home/fred/.julia/packages/oneAPI/Ne5Nd/lib/utils/call.jl:24
      From worker 2:	synchronize at /home/fred/.julia/packages/oneAPI/Ne5Nd/lib/level-zero/cmdqueue.jl:34 [inlined]
      From worker 2:	synchronize at /home/fred/.julia/packages/oneAPI/Ne5Nd/lib/level-zero/cmdqueue.jl:34 [inlined]
      From worker 2:	synchronize at /home/fred/.julia/packages/oneAPI/Ne5Nd/src/context.jl:59
      From worker 2:	macro expansion at /home/fred/.julia/packages/oneAPI/Ne5Nd/src/utils.jl:69 [inlined]
      From worker 2:	#143 at /home/fred/.julia/packages/oneAPI/Ne5Nd/test/setup.jl:127 [inlined]
      From worker 2:	RedirectStdStream at ./stream.jl:1427
      From worker 2:	#142 at /home/fred/.julia/packages/oneAPI/Ne5Nd/test/setup.jl:87 [inlined]
      From worker 2:	#open#378 at ./io.jl:384
      From worker 2:	open at ./io.jl:381 [inlined]
      From worker 2:	#141 at /home/fred/.julia/packages/oneAPI/Ne5Nd/test/setup.jl:86 [inlined]
      From worker 2:	mktemp at ./file.jl:736
      From worker 2:	unknown function (ip: 0x7f4f3a71daf3)
      From worker 2:	_jl_invoke at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
      From worker 2:	ijl_apply_generic at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2559
      From worker 2:	mktemp at ./file.jl:734
      From worker 2:	unknown function (ip: 0x7f4fbe541192)
      From worker 2:	_jl_invoke at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
      From worker 2:	ijl_apply_generic at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2559
      From worker 2:	jl_apply at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
      From worker 2:	do_call at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/interpreter.c:126
      From worker 2:	eval_value at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/interpreter.c:215
      From worker 2:	eval_stmt_value at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/interpreter.c:166 [inlined]
      From worker 2:	eval_body at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/interpreter.c:612
      From worker 2:	eval_body at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/interpreter.c:522
      From worker 2:	eval_body at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/interpreter.c:522
      From worker 2:	jl_interpret_toplevel_thunk at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/interpreter.c:750
      From worker 2:	jl_toplevel_eval_flex at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/toplevel.c:906
      From worker 2:	jl_toplevel_eval_flex at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/toplevel.c:850
      From worker 2:	ijl_toplevel_eval_in at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/toplevel.c:965
      From worker 2:	eval at ./boot.jl:368 [inlined]
      From worker 2:	include_string at ./loading.jl:1428
      From worker 2:	_jl_invoke at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
      From worker 2:	ijl_apply_generic at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2559
      From worker 2:	_include at ./loading.jl:1488
      From worker 2:	include at ./client.jl:476 [inlined]
      From worker 2:	#11 at /home/fred/.julia/packages/oneAPI/Ne5Nd/test/runtests.jl:84 [inlined]
      From worker 2:	macro expansion at /home/fred/.julia/packages/oneAPI/Ne5Nd/test/setup.jl:52 [inlined]
      From worker 2:	macro expansion at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/Test/src/Test.jl:1363 [inlined]
      From worker 2:	macro expansion at /home/fred/.julia/packages/oneAPI/Ne5Nd/test/setup.jl:52 [inlined]
      From worker 2:	macro expansion at ./timing.jl:463 [inlined]
      From worker 2:	top-level scope at /home/fred/.julia/packages/oneAPI/Ne5Nd/test/setup.jl:51
      From worker 2:	jl_toplevel_eval_flex at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/toplevel.c:897
      From worker 2:	ijl_toplevel_eval_in at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/toplevel.c:965
      From worker 2:	eval at ./boot.jl:368 [inlined]
      From worker 2:	runtests at /home/fred/.julia/packages/oneAPI/Ne5Nd/test/setup.jl:55
      From worker 2:	unknown function (ip: 0x7f4ff5d45956)
      From worker 2:	_jl_invoke at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
      From worker 2:	ijl_apply_generic at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2559
      From worker 2:	jl_apply at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
      From worker 2:	jl_f__call_latest at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/builtins.c:774
      From worker 2:	_jl_invoke at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
      From worker 2:	ijl_apply_generic at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2559
      From worker 2:	jl_apply at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
      From worker 2:	do_apply at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/builtins.c:730
      From worker 2:	#invokelatest#2 at ./essentials.jl:729
      From worker 2:	_jl_invoke at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
      From worker 2:	ijl_apply_generic at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2559
      From worker 2:	jl_apply at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
      From worker 2:	do_apply at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/builtins.c:730
      From worker 2:	invokelatest at ./essentials.jl:726
      From worker 2:	_jl_invoke at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
      From worker 2:	ijl_apply_generic at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2559
      From worker 2:	jl_apply at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
      From worker 2:	do_apply at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/builtins.c:730
      From worker 2:	#110 at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/Distributed/src/process_messages.jl:285
      From worker 2:	run_work_thunk at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/Distributed/src/process_messages.jl:70
      From worker 2:	macro expansion at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/Distributed/src/process_messages.jl:285 [inlined]
      From worker 2:	#109 at ./task.jl:484
      From worker 2:	unknown function (ip: 0x7f4ff5d4461f)
      From worker 2:	_jl_invoke at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2377 [inlined]
      From worker 2:	ijl_apply_generic at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/gf.c:2559
      From worker 2:	jl_apply at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/julia.h:1843 [inlined]
      From worker 2:	start_task at /cache/build/default-amdci4-2/julialang/julia-release-1-dot-8/src/task.c:931
      From worker 2:	Allocations: 236739512 (Pool: 236569439; Big: 170073); GC: 254
device/intrinsics                             (2) |         failed at 2023-03-28T19:55:03.920
Worker 2 terminated.
Unhandled Task ERROR: EOFError: read end of file
Stacktrace:
 [1] (::Base.var"#wait_locked#680")(s::Sockets.TCPSocket, buf::IOBuffer, nb::Int64)
   @ Base ./stream.jl:945
 [2] unsafe_read(s::Sockets.TCPSocket, p::Ptr{UInt8}, nb::UInt64)
   @ Base ./stream.jl:953
 [3] unsafe_read
   @ ./io.jl:759 [inlined]
 [4] unsafe_read(s::Sockets.TCPSocket, p::Base.RefValue{NTuple{4, Int64}}, n::Int64)
   @ Base ./io.jl:758
 [5] read!
   @ ./io.jl:760 [inlined]
 [6] deserialize_hdr_raw
   @ ~/julia-1.8.5/share/julia/stdlib/v1.8/Distributed/src/messages.jl:167 [inlined]
 [7] message_handler_loop(r_stream::Sockets.TCPSocket, w_stream::Sockets.TCPSocket, incoming::Bool)
   @ Distributed ~/julia-1.8.5/share/julia/stdlib/v1.8/Distributed/src/process_messages.jl:172
 [8] process_tcp_streams(r_stream::Sockets.TCPSocket, w_stream::Sockets.TCPSocket, incoming::Bool)
   @ Distributed ~/julia-1.8.5/share/julia/stdlib/v1.8/Distributed/src/process_messages.jl:133
 [9] (::Distributed.var"#103#104"{Sockets.TCPSocket, Sockets.TCPSocket, Bool})()
   @ Distributed ./task.jl:484
gpuarrays/indexing scalar                     (3) |    34.47 |   1.11 |  3.2 |    2999.92 |  1769.79 |
gpuarrays/reductions/reducedim!               (3) |   131.51 |   4.76 |  3.6 |   12209.53 |  1769.79 |
gpuarrays/linalg                              (3) |   187.70 |   7.06 |  3.8 |   18466.36 |  2068.39 |
gpuarrays/math/power                          (3) |    38.63 |   1.96 |  5.1 |    4071.89 |  2068.39 |
gpuarrays/linalg/mul!/vector-matrix           (3) |   104.54 |   3.56 |  3.4 |    8763.68 |  2185.79 |
gpuarrays/indexing find                       (3) |    44.25 |   1.71 |  3.9 |    3974.84 |  2361.82 |
gpuarrays/indexing multidimensional           (3) |    49.86 |   1.97 |  3.9 |    4520.19 |  2448.53 |
gpuarrays/interface                           (3) |     7.66 |   0.30 |  3.9 |     714.76 |  2474.74 |
gpuarrays/reductions/any all count            (3) |    32.26 |   1.27 |  3.9 |    2856.56 |  2528.68 |
gpuarrays/reductions/minimum maximum extrema  (3) |   381.23 |  13.47 |  3.5 |   29572.93 |  3458.21 |
gpuarrays/uniformscaling                      (3) |    14.70 |   0.43 |  2.9 |     978.98 |  3458.21 |
gpuarrays/reductions/mapreducedim!_large      (3) |         failed at 2023-03-28T20:14:44.549
gpuarrays/linalg/mul!/matrix-matrix           (4) |   206.99 |   6.62 |  3.2 |   17902.71 |  1769.98 |
gpuarrays/math/intrinsics                     (4) |     5.35 |   0.19 |  3.5 |     486.73 |  1769.98 |
gpuarrays/linalg/norm                         (4) |    94.50 |   4.53 |  4.8 |    9079.11 |  1781.64 |
gpuarrays/statistics                          (4) |    91.10 |   4.06 |  4.5 |    9237.41 |  1919.26 |
gpuarrays/reductions/mapreduce                (4) |   291.29 |  13.67 |  4.7 |   30625.48 |  2477.53 |
gpuarrays/constructors                        (4) |    27.53 |   1.09 |  4.0 |    2507.59 |  2502.00 |
gpuarrays/random                              (4) |    30.77 |   1.36 |  4.4 |    3030.48 |  2620.46 |
gpuarrays/base                                (4) |    65.10 |   2.56 |  3.9 |    6565.66 |  3061.15 |
gpuarrays/reductions/== isequal               (4) |   120.22 |   4.02 |  3.3 |    8762.59 |  3158.81 |
gpuarrays/broadcasting                        (4) |   390.12 |  15.20 |  3.9 |   32990.61 |  4375.66 |
gpuarrays/reductions/mapreducedim!            (4) |   251.02 |   7.99 |  3.2 |   16826.87 |  4956.14 |
gpuarrays/reductions/reduce                   (4) |    16.19 |   0.48 |  2.9 |    1295.02 |  4956.14 |
gpuarrays/reductions/sum prod                 (4) |   270.93 |   8.19 |  3.0 |   17671.01 |  5661.66 |
Testing finished in 54 minutes, 29 seconds, 277 milliseconds
device/intrinsics: Error During Test at none:1
  Got exception outside of a @test
  ProcessExitedException(2)
Worker 3 failed running test gpuarrays/reductions/mapreducedim!_large:
Some tests did not pass: 33 passed, 7 failed, 0 errored, 0 broken.
gpuarrays/reductions/mapreducedim!_large: Test Failed at /home/fred/.julia/packages/GPUArrays/XR4WO/test/testsuite/reductions.jl:29
  Expression: compare(((A, R)->begin
            Base.mapreducedim!(identity, +, R, A)
        end), AT, rand(range, sz), zeros(ET, red))
Stacktrace:
 [1] record(ts::Test.DefaultTestSet, t::Union{Test.Error, Test.Fail})
   @ Test ~/julia-1.8.5/share/julia/stdlib/v1.8/Test/src/Test.jl:986
 [2] top-level scope
   @ ~/.julia/packages/oneAPI/Ne5Nd/test/runtests.jl:358
 [3] include(fname::String)
   @ Base.MainInclude ./client.jl:476
 [4] top-level scope
   @ none:6
 [5] eval
   @ ./boot.jl:368 [inlined]
 [6] exec_options(opts::Base.JLOptions)
   @ Base ./client.jl:276
 [7] _start()
   @ Base ./client.jl:522
gpuarrays/reductions/mapreducedim!_large: Test Failed at /home/fred/.julia/packages/GPUArrays/XR4WO/test/testsuite/reductions.jl:29
  Expression: compare(((A, R)->begin
            Base.mapreducedim!(identity, +, R, A)
        end), AT, rand(range, sz), zeros(ET, red))
Stacktrace:
 [1] record(ts::Test.DefaultTestSet, t::Union{Test.Error, Test.Fail})
   @ Test ~/julia-1.8.5/share/julia/stdlib/v1.8/Test/src/Test.jl:986
 [2] top-level scope
   @ ~/.julia/packages/oneAPI/Ne5Nd/test/runtests.jl:358
 [3] include(fname::String)
   @ Base.MainInclude ./client.jl:476
 [4] top-level scope
   @ none:6
 [5] eval
   @ ./boot.jl:368 [inlined]
 [6] exec_options(opts::Base.JLOptions)
   @ Base ./client.jl:276
 [7] _start()
   @ Base ./client.jl:522
gpuarrays/reductions/mapreducedim!_large: Test Failed at /home/fred/.julia/packages/GPUArrays/XR4WO/test/testsuite/reductions.jl:29
  Expression: compare(((A, R)->begin
            Base.mapreducedim!(identity, +, R, A)
        end), AT, rand(range, sz), zeros(ET, red))
Stacktrace:
 [1] record(ts::Test.DefaultTestSet, t::Union{Test.Error, Test.Fail})
   @ Test ~/julia-1.8.5/share/julia/stdlib/v1.8/Test/src/Test.jl:986
 [2] top-level scope
   @ ~/.julia/packages/oneAPI/Ne5Nd/test/runtests.jl:358
 [3] include(fname::String)
   @ Base.MainInclude ./client.jl:476
 [4] top-level scope
   @ none:6
 [5] eval
   @ ./boot.jl:368 [inlined]
 [6] exec_options(opts::Base.JLOptions)
   @ Base ./client.jl:276
 [7] _start()
   @ Base ./client.jl:522
gpuarrays/reductions/mapreducedim!_large: Test Failed at /home/fred/.julia/packages/GPUArrays/XR4WO/test/testsuite/reductions.jl:29
  Expression: compare(((A, R)->begin
            Base.mapreducedim!(identity, +, R, A)
        end), AT, rand(range, sz), zeros(ET, red))
Stacktrace:
 [1] record(ts::Test.DefaultTestSet, t::Union{Test.Error, Test.Fail})
   @ Test ~/julia-1.8.5/share/julia/stdlib/v1.8/Test/src/Test.jl:986
 [2] top-level scope
   @ ~/.julia/packages/oneAPI/Ne5Nd/test/runtests.jl:358
 [3] include(fname::String)
   @ Base.MainInclude ./client.jl:476
 [4] top-level scope
   @ none:6
 [5] eval
   @ ./boot.jl:368 [inlined]
 [6] exec_options(opts::Base.JLOptions)
   @ Base ./client.jl:276
 [7] _start()
   @ Base ./client.jl:522
gpuarrays/reductions/mapreducedim!_large: Test Failed at /home/fred/.julia/packages/GPUArrays/XR4WO/test/testsuite/reductions.jl:29
  Expression: compare(((A, R)->begin
            Base.mapreducedim!(identity, +, R, A)
        end), AT, rand(range, sz), zeros(ET, red))
Stacktrace:
 [1] record(ts::Test.DefaultTestSet, t::Union{Test.Error, Test.Fail})
   @ Test ~/julia-1.8.5/share/julia/stdlib/v1.8/Test/src/Test.jl:986
 [2] top-level scope
   @ ~/.julia/packages/oneAPI/Ne5Nd/test/runtests.jl:358
 [3] include(fname::String)
   @ Base.MainInclude ./client.jl:476
 [4] top-level scope
   @ none:6
 [5] eval
   @ ./boot.jl:368 [inlined]
 [6] exec_options(opts::Base.JLOptions)
   @ Base ./client.jl:276
 [7] _start()
   @ Base ./client.jl:522
gpuarrays/reductions/mapreducedim!_large: Test Failed at /home/fred/.julia/packages/GPUArrays/XR4WO/test/testsuite/reductions.jl:29
  Expression: compare(((A, R)->begin
            Base.mapreducedim!(identity, +, R, A)
        end), AT, rand(range, sz), zeros(ET, red))
Stacktrace:
 [1] record(ts::Test.DefaultTestSet, t::Union{Test.Error, Test.Fail})
   @ Test ~/julia-1.8.5/share/julia/stdlib/v1.8/Test/src/Test.jl:986
 [2] top-level scope
   @ ~/.julia/packages/oneAPI/Ne5Nd/test/runtests.jl:358
 [3] include(fname::String)
   @ Base.MainInclude ./client.jl:476
 [4] top-level scope
   @ none:6
 [5] eval
   @ ./boot.jl:368 [inlined]
 [6] exec_options(opts::Base.JLOptions)
   @ Base ./client.jl:276
 [7] _start()
   @ Base ./client.jl:522
gpuarrays/reductions/mapreducedim!_large: Test Failed at /home/fred/.julia/packages/GPUArrays/XR4WO/test/testsuite/reductions.jl:29
  Expression: compare(((A, R)->begin
            Base.mapreducedim!(identity, +, R, A)
        end), AT, rand(range, sz), zeros(ET, red))
Stacktrace:
 [1] record(ts::Test.DefaultTestSet, t::Union{Test.Error, Test.Fail})
   @ Test ~/julia-1.8.5/share/julia/stdlib/v1.8/Test/src/Test.jl:986
 [2] top-level scope
   @ ~/.julia/packages/oneAPI/Ne5Nd/test/runtests.jl:358
 [3] include(fname::String)
   @ Base.MainInclude ./client.jl:476
 [4] top-level scope
   @ none:6
 [5] eval
   @ ./boot.jl:368 [inlined]
 [6] exec_options(opts::Base.JLOptions)
   @ Base ./client.jl:276
 [7] _start()
   @ Base ./client.jl:522

Test Summary:                                  | Pass  Fail  Error  Total  Time
  Overall                                      | 5260     7      1   5268      
    array                                      |   15                  15      
    examples                                   |    2                   2      
    execution                                  |   43                  43      
    level-zero                                 |   34                  34      
    onemkl                                     |  167                 167      
    pointer                                    |   35                  35      
    sycl                                       |    4                   4      
    device/intrinsics                          |                 1      1      
    gpuarrays/indexing scalar                  |  359                 359      
    gpuarrays/reductions/reducedim!            |  144                 144      
    gpuarrays/linalg                           |  218                 218      
    gpuarrays/math/power                       |   54                  54      
    gpuarrays/linalg/mul!/vector-matrix        |  126                 126      
    gpuarrays/indexing find                    |   45                  45      
    gpuarrays/indexing multidimensional        |   40                  40      
    gpuarrays/interface                        |    7                   7      
    gpuarrays/reductions/any all count         |  101                 101      
    gpuarrays/reductions/minimum maximum extrema |  555                 555      
    gpuarrays/uniformscaling                   |   56                  56      
    gpuarrays/reductions/mapreducedim!_large   |   33     7            40      
    gpuarrays/linalg/mul!/matrix-matrix        |  324                 324      
    gpuarrays/math/intrinsics                  |   10                  10      
    gpuarrays/linalg/norm                      |  198                 198      
    gpuarrays/statistics                       |   52                  52      
    gpuarrays/reductions/mapreduce             |  297                 297      
    gpuarrays/constructors                     |  705                 705      
    gpuarrays/random                           |   46                  46      
    gpuarrays/base                             |   73                  73      
    gpuarrays/reductions/== isequal            |  216                 216      
    gpuarrays/broadcasting                     |  299                 299      
    gpuarrays/reductions/mapreducedim!         |  234                 234      
    gpuarrays/reductions/reduce                |  198                 198      
    gpuarrays/reductions/sum prod              |  570                 570      
    FAILURE

Error in testset device/intrinsics:
Error During Test at none:1
  Got exception outside of a @test
  ProcessExitedException(2)
Error in testset gpuarrays/reductions/mapreducedim!_large:
Test Failed at /home/fred/.julia/packages/GPUArrays/XR4WO/test/testsuite/reductions.jl:29
  Expression: compare(((A, R)->begin
            Base.mapreducedim!(identity, +, R, A)
        end), AT, rand(range, sz), zeros(ET, red))
Error in testset gpuarrays/reductions/mapreducedim!_large:
Test Failed at /home/fred/.julia/packages/GPUArrays/XR4WO/test/testsuite/reductions.jl:29
  Expression: compare(((A, R)->begin
            Base.mapreducedim!(identity, +, R, A)
        end), AT, rand(range, sz), zeros(ET, red))
Error in testset gpuarrays/reductions/mapreducedim!_large:
Test Failed at /home/fred/.julia/packages/GPUArrays/XR4WO/test/testsuite/reductions.jl:29
  Expression: compare(((A, R)->begin
            Base.mapreducedim!(identity, +, R, A)
        end), AT, rand(range, sz), zeros(ET, red))
Error in testset gpuarrays/reductions/mapreducedim!_large:
Test Failed at /home/fred/.julia/packages/GPUArrays/XR4WO/test/testsuite/reductions.jl:29
  Expression: compare(((A, R)->begin
            Base.mapreducedim!(identity, +, R, A)
        end), AT, rand(range, sz), zeros(ET, red))
Error in testset gpuarrays/reductions/mapreducedim!_large:
Test Failed at /home/fred/.julia/packages/GPUArrays/XR4WO/test/testsuite/reductions.jl:29
  Expression: compare(((A, R)->begin
            Base.mapreducedim!(identity, +, R, A)
        end), AT, rand(range, sz), zeros(ET, red))
Error in testset gpuarrays/reductions/mapreducedim!_large:
Test Failed at /home/fred/.julia/packages/GPUArrays/XR4WO/test/testsuite/reductions.jl:29
  Expression: compare(((A, R)->begin
            Base.mapreducedim!(identity, +, R, A)
        end), AT, rand(range, sz), zeros(ET, red))
Error in testset gpuarrays/reductions/mapreducedim!_large:
Test Failed at /home/fred/.julia/packages/GPUArrays/XR4WO/test/testsuite/reductions.jl:29
  Expression: compare(((A, R)->begin
            Base.mapreducedim!(identity, +, R, A)
        end), AT, rand(range, sz), zeros(ET, red))
ERROR: LoadError: Test run finished with errors
in expression starting at /home/fred/.julia/packages/oneAPI/Ne5Nd/test/runtests.jl:389
ERROR: Package oneAPI errored during testing
@maleadt
Copy link
Member

maleadt commented Mar 29, 2023

Also, can the test suite use more than 1 core?

It looks at the thread count, but since that is generally set to 1 in #304 I've changed it to look at the number of CPU threads instead.

@maleadt maleadt added the bug Something isn't working label Mar 29, 2023
@fredlarochelle
Copy link
Author

Small update, I tried the lastest code from the master branch, I can confirm that the test suite uses all the CPU threads and that I still get the same tests not passing.

@maleadt
Copy link
Member

maleadt commented Mar 31, 2023

I can confirm the failures. One was only introduced recently, in the test from JuliaGPU/GPUArrays.jl#459. They should be reduced to MWEs we can debug further, but I don't have the time for that right now.

@maleadt
Copy link
Member

maleadt commented Apr 20, 2023

Isolated the device/intrinsics issue, looks related to printing empty strings. Reported upstream: intel/compute-runtime#635

@maleadt
Copy link
Member

maleadt commented Apr 20, 2023

MWE for the other issue:

julia> GPUArrays.mapreducedim!(identity, +, oneAPI.zeros(Float32, 1), oneAPI.ones(Float32, 100000))
1-element oneVector{Float32, oneAPI.oneL0.DeviceBuffer}:
 99952.0

Reduced to:

using oneAPI

# complete reduction values by a group, using local memory for communication
@inline function partialsum_group(val::T, neutral) where {T}
    items = get_local_size(0)
    item = get_local_id(0)

    # local mem for a complete reduction
    shared = oneLocalArray(T, (1024,))
    @inbounds shared[item] = val

    # perform a reduction
    d = 1
    while d < items
        barrier()
        index = 2 * d * (item-1) + 1
        @inbounds if index <= items
            other_val = if index + d <= items
                shared[index+d]
            else
                neutral
            end
            shared[index] = shared[index] + other_val
        end
        d *= 2
    end

    # load the final value on the first item
    if item == 1
        val = @inbounds shared[item]
    end

    return val
end

# partial reduction of the input vector, using a grid-stride loop
function partialsum(elements, reduced, input)
    localIdx = get_local_id(0)
    localDim = get_local_size(0)
    groupIdx = get_group_id(0)
    groupDim = get_num_groups(0)

    @inbounds begin
        # load the neutral value
        #
        # for this MWE, the value is always 0, but hard-coding it makes the bug occur
        # less often.
        neutral = unsafe_load(reduced, groupIdx)
        val = neutral + neutral

        # reduce serially across chunks of the input vector.
        #
        # for this MWE, we only execute the loop body once (as we allocate exactly
        # items * groups elements), but removing the loop makes the bug happen less often.
        ireduce = localIdx + (groupIdx - 1) * localDim
        while ireduce <= elements
            val = val + unsafe_load(input, ireduce)
            ireduce += localDim * groupDim
        end

        # reduce all values within the group
        val = partialsum_group(val, neutral)

        # write back to memory
        if localIdx == 1
            unsafe_store!(reduced, val, groupIdx)
        end
    end

    return
end

function reduced(items=800, groups=100)
    elements = Int32(items * groups)
    input = oneAPI.ones(Int32, elements)
    reduced = oneAPI.zeros(Int32, groups)

    @oneapi items=items groups=groups partialsum(elements, pointer(reduced), pointer(input))
    reduced
end

function main()
    for x in 1:10
        x = reduced()
        println(x)
        all(isequal(800), Array(x)) || return false
    end
    return true
end

Filed upstream as intel/compute-runtime#636

@maleadt
Copy link
Member

maleadt commented Feb 27, 2024

These should be fixed.

@maleadt maleadt closed this as completed Feb 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants