Adding MPI test #518

michel2323 · 2022-10-13T17:39:59Z

Let me know if you want to keep the trailing whitespaces.

michel2323 · 2023-01-31T21:44:53Z

#585 @vchuravy This should be merged I guess. Just checking again.

test/runtests.jl

vchuravy · 2023-06-19T15:10:57Z

Can you rebase once more?

michel2323 · 2023-06-20T17:48:22Z

@vchuravy @wsmoses The MPI test crashes now. With 1.9 we have the finalizer issue, and with 1.8 the logs and Manifest.toml are attached.
log.tar.gz

wsmoses · 2023-11-08T17:40:16Z

Can you open an issue with the error message?

codecov-commenter · 2023-11-08T17:51:28Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.43%. Comparing base (90e07c1) to head (f16f50b).
Report is 6 commits behind head on main.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@             Coverage Diff             @@
##             main     #518       +/-   ##
===========================================
+ Coverage   75.39%   93.43%   +18.03%     
===========================================
  Files          35        7       -28     
  Lines       10653      259    -10394     
===========================================
- Hits         8032      242     -7790     
+ Misses       2621       17     -2604

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

michel2323 · 2023-11-08T17:56:40Z

@wsmoses @vchuravy I marked the test here as broken so we can merge the MPI tests and mark it unbroken with the fix. Issue opened #1138.

wsmoses · 2023-11-20T02:37:45Z

@michel2323 i rebased this PR with the jll with your fix, if it passes let's merge this!

test/runtests.jl

wsmoses · 2023-11-20T04:42:25Z

@vchuravy this still fails presumably for the need for: #669

from

2023-11-20T03:26:35.5348040Z error: /home/runner/.julia/packages/MPI/hhI6i/src/api/generated_api.jl:2151:0: in function preprocess_julia_Isend_50847 {} addrspace(10)* ({} addrspace(10)*, i64, {} addrspace(10)*): Enzyme: failed to deduce type of copy   %65 = call i32 @MPI_Isend(i64 %58, i32 %value_phi4, i32 %61, i32 %52, i32 0, i32 %62, i64 %64) #34 [ "jl_roots"({} addrspace(10)* %2, {} addrspace(10)* addrspacecast ({}* inttoptr (i64 139848936934304 to {}*) to {} addrspace(10)*), {} addrspace(10)* %value_phi5, {} addrspace(10)* %value_phi3) ], !dbg !107

2023-11-20T03:26:35.5120729Z   %value_phi5 = phi {} addrspace(10)* [ null, %pass ], [ addrspacecast ({}* inttoptr (i64 139848936932704 to {}*) to {} addrspace(10)*), %L54 ]: {[-1]:Pointer}, intvals: {0,}
2023-11-20T03:26:35.4979683Z   %59 = bitcast {} addrspace(10)* %value_phi5 to i32 addrspace(10)*, !dbg !123: {[-1]:Pointer}, intvals: {0,}
2023-11-20T03:26:35.4980592Z   %60 = addrspacecast i32 addrspace(10)* %59 to i32 addrspace(11)*, !dbg !123: {[-1]:Pointer}, intvals: {0,}
2023-11-20T03:26:35.4981343Z   %61 = load i32, i32 addrspace(11)* %60, align 4, !dbg !123, !tbaa !45: {}, intvals: {}

While I do agree custom rules are good (and we can redo that API to be a custom global invariant rule, the issue of that MPI.Double being hidden behind a separate julia-specific global int is hindering the analysis here (and also likely would for other libXYZ calls potentially, for example CUDA).

github-actions · 2024-04-01T00:38:29Z

test/mpi.jl

+        buf =  @view x[1:1]
+        push!(requests, MPI.Isend(x[2:2], MPI.COMM_WORLD; dest=rank-1, tag=0))
+        push!(requests, MPI.Irecv!(buf, MPI.COMM_WORLD; source=rank-1, tag=0))


[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

buf = @view x[1:1]

push!(requests, MPI.Isend(x[2:2], MPI.COMM_WORLD; dest=rank-1, tag=0))

push!(requests, MPI.Irecv!(buf, MPI.COMM_WORLD; source=rank-1, tag=0))

buf = @view x[1:1]

push!(requests, MPI.Isend(x[2:2], MPI.COMM_WORLD; dest=rank - 1, tag=0))

push!(requests, MPI.Irecv!(buf, MPI.COMM_WORLD; source=rank - 1, tag=0))

github-actions · 2024-04-01T00:38:29Z

test/mpi.jl

+    if rank != np-1
+        buf =  @view x[end:end]
+        push!(requests, MPI.Isend(x[end-1:end-1], MPI.COMM_WORLD; dest=rank+1, tag=0))
+        push!(requests, MPI.Irecv!(buf, MPI.COMM_WORLD; source=rank+1, tag=0))


[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

if rank != np-1

buf = @view x[end:end]

push!(requests, MPI.Isend(x[end-1:end-1], MPI.COMM_WORLD; dest=rank+1, tag=0))

push!(requests, MPI.Irecv!(buf, MPI.COMM_WORLD; source=rank+1, tag=0))

if rank != np - 1

buf = @view x[end:end]

push!(requests,

MPI.Isend(x[(end - 1):(end - 1)], MPI.COMM_WORLD; dest=rank + 1, tag=0))

push!(requests, MPI.Irecv!(buf, MPI.COMM_WORLD; source=rank + 1, tag=0))

github-actions · 2024-04-01T00:38:29Z

test/mpi.jl

+n = np*10
+n1 = Int(round(rank / np * (n+np))) - rank
+n2 = Int(round((rank + 1) / np * (n+np))) - rank
+nl = rank == 0 ? n1+1 : n1
+nr = rank == np-1 ? n2-1 : n2
+nlocal = nr-nl+1


[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

n = np*10

n1 = Int(round(rank / np * (n+np))) - rank

n2 = Int(round((rank + 1) / np * (n+np))) - rank

nl = rank == 0 ? n1+1 : n1

nr = rank == np-1 ? n2-1 : n2

nlocal = nr-nl+1

n = np * 10

n1 = Int(round(rank / np * (n + np))) - rank

n2 = Int(round((rank + 1) / np * (n + np))) - rank

nl = rank == 0 ? n1 + 1 : n1

nr = rank == np - 1 ? n2 - 1 : n2

nlocal = nr - nl + 1

github-actions · 2024-04-01T00:38:29Z

test/mpi.jl

+fill!(context.x, Float64(rank))
+halo(context)
+if rank != 0
+    @test context.x[1] == Float64(rank-1)


[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

@test context.x[1] == Float64(rank-1)

@test context.x[1] == Float64(rank - 1)

github-actions · 2024-04-01T00:38:29Z

test/mpi.jl

+if rank != np-1
+    @test context.x[end] == Float64(rank+1)


[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

if rank != np-1

@test context.x[end] == Float64(rank+1)

if rank != np - 1

@test context.x[end] == Float64(rank + 1)

github-actions · 2024-04-01T00:38:30Z

test/runtests.jl

+    try
+        include("mpi.jl")
+        mpiexec() do cmd
+                run(`$cmd -n 2 $(Base.julia_cmd()) --project=$testdir $testdir/mpi.jl`)


[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

run(`$cmd -n 2 $(Base.julia_cmd()) --project=$testdir $testdir/mpi.jl`)

return run(`$cmd -n 2 $(Base.julia_cmd()) --project=$testdir $testdir/mpi.jl`)

github-actions · 2024-04-01T00:38:30Z

test/runtests.jl

+    end
+    @test mpi_test
+end
+



[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

github-actions · 2024-04-01T00:38:30Z

test/runtests.jl

+    b = Float64[11, 13]
+    db = zero(b)
+
+    forward, pullback = Enzyme.autodiff_thunk(ReverseSplitNoPrimal, Const{typeof(\)}, Duplicated, Duplicated{typeof(A)}, Duplicated{typeof(b)})


[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

forward, pullback = Enzyme.autodiff_thunk(ReverseSplitNoPrimal, Const{typeof(\)}, Duplicated, Duplicated{typeof(A)}, Duplicated{typeof(b)})

forward, pullback = Enzyme.autodiff_thunk(ReverseSplitNoPrimal, Const{typeof(\)},

Duplicated, Duplicated{typeof(A)},

Duplicated{typeof(b)})

github-actions · 2024-04-01T00:38:30Z

test/runtests.jl

+
+    db = zero(b)
+
+    forward, pullback = Enzyme.autodiff_thunk(ReverseSplitNoPrimal, Const{typeof(\)}, Duplicated, Const{typeof(A)}, Duplicated{typeof(b)})


[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

forward, pullback = Enzyme.autodiff_thunk(ReverseSplitNoPrimal, Const{typeof(\)}, Duplicated, Const{typeof(A)}, Duplicated{typeof(b)})

forward, pullback = Enzyme.autodiff_thunk(ReverseSplitNoPrimal, Const{typeof(\)},

Duplicated, Const{typeof(A)},

Duplicated{typeof(b)})

github-actions · 2024-04-01T00:38:30Z

test/runtests.jl

+
+    dA = zero(A)
+
+    forward, pullback = Enzyme.autodiff_thunk(ReverseSplitNoPrimal, Const{typeof(\)}, Duplicated, Duplicated{typeof(A)}, Const{typeof(b)})


[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

forward, pullback = Enzyme.autodiff_thunk(ReverseSplitNoPrimal, Const{typeof(\)}, Duplicated, Duplicated{typeof(A)}, Const{typeof(b)})

forward, pullback = Enzyme.autodiff_thunk(ReverseSplitNoPrimal, Const{typeof(\)},

Duplicated, Duplicated{typeof(A)},

Const{typeof(b)})

michel2323 · 2024-04-01T14:53:10Z

I get this now on the most recent Enzyme build:

ERROR: LoadError: ERROR: LoadError: Enzyme execution failed.
Enzyme: jl_call calling convention not implemented in aug_forward for   %33 = call nonnull {} addrspace(10)* ({} addrspace(10)* ({} addrspace(10)*, {} addrspace(10)**, i32)*, {} addrspace(10)*, ...) @julia.call({} addrspace(10)* ({} addrspace(10)*, {} addrspace(10)**, i32)* noundef nonnull @jl_f_finalizer, {} addrspace(10)* noundef null, {} addrspace(10)* addrspacecast ({}* inttoptr (i64 140298333135136 to {}*) to {} addrspace(10)*), {} addrspace(10)* nofree nonnull align 8 dereferenceable(16) %newstruct15.i) #29, !dbg !149
Stacktrace:
 [1] finalizer
   @ ./gcutils.jl:87
 [2] Request
   @ /disk/mschanen/julia_depot/packages/MPI/z2owj/src/nonblocking.jl:183

This is the request issue @vchuravy mentioned. How do we proceed with MPI in Julia? Should we start an Enzyme extension for MPI/MPI extension for Enzyme?

michel2323 requested a review from vchuravy October 13, 2022 17:39

vchuravy approved these changes Oct 14, 2022

View reviewed changes

michel2323 force-pushed the ms/mpi branch 3 times, most recently from 8d782c4 to b46613f Compare November 18, 2022 18:38

michel2323 force-pushed the ms/mpi branch from b46613f to 5e5f560 Compare January 31, 2023 21:44

vchuravy reviewed Jan 31, 2023

View reviewed changes

test/runtests.jl Outdated Show resolved Hide resolved

michel2323 force-pushed the ms/mpi branch 3 times, most recently from 409d930 to b9e0668 Compare January 31, 2023 22:27

michel2323 force-pushed the ms/mpi branch from b9e0668 to 6cafa4e Compare June 20, 2023 17:46

michel2323 force-pushed the ms/mpi branch 2 times, most recently from 0f2cbdb to 6754c67 Compare November 7, 2023 22:52

michel2323 mentioned this pull request Nov 8, 2023

MPI test crashing #1138

Open

michel2323 force-pushed the ms/mpi branch from 2b6dc2d to ad70ee6 Compare November 8, 2023 17:55

wsmoses force-pushed the ms/mpi branch 2 times, most recently from 0e829b9 to a1745a5 Compare November 20, 2023 02:37

wsmoses reviewed Nov 20, 2023

View reviewed changes

test/runtests.jl Outdated Show resolved Hide resolved

vchuravy added this to the release-0.12 milestone Apr 1, 2024

github-actions bot reviewed Apr 1, 2024

View reviewed changes

michel2323 force-pushed the ms/mpi branch from 6340c27 to f16f50b Compare April 1, 2024 13:59

Adding MPI test

723a1ce

michel2323 and others added 2 commits April 1, 2024 09:02

Mark MPI test as broken

ecf4e4e

Update test/runtests.jl

4cdb9f5

michel2323 force-pushed the ms/mpi branch from f16f50b to 4cdb9f5 Compare April 1, 2024 19:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding MPI test #518

Adding MPI test #518

michel2323 commented Oct 13, 2022

michel2323 commented Jan 31, 2023

vchuravy commented Jun 19, 2023

michel2323 commented Jun 20, 2023

wsmoses commented Nov 8, 2023

codecov-commenter commented Nov 8, 2023 •

edited

Loading

michel2323 commented Nov 8, 2023 •

edited

Loading

wsmoses commented Nov 20, 2023

wsmoses commented Nov 20, 2023

github-actions bot Apr 1, 2024

github-actions bot Apr 1, 2024

github-actions bot Apr 1, 2024

github-actions bot Apr 1, 2024

github-actions bot Apr 1, 2024

github-actions bot Apr 1, 2024

github-actions bot Apr 1, 2024

github-actions bot Apr 1, 2024

github-actions bot Apr 1, 2024

github-actions bot Apr 1, 2024

michel2323 commented Apr 1, 2024 •

edited

Loading

	@test context.x[1] == Float64(rank-1)
	@test context.x[1] == Float64(rank - 1)

	run(`$cmd -n 2 $(Base.julia_cmd()) --project=$testdir $testdir/mpi.jl`)
	return run(`$cmd -n 2 $(Base.julia_cmd()) --project=$testdir $testdir/mpi.jl`)


		db = zero(b)

		forward, pullback = Enzyme.autodiff_thunk(ReverseSplitNoPrimal, Const{typeof(\)}, Duplicated, Const{typeof(A)}, Duplicated{typeof(b)})


		dA = zero(A)

		forward, pullback = Enzyme.autodiff_thunk(ReverseSplitNoPrimal, Const{typeof(\)}, Duplicated, Duplicated{typeof(A)}, Const{typeof(b)})

Adding MPI test #518

Are you sure you want to change the base?

Adding MPI test #518

Conversation

michel2323 commented Oct 13, 2022

michel2323 commented Jan 31, 2023

vchuravy commented Jun 19, 2023

michel2323 commented Jun 20, 2023

wsmoses commented Nov 8, 2023

codecov-commenter commented Nov 8, 2023 • edited Loading

Codecov Report

michel2323 commented Nov 8, 2023 • edited Loading

wsmoses commented Nov 20, 2023

wsmoses commented Nov 20, 2023

github-actions bot Apr 1, 2024

Choose a reason for hiding this comment

github-actions bot Apr 1, 2024

Choose a reason for hiding this comment

github-actions bot Apr 1, 2024

Choose a reason for hiding this comment

github-actions bot Apr 1, 2024

Choose a reason for hiding this comment

github-actions bot Apr 1, 2024

Choose a reason for hiding this comment

github-actions bot Apr 1, 2024

Choose a reason for hiding this comment

github-actions bot Apr 1, 2024

Choose a reason for hiding this comment

github-actions bot Apr 1, 2024

Choose a reason for hiding this comment

github-actions bot Apr 1, 2024

Choose a reason for hiding this comment

github-actions bot Apr 1, 2024

Choose a reason for hiding this comment

michel2323 commented Apr 1, 2024 • edited Loading

codecov-commenter commented Nov 8, 2023 •

edited

Loading

michel2323 commented Nov 8, 2023 •

edited

Loading

michel2323 commented Apr 1, 2024 •

edited

Loading