Skip to content

Commit

Permalink
Merge pull request #156 from traktofon/for07
Browse files Browse the repository at this point in the history
Update for julia-0.7
  • Loading branch information
andreasnoack committed Jul 28, 2018
2 parents 76d8be4 + 810e142 commit 1241687
Show file tree
Hide file tree
Showing 14 changed files with 577 additions and 415 deletions.
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ os:
- linux
- osx
julia:
- 0.6
- 0.7
- nightly
matrix:
# allow_failures:
Expand Down
29 changes: 14 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,10 @@
Distributed Arrays for Julia

***NOTE***
Distributed Arrays will only work on Julia v0.4.0 or later.

`DArray`s have been removed from Julia Base library in v0.4 so it is now necessary to import the `DistributedArrays` package on all spawned processes.
This package will only work on Julia v0.7 or later.

```julia
@everywhere using DistributedArrays
using DistributedArrays
```

Distributed Arrays
Expand Down Expand Up @@ -76,12 +74,12 @@ Indexing via symbols is used for this, specifically symbols `:L`,`:LP`,`:l`,`:lp
are all equivalent. For example, `d[:L]` returns the localpart of `d`
while `d[:L]=v` sets `v` as the localpart of `d`.

* `localindexes(a::DArray)` gives a tuple of the index ranges owned by the
* `localindices(a::DArray)` gives a tuple of the index ranges owned by the
local process.

* `convert(Array, a::DArray)` brings all the data to the local process.

Indexing a `DArray` (square brackets) with ranges of indexes always
Indexing a `DArray` (square brackets) with ranges of indices always
creates a `SubArray`, not copying any data.


Expand Down Expand Up @@ -154,7 +152,7 @@ following code accomplishes this::
left = mod(first(I[2])-2,size(d,2))+1
right = mod( last(I[2]) ,size(d,2))+1

old = Array(Bool, length(I[1])+2, length(I[2])+2)
old = Array{Bool}(undef, length(I[1])+2, length(I[2])+2)
old[1 , 1 ] = d[top , left] # left side
old[2:end-1, 1 ] = d[I[1], left]
old[end , 1 ] = d[bot , left]
Expand Down Expand Up @@ -205,7 +203,7 @@ seen in this simple example:
```julia
julia> addprocs(8);

julia> @everywhere using DistributedArrays
julia> using DistributedArrays

julia> A = fill(1.1, (100,100));

Expand All @@ -227,7 +225,7 @@ Garbage Collection and DArrays
------------------------------

When a DArray is constructed (typically on the master process), the returned DArray objects stores information on how the
array is distributed, which procesor holds which indexes and so on. When the DArray object
array is distributed, which procesor holds which indices and so on. When the DArray object
on the master process is garbage collected, all particpating workers are notified and
localparts of the DArray freed on each worker.

Expand Down Expand Up @@ -317,18 +315,19 @@ Example
This toy example exchanges data with each of its neighbors `n` times.

```
using Distributed
using DistributedArrays
addprocs(8)
@everywhere importall DistributedArrays
@everywhere importall DistributedArrays.SPMD
@everywhere using DistributedArrays
@everywhere using DistributedArrays.SPMD
d_in=d=DArray(I->fill(myid(), (map(length,I)...)), (nworkers(), 2), workers(), [nworkers(),1])
d_out=ddata()
d_in=d=DArray(I->fill(myid(), (map(length,I)...,)), (nworkers(), 2), workers(), [nworkers(),1])
d_out=ddata();
# define the function everywhere
@everywhere function foo_spmd(d_in, d_out, n)
pids = sort(vec(procs(d_in)))
pididx = findfirst(pids, myid())
pididx = findfirst(isequal(myid()), pids)
mylp = d_in[:L]
localsum = 0
Expand All @@ -352,7 +351,7 @@ d_out=ddata()
end
# run foo_spmd on all workers
spmd(foo_spmd, d_in, d_out, 10)
spmd(foo_spmd, d_in, d_out, 10, pids=workers())
# print values of d_in and d_out after the run
println(d_in)
Expand Down
2 changes: 1 addition & 1 deletion REQUIRE
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
julia 0.6
julia 0.7-beta
Primes
14 changes: 9 additions & 5 deletions src/DistributedArrays.jl
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,20 @@ __precompile__()

module DistributedArrays

importall Base
using Distributed
using Serialization
using LinearAlgebra

import Base: +, -, *, div, mod, rem, &, |, xor
import Base.Callable
import Base.BLAS: axpy!
import LinearAlgebra: axpy!, dot, norm

using Primes
using Primes: factor
import Primes
import Primes: factor

# DArray exports
export DArray, SubDArray, SubOrDArray, @DArray
export dzeros, dones, dfill, drand, drandn, distribute, localpart, localindexes, ppeval
export dzeros, dones, dfill, drand, drandn, distribute, localpart, localindices, ppeval

# non-array distributed data
export ddata, gather
Expand Down
11 changes: 6 additions & 5 deletions src/core.jl
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ release_localpart(id::Tuple) = (delete!(registry, id); nothing)
release_localpart(d) = release_localpart(d.id)

function close_by_id(id, pids)
# @schedule println("Finalizer for : ", id)
# @async println("Finalizer for : ", id)
global refs
@sync begin
for p in pids
Expand All @@ -31,10 +31,10 @@ function close_by_id(id, pids)
nothing
end

function close(d::DArray)
# @schedule println("close : ", d.id, ", object_id : ", object_id(d), ", myid : ", myid() )
function Base.close(d::DArray)
# @async println("close : ", d.id, ", object_id : ", object_id(d), ", myid : ", myid() )
if (myid() == d.id[1]) && d.release
@schedule close_by_id(d.id, d.pids)
@async close_by_id(d.id, d.pids)
d.release = false
end
nothing
Expand All @@ -55,7 +55,8 @@ end
Get the vector of processes storing pieces of DArray `d`.
"""
Base.procs(d::DArray) = d.pids
Distributed.procs(d::DArray) = d.pids
Distributed.procs(d::SubDArray) = procs(parent(d))

"""
localpart(A)
Expand Down

0 comments on commit 1241687

Please sign in to comment.