Make n-d arrays conform to Julia's iterator protocol

README.md

@@ -1,19 +1,20 @@
 # DotNET.jl
 
-![Build Status](https://github.com/azurefx/DotNET.jl/actions/workflows/ci.yml/badge.svg)
-![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com)
+[![Build Status](https://github.com/azurefx/DotNET.jl/actions/workflows/ci.yml/badge.svg)](https://github.com/azurefx/DotNET.jl/actions/workflows/ci.yml)
+[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com)
 
 This package provides interoperability between Julia and [`Common Language Runtime`](https://docs.microsoft.com/dotnet/standard/clr), the execution engine of `.NET` applications. Many languages run on CLR, including `C#`, `Visual Basic .NET` and `PowerShell`.
 
 ## Prerequisites
 
-You will need to have `.NET Core` runtime 2.0 or higher installed on the machine ([Download](https://dotnet.microsoft.com/download)). If the package fails to locate the runtime, set `DOTNET_ROOT` environment variable to the path containing the `dotnet` or `dotnet.exe` binary.
+- `Julia` version 1.3+
+- `.NET Core Runtime` version 2.2+ ([Download](https://dotnet.microsoft.com/download))
 
-Using `WinForms` and GUI-related things requires a [desktop runtime](https://github.com/azurefx/DotNET.jl/issues/11).
+`WinForms` and other GUI-related features require a [desktop runtime](https://github.com/azurefx/DotNET.jl/issues/11).
 
-⚠ `.NET Framework` is currently not supported.
+If the package fails to locate the runtime, set `DOTNET_ROOT` environment variable to the path containing the `dotnet` or `dotnet.exe` binary.
 
-This package uses `Artifacts` to provide binary dependencies, so Julia 1.3+ is required.
+`.NET Framework` is not supported yet.
 
 ## Installation
 
@@ -22,12 +23,12 @@ In the REPL, type `]add DotNET` and press `Enter`.
 (v1.x) pkg> add DotNET
 ```
 
-or use `Pkg.add` for [more options](https://pkgdocs.julialang.org/v1/api/):
+Or use `Pkg.add` for [more options](https://pkgdocs.julialang.org/v1/api/):
 
 ```julia
 julia> using Pkg
 
-julia> Pkg.add(PackageSpec(url="https://github.com/azurefx/DotNET.jl"))
+julia> Pkg.add(PackageSpec(url = "https://github.com/azurefx/DotNET.jl"))
 ```
 
 ## Usage
@@ -36,63 +37,183 @@ julia> Pkg.add(PackageSpec(url="https://github.com/azurefx/DotNET.jl"))
 julia> using DotNET
 ```
 
-1. Use [`T"AssemblyQualifiedTypeName"`](https://docs.microsoft.com/dotnet/standard/assembly/find-fully-qualified-name) to address a type:
+### Types and Objects
+
+`DotNET.jl` provides the [`T"AssemblyQualifiedTypeName"`](https://docs.microsoft.com/dotnet/standard/assembly/find-fully-qualified-name) literal for type reference:
 
 ```julia
-julia> Console=T"System.Console, mscorlib"
+julia> Console = T"System.Console, mscorlib"
 System.Console
 ```
 
-2. Use `.` to access a member:
+Given a type object, you can access its properties or methods using the dot operator:
+
+```julia
+julia> Console.WriteLine("Hello from .NET!");
+Hello from .NET!
+```
+
+To create an object, use the `new` syntax:
+
+```julia
+julia> T"System.Guid".new("CA761232-ED42-11CE-BACD-00AA0057B223")
+System.Guid("ca761232-ed42-11ce-bacd-00aa0057b223")
+```
+
+All `.NET` objects are represented by `CLRObject`s in Julia, including types:
+
+```julia
+julia> typeof(Console)
+CLRObject
+
+julia> typeof(null)
+CLRObject
+```
+
+`null` is a built-in object that does not refer to a valid `.NET` object. When you try to access a member on a `null` value, a `System.NullReferenceException` is thrown.
+
+Arguments passed to `.NET` methods are automatically converted to `CLRObject`s, and return values are converted to corresponding Julia types:
+
+```julia
+julia> T"System.Convert".ToInt64("42")
+42
+```
+
+Or you could do some explicit conversions:
+
+```julia
+julia> s = convert(CLRObject, "❤")
+System.String("❤")
+
+julia> DotNET.unbox(s)
+"❤"
+```
+
+### Arrays and Collections
+
+To copy a multidimensional array from `.NET` to Julia, use `collect` method:
+
+```julia
+julia> arr = convert(CLRObject, reshape(1:8, 2, 2, 2))
+System.Int64[,,]("System.Int64[,,]")
+
+julia> collect(arr)
+2×2×2 Array{Int64, 3}:
+[:, :, 1] =
+ 1  3
+ 2  4
+
+[:, :, 2] =
+ 5  7
+ 6  8
+```
+
+CLI `Array` elements are stored in *row-major* order, thus the equivalent definition in `C#` is
+```csharp
+public static int[,,] Get3DArray() {
+  return new int[2, 2, 2] {
+    {{1, 2}, {3, 4}},
+    {{5, 6}, {7, 8}}
+  };
+}
+```
+
+To index into arrays, use `arraystore` and `arrayload` methods. Note that CLI `Array`s use zero-based indexing.
+
+```julia
+julia> DotNET.arraystore(arr, (1, 1, 1), 0)
+null
+
+julia> DotNET.arrayload(arr, (1, 1, 1)) == collect(arr)[2, 2, 2]
+true
+```
+
+If an object implements `IEnumerable` interface, you can call `GetEnumerator` to iterate over the array:
+```julia
+julia> ch = Channel() do it
+           e = arr.GetEnumerator()
+           while e.MoveNext()
+               put!(it, e.Current)
+           end
+       end
+Channel{Any}(0) (1 item available)
+
+julia> collect(ch)
+8-element Vector{Any}:
+ 1
+ ⋮
+ 8
+```
 
+Or just use the `for-in` loop:
 ```julia
-julia> Console.WriteLine("Hello, CLR!");
-Hello, CLR!
+for x in arr
+    println(x)
+end
 ```
 
-3. Use reflection to load assemblies from file:
+### Loading External Assemblies
+
+If you have a `DLL` file, you can load it using reflection:
 
 ```julia
 julia> T"System.Reflection.Assembly".LoadFrom(raw"C:\Users\Azure\Desktop\test.dll")
 System.Reflection.RuntimeAssembly("test, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null")
 ```
 
-4. To create an object:
+Now you have access to types defined in the assembly.
 
-```julia
-julia> T"System.String".new('6',Int32(3))
-"666"
+### Generics
+
+Generic types are be expressed in the following ways:
 
-julia> List=T"System.Collections.Generic.List`1"
+```julia
+julia> ListT = T"System.Collections.Generic.List`1"
 System.Collections.Generic.List`1[T]
 
-julia> List.new[T"System.Int64"]()
-System.Collections.Generic.List`1[System.Int64]("System.Collections.Generic.List`1[System.Int64]")
+julia> ListInt64 = T"System.Collections.Generic.List`1[System.Int64]"
+System.Collections.Generic.List`1[System.Int64]
+```
+
+The number `1` after the backtick indicates the type `System.Collections.Generic.List<T>` has one type parameter. `ListT` has a free type variable, just like `Vector{T} where T` in Julia. A type that includes at least one type argument is called a *constructed type*. `ListInt64` is a constructed type.
+
+One can substitute type variables and make a constructed type by calling `makegenerictype` method:
+
+```julia
+julia> DotNET.makegenerictype(ListT, T"System.String")
+System.Collections.Generic.List`1[System.String]
+```
+
 
+To invoke a generic method, put type arguments into square brackets:
+```julia
+julia> list = ListT.new[T"System.Int64"]()
+System.Collections.Generic.List`1[System.Int64]("System.Collections.Generic.List`1[System.Int64]")
 ```
 
-5. To create delegates from Julia methods:
+### Delegates
+
+To create a delegate from a Julia method, use `delegate` method:
 
 ```julia
-julia> list=List.new[T"System.Int64"](1:5);
+julia> list = ListT.new[T"System.Int64"](1:5)
+System.Collections.Generic.List`1[System.Int64]("System.Collections.Generic.List`1[System.Int64]")
 
 julia> list.RemoveAll(delegate(iseven,T"System.Predicate`1[System.Int64]"))
 2
 
 julia> collect(list)
-3-element Array{Int64,1}:
+3-element Vector{Int64}:
  1
  3
  5
 ```
 
-## TODO
-
+<!-- 
 - Implicit conversions when calling CLR methods
 - More operators
 - `using` directive like C#
 - Smart assembly/type resolution
 - Configurable runtime versions
-- Julia type system consistency
 - .NET Framework support
-- PowerShell support (maybe in another package)
+- PowerShell support (maybe in another package) -->

src/array.jl

@@ -30,9 +30,11 @@ function Base.iterate(obj::CLRObject)
             return iterate_ienumerable(enumerator)
         end
     end
-    throw(ArgumentError("Object is not iterable"))
+    (unbox(obj), nothing)
 end
 
+Base.iterate(::CLRObject, ::Nothing) = nothing
+
 function Base.iterate(::CLRObject, state)
     enumerator, enumeratorty = state
     hasnext = invokemember(enumeratorty, enumerator, :MoveNext)
@@ -51,7 +53,7 @@ end
 
 function Base.eltype(obj::CLRObject)
     elt = clreltype(obj)
-    typestr = string(isnull(elt) ? clrtypeof(obj) : elt)
+    typestr = string(isnull(elt) ? Any : elt)
     return if haskey(TYPES_TO_UNBOX, typestr)
         TYPES_TO_UNBOX[typestr][1]
     else
@@ -66,7 +68,31 @@ function Base.length(obj::CLRObject)
     elseif isassignable(T"System.Collections.IList", objty)
         invokemember(obj, :Count)
     else
-        throw(ArgumentError("Cannot determine length from type $objty"))
+        throw(ArgumentError("length() is not supported by $objty"))
+    end
+end
+
+function Base.axes(obj::CLRObject)
+    objty = clrtypeof(obj)
+    if isassignable(T"System.Array", objty)
+        rank = invokemember(obj, :Rank)
+        tuple((invokemember(obj, :GetLength, Int32(d)) for d = 0:rank-1)...)
+    elseif isassignable(T"System.Collections.IList", objty)
+        (Base.OneTo(invokemember(obj, :Count)),)
+    else
+        throw(ArgumentError("axes() is not supported by $objty"))
+    end
+end
+
+function Base.IteratorSize(obj::CLRObject)
+    objty = clrtypeof(obj)
+    if isassignable(T"System.Array", objty)
+        rank = invokemember(obj, :Rank)
+        Base.HasShape{rank}()
+    elseif isassignable(T"System.Collections.IList", objty)
+        Base.HasLength()
+    else
+        Base.SizeUnknown()
     end
 end
 
@@ -86,7 +112,7 @@ makearraytype(ty::CLRObject, rank) = invokemember(ty, :MakeArrayType, rank)
 
 function box(x::AbstractArray{T,N}, handle) where {T,N}
     a = arrayof(boxedtype(T), size(x))
-    clrind = CartesianIndices(size(x))
+    clrind = PermutedDimsArray(CartesianIndices(size(x)), ndims(x):-1:1)
     for i in LinearIndices(x)
         arraystore(a, Tuple(clrind[i]) .- 1, x[i])
     end

src/marshalling.jl

@@ -58,10 +58,12 @@ boxedtype(::Type{Char}) = T"System.Char"
 box(x::String, handle) = CLRBridge.PutString(handle, x)
 boxedtype(::Type{String}) = T"System.String"
 
-function Base.convert(CLRObject, x)
+function Base.convert(::Type{CLRObject}, x)
     CLRBridge.Duplicate(box(x, 1))
 end
 
+Base.convert(::Type{CLRObject}, x::CLRObject) = x
+
 function invokemember(flags, type::CLRObject, this::CLRObject, name, args...)
     boxed = map(args, 1:length(args)) do arg, i
         box(arg, i)

test/runtests.jl

@@ -1,6 +1,6 @@
 using Test
 using DotNET
-import DotNET:unbox
+import DotNET: unbox, arrayof
 
 @testset "Type Loader" begin
     @test T"System.Int64".Name == "Int64"
@@ -10,7 +10,8 @@ import DotNET:unbox
 end # testset
 
 @testset "Marshaller" begin
-    uturn(x) = @test (unbox(convert(CLRObject, x)) === x);true
+    uturn(x) = @test (unbox(convert(CLRObject, x)) === x)
+    true
     uturn(Int8(42))
     uturn(UInt8(42))
     uturn(Int16(42))
@@ -23,8 +24,8 @@ end # testset
     uturn(Float64(42))
     uturn('A')
     uturn("Hello, World!")
-    array = convert(CLRObject, [1,2,3])
-    @test collect(array) == [1,2,3]
+    array = convert(CLRObject, [1, 2, 3])
+    @test collect(array) == [1, 2, 3]
     for (i, x) in enumerate(array)
         @test x == i
     end
@@ -46,11 +47,24 @@ end
     @test T"System.Object, mscorlib".Equals(ref.Target, WeakReference)
 end
 
-@testset "Type System" begin
-    li = convert(CLRObject, Int64[1])
-    @test eltype(collect(li)) == Int64
+@testset "Generics" begin
     List = T"System.Collections.Generic.List`1"
     li = List.new[T"System.Int64"]()
     li.Add(Int64(1))
     @test eltype(collect(li)) == Int64
 end
+
+@testset "Arrays and Iteration" begin
+    @test eltype(convert(CLRObject, Int64[1])) == Int64
+    jlarr = reshape(1:24, (2, 3, 4))
+    arr = convert(CLRObject, jlarr)
+    @test axes(arr) == (2, 3, 4)
+    @test collect(arr) == jlarr
+    ch = Channel() do ch
+        e = arr.GetEnumerator()
+        while e.MoveNext()
+            put!(ch, e.Current)
+        end
+    end
+    @test collect(ch) == jlarr[:]
+end