[ndslice] creation/allocation utilities

[9il]

Allocation utilities was not implemented before because I needed a time to get real world experience with ndslice and allocators.

In addition, this PR fixes ugly unittests like

auto slice = (3*4*5*6).iota.array.sliced(3, 4, 5, 6);

Now they looks like

auto slice = iotaSlice(3, 4, 5, 6, 7).slice;

In addition, iotaSlice is faster than sliced iota, it has zero size and zero cost.

Mir

libmir/mir#38
libmir/mir#42
libmir/mir#44

New names

slice

• SliceException
• shape
• slice (createSlice in mir PR)
• makeSlice
• ndarray
• makeNdarray

selection

• iotaSlice/IotaSlice

[9il]

ping @John-Colvin @wilzbach @JackStouffer

[wilzbach]

needs @nogc

[9il]

thanks!

[wilzbach]

ping @John-Colvin @wilzbach @JackStouffer

I already tried my best to review the changes at mir - and I am still happy with the changes :)
Btw @9il I still think that diffing the changes between mir and std.experimental.ndslice will require quite some work in the future, especially when more changes come in at mir.

[9il]

Btw @9il I still think that diffing the changes between mir and std.experimental.ndslice will require quite some work in the future, especially when more changes come in at mir.

I am thinking about automatic convetion tool ;)

[9il]

@wilzbach what do you think about to rename createSlice to slice? We already have for example array and makeArray, ndarray and makeNdarray.

[wilzbach]

why is there so much whitespace here?

[9il]

allgned with lines below

[JackStouffer]

Besides testing, what's the rationale for iotaSlice?

[wilzbach]

@wilzbach what do you think about to rename createSlice to slice? We already have for example array and makeArray, ndarray and makeNdarray.

I was actually thinking about this too. Initially I wanted to suggest that we use the same method name for both the gc and allocation version, but prefixing it with make does make sense. So +1 for slice and a common naming convention.

[9il]

Besides testing, what's the rationale for iotaSlice?

This is good building block for abstract algorithms along with indexSlice.

[wilzbach]

could you add a test for a higher-dimensional array?
We only test 2d in all tests.

[9il]

2D is OK for recursion algorithms. btw, amount of unittests in ndslice already largest in Phobos except std.datetime.

[wilzbach]

2D is OK for recursion algorithms

I was just worried that we create a bias with the tests by only testing 2d.

amount of unittests in ndslice already largest in Phobos except std.datetime

How did you measure this?

Last time I checked there were some parts in Phobos with 100% coverage.

[9il]

Last time I checked there were some parts in Phobos with 100% coverage.

I mean amount of LOC

[9il]

How did you measure this?

I just know probos pretty well)

[wilzbach]

needs rebase :(

[PetarKirov]

@9il I have an alternative idea about the allocation API:
Merge ndarray and makeNdarray into a single function and use alias template parameter for the allocator:

// Before:
auto ndarray(size_t N, Range)
    (auto ref Slice!(N, Range) slice);
auto makeNdarray(T, Allocator, size_t N, Range)
    (auto ref Allocator alloc,  Slice!(N, Range) slice);

// After:
auto makeNdarray(alias allocator = GCAllocator, size_t N, Range)
    (auto ref Slice!(N, Range) slice)
{
    alias alloc = allocatorInstanceOf!(allocator);
    alias T = ElementType!Range;

    // same as before...
}

auto m1 = slice.makeNdarray;
auto m2 = slice.makeNdarray!MmapAllocator;
auto m3 = slice.makeNdarray!theAllocator;
auto m4 = slice.makeNdarray!(Mallocator.instance);

(Ditto for makeSlice).

Full example: http://dpaste.dzfl.pl/c3894699a269

[wilzbach]

Merge ndarray and makeNdarray into a single function and use alias template parameter for the allocator:

We tried this before, but unfortunately GCAllocator isn't pure :/

[9il]

Merge ndarray and makeNdarray into a single function and use alias template parameter for the allocator:
We tried this before, but unfortunately GCAllocator isn't pure :/

In addition, current API is similar to array and makeArray+dispose. ndarray is good name, btw)

[PetarKirov]

@wilzbach

We tried this before, but unfortunately GCAllocator isn't pure :/

Then we should fix GCAllocator ;) I find std.experimental.allocator surprisingly lacking in terms of attributes. I did add some, but I didn't look at gc_allocator and mmap_allocator because I wasn't needing them at the time. pure is going to be the hard one because Allocator.instance is mutable and shared. Also, it is debatable whether this attribute should be used for anything but GCAllocator.allocate.
OTOH, pure-ity is easy for stateful allocators - example.

@9il

In addition, current API is similar to array and makeArray+dispose. ndarray is good name, btw)

OK, but what do you think about only changing makeNdarray to the the API I proposed, which IMO is easier / more flexible to use:

auto makeNdarray(alias allocator = GCAllocator, size_t N, Range)
    (auto ref Slice!(N, Range) slice);

auto m1 = slice.makeNdarray;
auto m2 = slice.makeNdarray!MmapAllocator;
auto m3 = slice.makeNdarray!theAllocator;
auto m4 = slice.makeNdarray!(Mallocator.instance);

[9il]

OK, but what do you think about only changing makeNdarray to the the API I proposed, which IMO is easier / more flexible to use:

If @andralex change it for makeArray too.

[wilzbach]

Then we should fix GCAllocator ;) [...] pure is going to be the hard one because Allocator.instance is mutable and shared. On the other hand, it is debatable whether this attribute should be used for anything but GCAllocator.allocate.

I submitted #4190 which hopefully explains the problem that requires us to provide two functions for everything that uses the allocation API.

[9il]

@DmitryOlshansky Could you please merge this. It is blocker for other fixes ( #4203 ) and Mir.

[DmitryOlshansky]

Auto-merge toggled on

[9il]

Thanks!