Doc/dps arrays

[Divesh-Otwani]

I documented DPS arrays (and did some background reading on them).

(By the way, I don't see how this implementation reduces the number of allocations; it seems like it doesn't actually help -- though I'm probably missing something. The example I made is doable with exactly one allocation of a temporary array of size n in C. This is not the case here.)

[aspiwack]

I don't know what your question actually is. But if what you are trying to do is compare mutable arrays and destinations, my suggestion is to see how the interaction of split and freeze (the latter being part of the allocation primitive in destinations) is different.

[aspiwack]

@utdemir I'll leave it to you to review this PR, too.

[utdemir]

It looks like I am not the best person to review this, since I do not completely understand in which cases I would use this module, instead of Vector.fromList or Pull/Push arrays.

[utdemir]

I am probably mistaken, but I don't see how just using this module makes the programs more efficient. If I'm wrong, it'd be nice to explain why here :).

[utdemir]

After reading Arnaud's explanation, I think we should reword this last sentence with something like:

Destination arrays, makes it possible to reduce the amount of allocations
without relying on compiler optimisations or fusion rules.

[utdemir]

Instead of describing an IO action, I think this would be a shorter example if we were to just describe a Vector Int -> Vector Int function. eg.

computeDiff :: Vector Int -> DPS.DArray Int #-> ()
computeDiff = undefined

vectorDiff :: Vector Int -> Vector Int
vectorDiff vec =
  let diffSize = (Vector.length vec) - 1
  in  DPS.alloc diffSize (computeDiff vec)

[utdemir]

Typo here on destination.

[utdemir]

"because of linear types" sounds a bit hand-wavy. The explanation after this sentence is clear, so maybe we should just start with that ("The only way to create ...").

[aspiwack]

@utdemir I believe, on the contrary, that it makes you the best person to review: the documentation is supposed to make you understand why you want to use this.

I'm available for specific questions though.

[utdemir]

@aspiwack I think I am missing the main point, because I still don't understand how this can be more efficient than just calling something like Vector.fromListN inside the function.

The only thing I can think of is that creating a DArray once, splitting them and filling using two different functions would be faster than combining two vectors after the fact. But the module gives the impression that it is not the only point.

[aspiwack]

Sorry, sorry, I forgot to reply to this one. I promise that I'm catching up with my email and stuff.

“More efficient” is a bit vague. The general way that I think about destinations is that taking a destination instead of returning an array lets you ask a new question, namely: whose responsibility is it to allocate the array. When you return an array, you have no choice: it is you who will allocate the array. Where as when you take a destination as an argument, it can be someone else.

At this point, exercise: show that from a dps function f :: a -> DArray b #-> (), you can define an array-returning function f :: a -> Array b. So DPS is indeed the more general form.

There can be many reasons for doing such a thing: destinations can be split. So I can give one part to a thread, and another part to another thread. If I used array-returning functions, both of these thread would allocate an array, and I would have to copy them into a new array on my way out. Vector depends on array-fusion to avoid such extra-allocation, but it's really not reliable (if these are really threads (e.g. async), vector fusion will not trigger). Another case where fusion will most certainly not work, is when you want to fill in a memory-mapped buffers, since it is not a normal vector thing.

Further reading:

• Using DPS for memory-efficient code generation: https://www.microsoft.com/en-us/research/publication/using-destination-passing-style-compile-functional-language-efficient-low-level-code/
• My Haskell Exchange 2017 talk speaks about destination quite a bit: https://skillsmatter.com/skillscasts/10637-distributed-programming-with-linear-types

[Divesh-Otwani]

Closing in favor of #238.