make openArray first-class

[timotheecour]

I didn't find an issue to track this, it's only mentioned in comments in random places, eg [1][2].

proposal

allow:

let (pr, n) : (ptr int, int) = get_pointer_length(...) # eg, from C proc, etc
let a=toOpenArray(pr, 0, n-1) # a is openArray[int]

openArray can be implemented as something similar to:

type openArray[T] = object
  data: ptr T
  len: int

It could be enabled via a -d:unsafe_openArray_firstclass (and assert at runtime with this flag is not passed)

links

[1] nim-lang/Nim#5957 (comment)

Stylistic issues aside, it's mostly fine but I think it would be ill advised to introduce this. Once openArray and var openArray work with some minor Rust inspired borrow checking to make it safe (RFC soon to be written) there would be considerable overlap with this stdlib module.

[2] nitely/nim-graphemes#4

I'll have to wait until OpenArray can be used as a return value

• ICE's and bugs with openarray #8259

EDIT

• https://github.com/status-im/nim-ranges that supports (perhaps unsafe) slicing
• D's dynamic arrays that support safe slicing without reallocating https://dlang.org/articles/d-array-article.html

[Araq]

I don't see how this can be supported in a memory-safe way. Well ok, I do know how to do that (borrow checking), but I don't think it's worth the costs.

[timotheecour]

hence the -d:unsafe_openArray_firstclass which means: the programmer knows what he's doing, he's responsible for making sure memory isn't corrupted and using it at his own risk, just like cast[T]; it enables valid use cases in particular for performance reasons, bypassing copies.
This type of escape hatch should be available in a systems programming language.

[Araq]

If he knows what he is doing he is free to use ptr UncheckedArray[T] already.

[mratsim]

Duplicate of closed nim-lang/Nim#5437

Discussions with several proposals to read: https://github.com/nim-lang/Nim/issues/7337#issuecomment-373444925. (in the context of openarray[byte]/raw memory blob)

I feel like we are closing Openarrays/Views/Slices/Ranges/Spans feature requests very early.

[timotheecour]

If he knows what he is doing he is free to use ptr UncheckedArray[T] already.

UncheckedArray[T] doesn't have a len field, so it's not a suitable replacement to all the proposals that have been made so far regarding having a Openarrays/Views/Slices/Ranges/Spans

[Araq]

UncheckedArray[T] doesn't have a len field, so it's not a suitable replacement to all the proposals that have been made so far regarding having a Openarrays/Views/Slices/Ranges/Spans

Well but you want to slice into the data so you want control over the length too.

[timotheecour]

Well but you want to slice into the data so you want control over the length too.

yes, but as in all (or most?) proposals that I've seen, the length should be part of the object returned by toOpenArray / toSpan / etc:

let (pr, n) : (ptr int, int) = get_pointer_length(...) # eg, from C proc, etc
# with 1st class openArray:
let a=toOpenArray(pr, 0, n-1) # a is openArray[int]
doAssert a.len == n

# alternatively, if openArray can't be made 1st class for some reason: with Span:
let a=toSpan(pr, 0, n-1) # a is Span[int]
doAssert a.len == n

UncheckedArray[T] doesn't give you that. And OpenArray[T] isn't usable as a 1st class entity.

[timotheecour]

/cc @Araq
can we re-open? it was closed automatically by github's (over-reaching IMO) heuristic due to the following words close #9001 in PR msg I don't know if that improves the ergonomics enough to close #9001, but it is a pretty simple change maybe worth consideration in nim-lang/Nim#9316 (comment) by @c-blake
indeed, commit msg 1b3725e3954ca01e5f37e82845549bdcce7d3901 says: This PR relates to issue https://github.com/nim-lang/Nim/issues/9001 (relates, not closes) ; this auto-closing is nice but has false positives (happened a few times); btw it can be turned off in github settings but maybe it's worth living with the few false positives

/cc @mratsim also for your comment I feel like we are closing Openarrays/Views/Slices/Ranges/Spans feature requests very early.

[metagn]

--experimental:views should be enough to close this? Given #12 was closed

[timotheecour]

It's far from a working design, and prevents many use cases, I don't think we should close this.

[Araq]

We need a follow-up RFC.

[Araq]

It works reasonably well and things are improving incrementally.