Allow =destroy on pointer types. (previously experimental feature)

[mratsim]

The following used to work:

import opencl

{.experimental.}

type clResource = PCommandQueue | PKernel | PProgram | PMem | PContext

template release*(queue: PCommandQueue) = check releaseCommandQueue(queue)
template release*(kernel: PKernel) = check releaseKernel(kernel)
template release*(program: PProgram) = check releaseProgram(program)
template release*(buffer: PMem) = check releaseMemObject(buffer)
template release*(context: PContext) = check releaseContext(context)

proc `=destroy`*(clres: clResource) =
  release clres

The error now is:
Error: signature for '=destroy' must be proc[T: object](x: var T)

Looking at the previous code we had this:

Nim/compiler/semstmts.nim

Lines 1364 to 1387 in 8ce9e43

	proc semOverride(c: PContext, s: PSym, n: PNode) =
	case s.name.s.normalize
	of "destroy", "=destroy":
	if newDestructors:
	let t = s.typ
	var noError = false
	if t.len == 2 and t.sons[0] == nil and t.sons[1].kind == tyVar:
	var obj = t.sons[1].sons[0]
	while true:
	incl(obj.flags, tfHasAsgn)
	if obj.kind in {tyGenericBody, tyGenericInst}: obj = obj.lastSon
	elif obj.kind == tyGenericInvocation: obj = obj.sons[0]
	else: break
	if obj.kind in {tyObject, tyDistinct}:
	if obj.destructor.isNil:
	obj.destructor = s
	else:
	localError(n.info, errGenerated,
	"cannot bind another '" & s.name.s & "' to: " & typeToString(obj))
	noError = true
	if not noError and sfSystemModule notin s.owner.flags:
	localError(n.info, errGenerated,
	"signature for '" & s.name.s & "' must be proc[T: object](x: var T)")
	incl(s.flags, sfUsed)

And now:

Nim/compiler/semstmts.nim

Lines 1309 to 1331 in 6080178

	proc semOverride(c: PContext, s: PSym, n: PNode) =
	case s.name.s.normalize
	of "=destroy":
	let t = s.typ
	var noError = false
	if t.len == 2 and t.sons[0] == nil and t.sons[1].kind == tyVar:
	var obj = t.sons[1].sons[0]
	while true:
	incl(obj.flags, tfHasAsgn)
	if obj.kind in {tyGenericBody, tyGenericInst}: obj = obj.lastSon
	elif obj.kind == tyGenericInvocation: obj = obj.sons[0]
	else: break
	if obj.kind in {tyObject, tyDistinct}:
	if obj.destructor.isNil:
	obj.destructor = s
	else:
	localError(n.info, errGenerated,
	"cannot bind another '" & s.name.s & "' to: " & typeToString(obj))
	noError = true
	if not noError and sfSystemModule notin s.owner.flags:
	localError(n.info, errGenerated,
	"signature for '" & s.name.s & "' must be proc[T: object](x: var T)")
	incl(s.flags, sfUsed)

So it seems like the old experimental destructors were removed.

There is a plan to have destructors replace finalizer for ref objects. It would be also very useful to have destructors for ptr types especially for wrapping C libraries that gives an opaque type and the corresponding destroy/release proc.

[genotrance]

Simpler example to reproduce this:

type
  abc = ref object

proc `=destroy`(a: var abc) =
  echo "hello"

let a = abc()

ctest.nim(4, 1) Error: signature for '=destroy' must be proc[T: object](x: var T)

Note that this works just fine with #head on Windows but not on Linux.

[Araq]

Not gonna happen, the destructor design is for objects (it's already complex enough IMO) but it's easy enough to wrap a ptr in an object.

[genotrance]

But abc is a plain object in my example. And why does it work on Windows?

[Araq]

because your Nim version is only up-to-date on Windows?

[genotrance]

Okay, my Windows version was 0.18.0, sorry about that. Using #head on both, I get the same error if abc = ref object but per your comment, that won't be fixed.

However, with abc = object, I get:

Error: internal error: '=sink' operator not found for type abc

[Araq]

However, with abc = object, I get:

Works for me with --newruntime.

[genotrance]

It works with --newruntime but I see weird behavior.

type
  abc = object
    a: int
  
proc `=destroy`(a: var abc) =
  echo "Destroy"
    
echo "Start"
var c = abc()
echo "Preset"
c.a = 5
echo c.a
echo "Done"

Output:

Start
Destroy                   <=== should be after Done
Preset
5
Done

[genotrance]

Even more contrived example, along with some special weirdness.

type
  abc = object
    a: ptr UncheckedArray[int8]

proc prnt(a: ptr UncheckedArray[int8]) =
  for i in 0 .. 7:
    stdout.write(a[i])
  for i in 0 .. 7:
    echo a[i]

proc newAbc(): abc =
  echo "Allocate"
  result = abc()
  result.a = cast[ptr UncheckedArray[int8]](alloc0(8))
  result.a.prnt()

proc `=destroy`(a: var abc) =
  echo "Destroy"
  a.a.dealloc()
    
echo "Start"
var c = newAbc()
c.a.prnt()
echo "Preset"
c.a[4] = 4
c.a.prnt()
echo "Done"

Output:

Start
Allocate
000000000
0
0
0
0
0
0
0
Destroy
000000000                        <= reading deallocated memory
0
0
0
16                               <= garbage seen with echo but not with stdout.write()
0
0
0
Preset                           <= set a[5] = 4 in deallocated memory
000040000                        <= evidenced with stdout.write()
0
0
0
16                               <= still garbage with echo
0
0
0
Done

If I comment out the destructor, it all works as expected, but obviously memory isn't freed.

Start
Allocate
000000000
0
0
0
0
0
0
0
000000000
0
0
0
0
0
0
0
Preset
000040000
0
0
0
4
0
0
0
Done

[genotrance]

Looks like this is basically #8035. I should have searched first.

@garetbass, @andreaferretti @zah

I'm still curious why echo and stdout.write print out different values.

[mratsim]

Here is another use case for at least a lent lifetime analysis on pointer types.

For optimization purposes I need to take a ptr UncheckedArray[T] to the raw data from a Tensor[T] and apply compiler hints like __restrict__ and __builtin_assume_aligned.

This can be done in nim with:

{.pragma: restrict, codegenDecl: "$# __restrict__ $#".}
proc builtin_assume_aligned[T](data: ptr T, n: csize): ptr T {.importc: "__builtin_assume_aligned",noDecl.}

let x = @[0, 1, 2, 3, 4]

let x_ptr{.restrict.} = builtin_assume_aligned(x[0].unsafeAddr, 16) # I don't actually know the alignment of seq types

Unfortunately x_ptr can escape its scope.

At initial glance, we could think wrapping in an object to use with destructors would work

type MyPtr[T] = object
  p: ptr T

but you can only use lent if this type is already a part of another type (i.e. has an address). And if it already has an address you cannot apply codegenDecl pragma to it.

I see 3 solutions 4 solutions:

1. Pointers are unsafe, and by using them all bets are off. You should know what you're doing.

2. Add the possibility to codegenDecl in a type: that would allow restrict though it would not help for assume-aligned

3. Tag pointers with lent or a lifetime/escape analysis pragma

Edited:
4. Openarray as value (not sure how that would work with codegenDecl then)