New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: clarify how unsafe.Pointers may be used to determine offsets between addresses #12445

Open
kortschak opened this Issue Sep 2, 2015 · 33 comments

Comments

Projects
None yet
8 participants
@kortschak
Contributor

kortschak commented Sep 2, 2015

For background see discussion at https://groups.google.com/d/topic/golang-nuts/Uet5p_3JhZs/discussion

Currently it is possible to determine the offset between two values' addresses by finding the difference between uintptrs converted from unsafe.Pointer values. This is the only way to be able to determine whether two slices overlap in memory since the advent of three index slicing and the only way to determine in less than linear time whether blocked sparse slices overlap (relevant for views in constructed 2D slices for e.g. BLAS operations). In the event that a moving GC is implemented this may no longer be safe; if a GC move occurs between taking the conversion to uintptr of address of the first and second values, the offset may be incorrect. Note that the GC move does not otherwise invalidate the offset between overlapping slices or falsely result in non-overlapping being construed as overlapping.

@minux

This comment has been minimized.

Member

minux commented Sep 2, 2015

@kortschak

This comment has been minimized.

Contributor

kortschak commented Sep 2, 2015

unsafe.Offset(a, b) int would be more generalisable - I don't want to just be able to determine whether slices overlap, but also determine if strided blocks within the slice overlap. This requires an offset and a pair of lengths (and element size) - the latter which I can already get by other means.

@kortschak

This comment has been minimized.

Contributor

kortschak commented Oct 22, 2015

To add a data point to this. We now implement strided block overlap detection in github.com/gonum/matrix/mat64, documentation here and detection implementation here. This is done essentially as described in the original discussion, both with unsafe for environments that allow that and via reflect for those that don't.

I don't see a constant time implementation that can be achieved without access to an offset call.

@rsc rsc added this to the Go1.6 milestone Oct 23, 2015

@rsc rsc changed the title from cmd/compiler: clarify how unsafe.Pointers may be used to determine offsets between addresses to cmd/compile: clarify how unsafe.Pointers may be used to determine offsets between addresses Nov 4, 2015

@mdempsky

This comment has been minimized.

Member

mdempsky commented Nov 28, 2015

Something like below seems like it should be safe even under a moving GC.

// Overlap returns whether there exist i and j such that
// a[i*stride] and b[j*stride] address the same variable.
func overlap(a, b []float64, stride uintptr) bool {
    an, bn := uintptr(len(a)), uintptr(len(b))
    if an == 0 || bn == 0 {
        // Degenerate case; empty slices can't overlap.
        return false
    }

    // Keep as unsafe.Pointer for safety under moving GC.
    ap, bp := unsafe.Pointer(&a[0]), unsafe.Pointer(&b[0])

    // If a and b are entirely disjoint, there can't be any overlap.
    if uintptr(bp) - uintptr(ap) >= 8 * an && uintptr(ap) - uintptr(bp) >= 8 * bn {
        return false
    }

    // There is *some* overlap, so ap and bp must point into the same variable,
    // which means we can assume a stable offset between them.
    offset := uintptr(bp) - uintptr(ap)

    // ap and bp must be aligned modulo the stride.
    return offset % (stride * 8) == 0
}
@kortschak

This comment has been minimized.

Contributor

kortschak commented Nov 28, 2015

That does not do what the linked code does. We want to return whether there is an i,j and x,y such that a[i + jstride] and b[x + ystride] address the same variable when the indices are constrained such that they index a subset of the possible blocks that may be described with the given stride over the slices. However it does suggest a possible solution.

// offset returns the number of float64 values b[0] is after a[0].
func offset(a, b []float64) int {
    a0 := unsafe.Pointer(&a[0])
    b0 := unsafe.Pointer(&b[0])

    if a0 == b0 {
        return 0
    }
    if uintptr(a0) < uintptr(b0) {
        return int((uintptr(b0) - uintptr(a0)) / unsafe.Sizeof(float64(0)))
    }
    return -int((uintptr(a0) - uintptr(b0)) / unsafe.Sizeof(float64(0)))
}

and the family friendly version (this one is more tenuous since it depends on values passed back from a reflect call):

// offset returns the number of float64 values b[0] is after a[0].
func offset(a, b []float64) int {
    va0 := reflect.ValueOf(a).Index(0)
    vb0 := reflect.ValueOf(b).Index(0)

    if va0.UnsafeAddr() == vb0.UnsafeAddr() {
        return 0
    }
    if vb0.UnsafeAddr() < va0.UnsafeAddr() {
        return int((vb0.UnsafeAddr() - va0.UnsafeAddr()) / sizeOfFloat64)
    }
    return -int((va0.UnsafeAddr() - vb0.UnsafeAddr()) / sizeOfFloat64)
}

These are then used in the functions also linked above.

Note that it is OK for our use case for the offset to be used after a possible GC move since if a and b are part of the same allocation they move together and so the offset remains valid, and if they are not, an offset cannot become indicative of an overlap. The potential race in the sign condition is troubling though.

@RLH

This comment has been minimized.

Contributor

RLH commented Nov 28, 2015

There are certainly whole classes of concurrent GC algorithms for which
this will not work. GC's using a variation of a Brooks' pointer, such as
proposed for the Red Hat LLVM based GC as well as Sapphire style
collectors. Currently Go does not use these techniques.

On Sat, Nov 28, 2015 at 4:17 AM, Dan Kortschak notifications@github.com
wrote:

That does not do what the linked code does. We want to return whether
there is an i,j and x,y such that a[i + j_stride] and b[x + y_stride]
address the same variable. However it does suggest a possible solution.

// offset returns the number of float64 values b[0] is after a[0].
func offset(a, b []float64) int {
a0 := unsafe.Pointer(&a[0])
b0 := unsafe.Pointer(&b[0])

if a0 == b0 {
    return 0
}
if uintptr(a0) < uintptr(b0) {
    return int((uintptr(b0) - uintptr(a0)) / unsafe.Sizeof(float64(0)))
}
return -int((uintptr(a0) - uintptr(b0)) / unsafe.Sizeof(float64(0)))

}

and the family friendly version (this on is more tenuous since it depends
on values passed back from a reflect call):

// offset returns the number of float64 values b[0] is after a[0].
func offset(a, b []float64) int {
va0 := reflect.ValueOf(a).Index(0)
vb0 := reflect.ValueOf(b).Index(0)

if va0.UnsafeAddr() == vb0.UnsafeAddr() {
    return 0
}
if va0.UnsafeAddr() < vb0.UnsafeAddr() {
    return int((vb0.UnsafeAddr() - va0.UnsafeAddr()) / sizeOfFloat64)
}
return -int((va0.UnsafeAddr() - vb0.UnsafeAddr()) / sizeOfFloat64)

}

These are then used in the functions also linked above.

Note that it is OK for our use case for the offset to be used after a
possible GC move since if a and b are part of the same allocation they move
together and so the offset remains valid, and if they are not, an offset
cannot become indicative of an overlap.


Reply to this email directly or view it on GitHub
#12445 (comment).

@kortschak

This comment has been minimized.

Contributor

kortschak commented Nov 28, 2015

There are certainly whole classes of concurrent GC algorithms for which this will not work.

This is true if the GC algorithms are considered in isolation from the language's promises about behaviour. This issue aims to address that.

Currently there seems to be a de facto promise* see discussion in #7192 and #8994 that use of converted unsafe.Pointers is atomic wrt the GC if the use is in a single expression. This is how the unsafe.Pointer->uintptr happens in the code above, with the exception of the condition which troubles me.

* Not a promise.

Currently Go does not use these techniques.

This is understood; hence the comments in the linked code. The issue for us is that introduction of GC behaviour that breaks this code without a work around to obtain an offset will break gonum/matrix/mat64 behaviour. We use unsafe for the normal case and we don't expect Go1 coverage for that, but we also use the reflect equivalent for environments that disallow unsafe.

@mdempsky

This comment has been minimized.

Member

mdempsky commented Nov 29, 2015

@RLH Sorry, when you say "for which this will not work", what are you referring to by "this"? E.g., would the overlap function I wrote above (setting aside momentarily that it doesn't address @kortschak's actual use case) not be safe under some of those GC implementations?

The Go spec says "For struct s with field f: uintptr(unsafe.Pointer(&s)) + unsafe.Offsetof(s.f) == uintptr(unsafe.Pointer(&s.f))". I've generally extrapolated from this that within a single expression, pointers when converted to uintptr must represent a consistent view of memory, as otherwise the equality wouldn't necessarily hold. Also that other uintptr expressions like uintptr(unsafe.Pointer(&s.f)) - uintptr(unsafe.Pointer(&s)) == unsafe.Offsetof(s.f) and uintptr(unsafe.Pointer(&s.f)) >= uintptr(unsafe.Pointer(&s)) would be valid and guaranteed to evaluate to true.

If those aren't safe extrapolations, perhaps the spec should be revised to only guarantee unsafe.Pointer(uintptr(unsafe.Pointer(&s)) + unsafe.Offsetof(s.f)) == unsafe.Pointer(&s.f).

@kortschak

This comment has been minimized.

Contributor

kortschak commented Nov 29, 2015

@mdempsky I think this is #8994.

@mdempsky

This comment has been minimized.

Member

mdempsky commented Nov 29, 2015

@kortschak They're certainly very related. But my impression is #8994 is about the rules for how you can safely convert unsafe.Pointer to uintptr and back, whereas this issue seems to be more about which arithmetic laws still hold after converting unsafe.Pointer to uintptr. E.g., the overlap/offset functions being discussed above don't care about convert uintptr back to unsafe.Pointer.

@kortschak

This comment has been minimized.

Contributor

kortschak commented Nov 29, 2015

@RLH

This comment has been minimized.

Contributor

RLH commented Nov 29, 2015

Let me try again. Various concurrent moving collectors including those
using a Brooks' barrier as well as the Sapphire algorithm rely on the fact
that two different bit pattern can be used to represent the same (pointer
to an) object. One bit pattern may refer to the "old" version of the object
and another to the "new" version of the object. Depending on the algorithm
a read and/or write barrier is used to ensure that values returned from the
object are consistent. The literature explains how the algorithms
extinguish the "old" version by replacing them with "new" version and what
invariants the read and write barriers need to maintain to ensure object
identity, consistency, completeness, and termination.

This creates implementation issues around the implementation of == which
doesn't dereference the pointer and thus fall outside the barriers used
when the object is dereferenced. This == problem was first noted and solved
in the original Brooks paper as well as being noted, referenced, and solved
in the Sapphire paper. It is important to note that in at least the
Sapphire algorithm the implementation of == is the only place where reads
of (pointers to) objects require special consideration. Go's spec allows
for such an implementation.

The single example found in the unsafe part of the spec seems to extend the
semantics of == to include calls to unsafe.Pointer in the same ==
expression where the compiler can statically determine that the arguments
passed to unsafe.Pointer refer to the same object and eliminate one of the
calls using CSE. The unsafe.Offset call can be replaced with known static
offset to a field within a struct. I would be surprised if the compiler
doesn't already do this. I believe that this extension of == semantics is
tractable and object identity can be maintained.

The examples provided in this thread use unsafe.Pointer in different
statements and then expect the returned bits to be comparable. This is
unsafe though it works in current implementation.

In summary, the statement that "Something like below seems like it should
be safe even under a moving GC." is not true for at least two well known
concurrent copying collectors.

More to the point is that both #7192
#7192 and #8994
#8994 have resisted changing the spec.
What has changed since those decisions? Changing the stable spec is a very
high bar, particularly when the change will alter fundamental concepts like
object identity that GC algorithms have wrestled with for decades.

On Sat, Nov 28, 2015 at 9:32 PM, Dan Kortschak notifications@github.com
wrote:

Yes, that's true, though I think that once the rules for the other issue
are settled that defines what happens here.


Reply to this email directly or view it on GitHub
#12445 (comment).

@mdempsky

This comment has been minimized.

Member

mdempsky commented Nov 29, 2015

@RLH Thanks. I understand the idea that under a moving GC that a single "pointer value" (in the Go spec/language sense) might at a given time have multiple bit patterns at the machine implementation level, and so this complicates the implementation of Go pointer equality.

However, we seem to have different interpretations of the spec's uintptr(unsafe.Pointer(&s)) + unsafe.Offsetof(s.f) == uintptr(unsafe.Pointer(&s.f)) guarantee (which I'll emphasize again is currently written using integer equality, not pointer equality). It sounds like you interpret the guarantee is precisely worded and only holds when &s and &s.f are all locally visible to the compiler so it can be satisfied via CSE optimizations, whereas I've instead interpreted it generalizes to examples like this (even assuming no inlining/cross-package compiler optimizations):

package pkg

import "unsafe"

type S struct { E, F int }

func Func(p, q unsafe.Pointer) bool {
    return uintptr(p) + unsafe.Offsetof(S{}.F) == uintptr(q)
}

.

package main

import (
    "unsafe"
    "./pkg"
)

func main() {
    var s pkg.S
    if !pkg.Func(unsafe.Pointer(&s), unsafe.Pointer(&s.F)) {
        panic("spec violation")
    }
}

So at the very least I think it's worth deciding/clarifying which interpretation of the spec is intended.

@dr2chase

This comment has been minimized.

Contributor

dr2chase commented Nov 30, 2015

The problem with

uintptr(unsafe.Pointer(&s)) + unsafe.Offsetof(s.f) == uintptr(unsafe.Pointer(&s.f))

is that it can be viewed as making assumptions about atomicity with respect to garbage collection. The issue is not rules of integer equality, it is instead about what GC activity might occur between evaluating uintptr(unsafe.Pointer(&s)) and uintptr(unsafe.Pointer(&s.f)). If it is indeed a matter of exactly what the Go spec says, then one cost of some future collector might be the lock/barrier operations needed to obtain that atomicity -- be careful what you are asking for, because you just might get it.

I'd be far happier if the runtime had a variant of kortschak's offset(a,b) function that returned difference int, related bool. Arguing against this is that it might require some extra cost to determine that two pointers are unrelated, arguing for it is that it would keep unsafe stuff out of user's code and would be appropriately future-proof against changes in the garbage collector. Against the extra cost, you have that if the unrelated case is common, then you avoid doing all the calculations with lengths and sizes and proceed directly to code that does updates in any order whatsoever.

To someone who's worked on GC (and optimizers, and concurrency) it is extremely nervous-making to see people so happy to assign precise semantics to unsafe code (and generally, so happy to use unsafe code).

@RLH

This comment has been minimized.

Contributor

RLH commented Nov 30, 2015

Technically I think we understand each other.

Short term expanding the semantics of unsafe will lead to more and
increasingly clever uses of unsafe and that will lead to an increase in
corner case bugs. Clever uses of unsafe coupled with an increasing
sophisticated runtime that depends on the type system being safe is the
road to pain.

Long term expanding the semantics of unsafe will lead to technical debt and
cut off possible solutions that moving objects might solve. Some of the
possible problems include fragmented memory and poor spatial locality
complicated by deeper caches and various as yet undefined NUMA
architectures. We lack visibility on how important these things will be in
the future and whether the advantages of allowing Go to grow in unsafe ways
today outweighs the accumulated technical debt.

I think we should spend some time and see if we can come up with solutions
that lead to less use of unsafe instead of more.

On Sun, Nov 29, 2015 at 2:57 PM, Matthew Dempsky notifications@github.com
wrote:

@RLH https://github.com/RLH Thanks. I understand the idea that under a
moving GC that a single "pointer value" (in the Go spec/language sense)
might at a given time have multiple bit patterns at the machine
implementation level, and so this complicates the implementation of Go
pointer equality.

However, we seem to have different interpretations of the spec's uintptr(unsafe.Pointer(&s))

  • unsafe.Offsetof(s.f) == uintptr(unsafe.Pointer(&s.f)) guarantee (which
    I'll emphasize again is currently written using integer equality, not
    pointer equality). It sounds like you interpret the guarantee is
    precisely worded and only holds when &s and &s.f are all locally visible to
    the compiler so it can be satisfied via CSE optimizations, whereas I've
    instead interpreted it generalizes to examples like this (even assuming no
    inlining/cross-package compiler optimizations):

package pkg

import "unsafe"

type S struct { E, F int }

func Func(p, q unsafe.Pointer) bool {
return uintptr(p) + unsafe.Offsetof(S{}.F) == uintptr(q)
}

.

package main

import (
"unsafe"
"./pkg"
)

func main() {
var s pkg.S
if !pkg.Func(unsafe.Pointer(&s), unsafe.Pointer(&s.F)) {
panic("spec violation")
}
}

So at the very least I think it's worth deciding/clarifying which
interpretation of the spec is intended.


Reply to this email directly or view it on GitHub
#12445 (comment).

@mdempsky

This comment has been minimized.

Member

mdempsky commented Nov 30, 2015

@dr2chase Arguably there are atomicity constraints even in expressions like unsafe.Pointer(uintptr(unsafe.Pointer(&s)) + 1), which is why I'd previously proposed adding a function like func Add(unsafe.Pointer, uintptr) unsafe.Pointer to the standard library / runtime (#7192). However, it was rejected, so I've been operating under the impression that the preferred approach is codifying safe unsafe.Pointer/uintptr conversion patterns.

If adding more primitive operations is on the table, I'd suggest something like:

// Base returns a pointer to the beginning of the outermost enclosing variable,
// and the offset of p into that variable.
// If p does not point into a variable, then it returns (p, 0).
func Base(p unsafe.Pointer) (base unsafe.Pointer, offset uintptr)

So for example, Base(unsafe.Pointer(&s.f)) would return (unsafe.Pointer(&s), unsafe.Offsetof(s.f)).

Then @kortschak's offset can be implemented as:

func offset(p, q []float64) (difference int, related bool) {
    pb, po := Base(unsafe.Pointer(&p[0]))
    qb, qo := Base(unsafe.Pointer(&q[0]))
    if pb != qb {  // GC-aware pointer equality
        return 0, false
    }
    if po < qo {
        return int((qo - po) / unsafe.Sizeof(float64(0)))
    }
    return -int((po - qo) / unsafe.Sizeof(float64(0)))
}
@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Nov 30, 2015

@mdempsky It's not clear to me that unsafe.Add was rejected. It just hasn't been accepted.

@dr2chase

This comment has been minimized.

Contributor

dr2chase commented Nov 30, 2015

Base looks like a nice abstraction. How much do you also want the corresponding inverse? That would be more unsafe than Base, since you could use wrong arithmetic to do arbitrary unchecked damage.

I think the preferred approach is don't use unsafe.

@mdempsky

This comment has been minimized.

Member

mdempsky commented Nov 30, 2015

@dr2chase By corresponding inverse, do you mean the Add function I mentioned? If so, in my unsafe.Pointer arithmetic proposal, I further suggested that Add(p, n) should only be guaranteed to be safe if the resulting pointer points into the same variable as p, which is checkable at runtime.

@randall77

This comment has been minimized.

Contributor

randall77 commented Nov 30, 2015

I'm not a huge fan of Base because it implies that the runtime knows the answer and can compute it in a timely fashion. I think that's mostly true today but might not be in some possible GC implementations.

For instance, if p is a pointer into a global variable, we don't have an efficient way right now to determine, given p, the starting point of the containing global variable.

It also doesn't work on non-Go-managed memory, which will be confusing. If a function gets two slices and uses Base to determine aliasing, why should it work for slices whose backing store is allocated by Go and not for backing stores allocated by mmap?

@mdempsky

This comment has been minimized.

Member

mdempsky commented Nov 30, 2015

@randall77 Fair points!

So perhaps tying the wording to "Go variables" is wrong. Instead something like "indivisible block of memory" or whatever appropriately describes the granularity at which the GC may rearrange memory.

Also, for pointers to objects that will never be moved by the GC (e.g., global Go variables, or unmanaged memory), Base(p) would return (nil, uintptr(p)) (instead of (p, 0)).

That would allow offset (as written above) to work for slices whose backing store is allocated by mmap, because their base pointers would be equal (both nil).

Downside is it would also work for slices of even separate mmap allocations (or of an mmap allocation and a Go global variable), but at least since those objects aren't going to move around in memory anyway, I don't think it could cause problems? (Though I'll continue thinking about that.) Also, I don't see how Go could be expected to reliably distinguish those cases anyway.

@randall77

This comment has been minimized.

Contributor

randall77 commented Nov 30, 2015

Even for Go-allocated objects, I'm not sure we want to commit to Base. Imagine the following scenario. We allocate objects from a region using a bump pointer and record size&type info in a side buffer. So object start info is encoded in the side buffer, but not in any easily seekable way. Presumably GC would need to find object starts, but maybe it only does so in bulk - i.e. a whole region at a time. To answer an individual Base query would require a scan of the side buffer.

I'm not saying a GC like that would ever work, but I'm concerned that by adopting Base we'd be preventing any such implementation.

I'm with Minux that if we're trying to determine overlap, ask the runtime directly. I think this handles the common case, effectively the stride==1 case. unsafe.Overlap(p unsafe.Pointer, plen uintptr, q unsafe.Pointer, qlen uintptr) bool? The nice thing about this function is that the API is GC-agnostic (the implementation would have to understand the GC, of course).

Maybe we handle stride>1 queries as the stride==1 query above together with the guarantee that uintptr(p)-uintptr(q) is always correct if p and q are in the same allocation? I'm less sure about whether this makes sense and/or is possible given a moving GC.

@kortschak

This comment has been minimized.

Contributor

kortschak commented Nov 30, 2015

I'm with Minux that if we're trying to determine overlap, ask the runtime directly. I think this handles the common case, effectively the stride==1 case. unsafe.Overlap(p unsafe.Pointer, plen uintptr, q unsafe.Pointer, qlen uintptr) bool? The nice thing about this function is that the API is GC-agnostic (the implementation would have to understand the GC, of course).

This does not handle the block matrix use that we have.

Maybe we handle stride>1 queries as the stride==1 query above together with the guarantee that uintptr(p)-uintptr(q) is always correct if p and q are in the same allocation? I'm less sure about whether this makes sense and/or is possible given a moving GC.

Nor does this AFAICS.

@randall77

This comment has been minimized.

Contributor

randall77 commented Nov 30, 2015

@kortschak, could you be more specific about what you're trying to do then? I'm afraid I don't understand what you mean in the second half of your description:

We want to return whether there is an i,j and x,y such that a[i + j_stride] and b[x + y_stride] address the
same variable when the indices are constrained such that they index a subset of the possible blocks
that may be described with the given stride over the slices.

@kortschak

This comment has been minimized.

Contributor

kortschak commented Nov 30, 2015

@randall77, probably the best thing to do is look at the docs linked above and the implementation of the checkOverlap methods in the code linked above. However, in brief we need to be able to detect overlap in views of a 2-d matrix. An overlap is when two matrices share an element in the backing store - this is more complex that just a trivial overlap as described in minux's first proposal and also more complex than a stride>1 overlap, since a description of a matrix is struct { data []float64; rows, cols, stride int }, not just struct { data []float64; stride int }. To assess the overlap of two blocks when the stride-only bool-returning function, you would require (AFAICS) O(rows) operations for a row major storage.

@randall77

This comment has been minimized.

Contributor

randall77 commented Nov 30, 2015

I think you could do it with:

func checkOverlap(a, b matrix) bool {
    if !unsafe.Overlap(&a.data[0], a.rows*a.stride, &b.data[0], b.rows*b.stride) { return false }
    off := uintptr(&a.data[0])-uintptr(&b.data[0])
    ...do the same thing that your checkOverlap does, starting at the stride comparison...
}
@kortschak

This comment has been minimized.

Contributor

kortschak commented Nov 30, 2015

I don't see how that helps. The issue is that the test for a block overlap requires that arithmetic be performed on an offset, specifically the test in rectanglesOverlap. You are right that it simplifies these two conditions, but that is a minor component of the test.

@randall77

This comment has been minimized.

Contributor

randall77 commented Dec 1, 2015

But my code is computing an offset you can use. Isn't it exactly what your offset call returns? (Modulo size of the base type.) Just pass off/unsafe.Sizeof(float64(0)) to rectanglesOverlap.

@kortschak

This comment has been minimized.

Contributor

kortschak commented Dec 1, 2015

This uintptr(&a.data[0])-uintptr(&b.data[0])? We can't do that now and it seems to me that the legal equivalent (using the unsafe.Pointer conversion) is what I have currently (with the exception that I perform a sign check - I tried removing this condition and our tests fail - this was surprising to me and I would like to figure out why, but that's a separate issue).

@randall77

This comment has been minimized.

Contributor

randall77 commented Dec 1, 2015

Yes, that. After the overlap check, a and b are guaranteed to be in the same allocation. So we may be able to make that subtraction legal.
Yes, you'd need the sign check. You probably want
intptr(uintptr(&a.data[0]))-intptr(uintptr(&b.data[0]))
So there isn't an extra factor of 2^64 flying around.

@kortschak

This comment has been minimized.

Contributor

kortschak commented Dec 1, 2015

Ah, OK.

Presumably you mean uintptr(unsafe.Pointer(&a.data[0]))-unsafe.Pointer(uintptr(&b.data[0])) after the appropriate sign check, and intptr is an int that is the same size as uintptr - this must depend on build tags since there is no such built-in type. I did the int conversion after the subtraction since we might have an address where addr&(1<<63) != 0.

@kortschak kortschak closed this Dec 1, 2015

@kortschak kortschak reopened this Dec 1, 2015

@kortschak

This comment has been minimized.

Contributor

kortschak commented Dec 1, 2015

Click slip.

@kortschak

This comment has been minimized.

Contributor

kortschak commented Dec 1, 2015

So this works (I think my previous foray without the sign check did the int conversion too late):

func offset(a, b []float64) int {
    if &a[0] == &b[0] {
        return 0
    }
    return int(uintptr(unsafe.Pointer(&b[0]))-uintptr(unsafe.Pointer(&a[0]))) / int(unsafe.Sizeof(float64(0)))
}

If the guarantee can be made for overlapping allocations then this is good. The other half of the issue is how the reflect equivalent can be ensured to work since a move may happen between the uinptr conversion in reflect and the subtraction on return from the call to reflect's UnsafeAddr.

@rsc rsc modified the milestones: Unreleased, Go1.6 Dec 5, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment