cgo creating char** from CString corrupts initial entries

[jimmyfrasche]

$ uname -srvm
Linux 4.3.0-1-amd64 #1 SMP Debian 4.3.3-5 (2016-01-04) x86_64
$ go version
go version go1.6rc1 linux/amd64

(also tested on 1.5 before upgrading)

I'm trying to create a char** in Go to pass to a C function but the first entries are mangled when I read them back.

The smallest reproducible case I could manage is:

package main

import (
    "fmt"
    "unsafe"
)

//#include <stdlib.h>
import "C"

var xs = []string{
    "1",
    "",
    "squirrel",
    "2",
    "two",
}

func main() {
    fmt.Println("Test conversion")
    fmt.Printf("\tN\t=\t%20s\t%20s\n", "Go", "C")
    //if I delete this loop everything works as intended
    for i, x := range xs {
        cvt := C.GoString(C.CString(x))
        //always equal
        fmt.Printf("\t%d\t%t\t%20q\t%20q\n", i, x == cvt, x, cvt)
    }

    //create an array meant to be passed to C
    //and a view into it to update the data
    arr := (**C.char)(C.malloc(C.size_t(C.int(len(xs)))))
    view := (*[1 << 30]*C.char)(unsafe.Pointer(arr))[0:len(xs):len(xs)]

    fmt.Println("To C", &view[0], arr)
    fmt.Printf("\tN\t=\t%20s\t%20s\t&C\n", "Go", "C")
    for i, x := range xs {
        view[i] = C.CString(x)
        // (deleting the rest of this loop has no effect on the issue, but it's interesting that it still works at this point)
        g := C.GoString(view[i])
        //always equal
        fmt.Printf("\t%d\t%t\t%20q\t%20q\t%v\n", i, x == g, x, g, view[i])
    }

    //Everything is fine so far, but reading the values back out
    //(whether in Go or C) does not work
    fmt.Println("From C", &view[0], arr)
    fmt.Printf("\tN\t=\t%20s\t%20s\t&C\n", "Go", "C")
    for i, x := range view {
        g := C.GoString(x)
        //only the same after the first few entries
        fmt.Printf("\t%d\t%t\t%20q\t%20q\t%v\n", i, xs[i] == g, xs[i], g, x)
    }

}

When I run this I get something like

Test conversion
    N   =                     Go                       C
    0   true                     "1"                     "1"
    1   true                      ""                      ""
    2   true              "squirrel"              "squirrel"
    3   true                     "2"                     "2"
    4   true                   "two"                   "two"
To C 0x111b480 0x111b480
    N   =                     Go                       C    &C
    0   true                     "1"                     "1"    0x111b4a0
    1   true                      ""                      ""    0x111b4c0
    2   true              "squirrel"              "squirrel"    0x111b4e0
    3   true                     "2"                     "2"    0x111b500
    4   true                   "two"                   "two"    0x111b520
From C 0x111b480 0x111b480
    N   =                     Go                       C    &C
    0   false                    "1"         " \xb5\x11\x01"    0x111b4a0
    1   true                      ""                      ""    0x111b4c0
    2   true              "squirrel"              "squirrel"    0x111b4e0
    3   true                     "2"                     "2"    0x111b500
    4   true                   "two"                   "two"    0x111b520

The first entry get mangled. (Different every time, but mangled every time).

The number of mangled entries is proportional to the number of entries in xs, but the mangled entries are always at the start of the list. With less than 5 entries, there is no mangling. With 9, the first two are mangled. With 13, the first three are mangled.

As the code notes, if I delete the first loop I always get the expected, irrespective of the length of xs.
(I had no equivalent in the program I abstracted this from, but it's required to reproduce here).

To C 0x1fa8050 0x1fa8050
    N   =                     Go                       C    &C
    0   true                     "1"                     "1"    0x1fa8400
    1   true                      ""                      ""    0x1fa8420
    2   true              "squirrel"              "squirrel"    0x1fa8440
    3   true                     "2"                     "2"    0x1fa8460
    4   true                   "two"                   "two"    0x1fa8480
From C 0x1fa8050 0x1fa8050
    N   =                     Go                       C    &C
    0   true                     "1"                     "1"    0x1fa8400
    1   true                      ""                      ""    0x1fa8420
    2   true              "squirrel"              "squirrel"    0x1fa8440
    3   true                     "2"                     "2"    0x1fa8460
    4   true                   "two"                   "two"    0x1fa8480

If I'm just doing something wrong with cgo, I apologize for the noise.

[minux]

arr := (**C.char)(C.malloc(C.size_t(C.int(len(xs)))))

you only allocates len(xs) bytes of storage, and then
you treat it as len(xs) char* pointers. How could that
work?

[jimmyfrasche]

Gah, thanks.

For any future searchers, the fix is changing that line to

size := C.size_t(unsafe.Sizeof((*C.char)(nil)))
length := C.size_t(len(xs))
arr := (**C.char)(C.malloc(length * size))