The capacity of the result byte (or rune) slice of a conversion from a string is undefined

Go101 edited this page May 20, 2018 · 4 revisions

The capacity of the result slice of the conversion []byte(aString) is guaranteed to be not less than len(aString). However, how large the capacity is is undefined.

What do you expect the following Go program to print?

package main

import "fmt"

func main() {
	h := []byte("Hello")
	hc := append(h, " C"...)
	_ = append(h, " Go"...)
	fmt.Printf("%d, %d, %s\n", cap(h), len(h), hc)
}

If you compile this program with gccgo, then the result is

5, 5, Hello C

which is the expected result of many Go gophers. However, if you compile this program with the standard Go compiler (gc) v1.10.2, the result becomes

8, 5, Hello G

What! What happens here? The reason is gccgo and gc use different memory allocation policies when allocating the memory block for hosting the elements of the result slice of []byte("Hello"). Both implementations don't violate Go specification and other official Go documentations. In other words, the capacity of the result slice of []byte("Hello") is compiler dependent.

To make the above program print consistent outputs, we should modify it as

package main

import "fmt"

func main() {
	h := []byte("Hello")
	h = h[:len(h):len(h)] // need this line
	hc := append(h, " C"...)
	_ = append(h, " Go"...)
	fmt.Printf("%d, %d, %s\n", cap(h), len(h), hc)
}

An interesting fact is that, gc may produce two byte slices with different capacities even for two identical conversions. For example,

package main

import "fmt"

func main() {
	h1 := []byte("Hello")
	h2 := []byte("Hello")
	fmt.Println(cap(h1), len(h1)) // 32 5
	fmt.Println(cap(h2), len(h2)) // 8 5
	fmt.Println(h2)
}

The reason why the two capacities of the result slices the two conversions are different is that the last fmt.Println call makes h2 escape to heap, whereas h1 is still allocated on stack. For gc, the allocation policies are different for heap and stack memory blocks.

The above described facts also apply to []rune(aString) conversions.

[Update]: (mentioned by badu), it looks gc adopts the third policy if the result slice is stored in a package-level variable. The print result of the following program is the same as gccgo: 5, 5, Hello C.

package main

import "fmt"

var h = []byte("Hello")

func main() {
	hc := append(h, " C"...)
	_ = append(h, " Go"...)
	fmt.Printf("%d, %d, %s\n", cap(h), len(h), hc)
}
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.