encoding/gob: decoding fails for structs with anonymous pointer fields that implement GobDecoder interface

[skuppa]

What version of Go are you using (go version)?

$ go version
go version go1.12.1 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/kuppas/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/kuppas/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.12.1/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.12.1/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/1f/s_1g3pg54b123lds0c08xfmw0000gn/T/go-build807164800=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I am using the gob encoder/decoder to serialize and deserialize the strut into a byte array. I ran into an issue when big.Rat embedded into another struct. I am able to successfully encode and decodebig.Rat type directly without any issue. But big.Rat is embedded into another strut then I am able to encode successfully but during the decoding process, it throws panic: runtime error: invalid memory address or nil pointer dereference.

BTW, to solve this issue, I added implemented the GobEncoder/GobDecoder for the custom strut.

Link to go playground https://play.golang.org/p/98PzJDqm0QL

The below code produces the issue and we can uncomment GobEncoder/GobDecoder to successfully run the example.

Code sample:

package main

import (
	"bytes"
	"encoding/gob"
	"fmt"
	"math/big"
)

func main() {
	fmt.Println("Testing Rat")

	rat1 := big.NewRat(10, 5)

	// The Rat works gob encode and decode
	rat1Bytes, err := Serialize(rat1)
	if err != nil {
		fmt.Printf("Error on serializing rat1: %v", err)
	}

	rat2 := &big.Rat{}
	err = Deserialize(rat1Bytes, rat2)
	if err != nil {
		fmt.Printf("Error on deserializing rat1: %v", err)
	}

	if rat2.String() == rat1.String() {
		fmt.Println("rat1 == rat2")
	}

	fmt.Println("Testing CustomRat")

	// CustomRat encode works and decode fails due nil object
	// Uncomment GobEncode and GobDecode to make encode and decode works
	customRat1 := &CustomRat{big.NewRat(10, 5)}

	customRat1Bytes, err := Serialize(customRat1)
	if err != nil {
		fmt.Printf("Error on serializing rat1: %v", err)
	}

	customRat2 := &CustomRat{}
	err = Deserialize(customRat1Bytes, customRat2)
	if err != nil {
		fmt.Printf("Error on deserializing rat1: %v", err)
	}

	if rat2.String() == rat1.String() {
		fmt.Println("customRat1 == customRat2")
	}
}

func Serialize(val interface{}) ([]byte, error) {
	b := new(bytes.Buffer)
	if err := gob.NewEncoder(b).Encode(val); err != nil {
		return nil, err
	}
	return b.Bytes(), nil
}

func Deserialize(data []byte, result interface{}) error {
	return gob.NewDecoder(bytes.NewBuffer(data)).Decode(result)
}

type CustomRat struct {
	*big.Rat
}

// Uncomment the below GobDecode and GobEncode for CustomRat to work.
//func (cr *CustomRat) GobDecode(data []byte) error {
//	cr.Rat = &big.Rat{}
//	return cr.Rat.GobDecode(data)
//}
//
//func (cr *CustomRat) GobEncode() ([]byte, error) {
//	return cr.Rat.GobEncode()
//}

What did you expect to see?

I expect to see successful encoding/decoding when big.Rat whether it is embedded into another strut or not.

What did you see instead?

The big.Rat decoding fails when it is embedded into another strut. Please see the example code above.

[bmkessler]

I am not sure the correct behavior here. How is the custom GobDecode implementation on big.Int, big.Rat, big.Float supposed to handle decoding into a nil pointer? The documentation for the GobDecoder interface doesn't specify anything about requiring the pointer to be non-nil, so it seems like implementations should handle that case by allocating a new instance.

For the code above, you can avoid the custom GobEncode/Decode workaround for now by initializing the embedded pointer:

customRat2 := &CustomRat{&big.Rat{}}

[katiehockman]

/cc @robpike

[smw1218]

@bmkessler the example was simplified for clarity (I work with @skuppa). In our actual use case, the big.Rat was nested several levels deep inside a more complicated struct necessitating the override of GobDecode.
Given that all the other structs were initialized until we got to the big.Rat, it certainly seems like it should allocate it. Additionally, the big.Rat.GobDecode does initialize the zero value if the length of the buffer is zero: https://golang.org/src/math/big/ratmarsh.go?s=1058:1099#L32

[bmkessler]

Yeah, this looks to be a general issue with the gob decode handling anonymous pointer fields that implement the GobDecoder interface. I made a simple reproducer for a custom type that properly initializes the struct if either the pointer field is not anonymous or if the struct doesn't implement the GobDecoder interface.

https://play.golang.org/p/TxD3mjSpI-2

I think the issue title should be updated to something like encoding/gob: decoding fails for structs with anonymous pointer fields that implement GobDecoder interface as the issue is not specific to big.Rat

Note, https://golang.org/src/encoding/gob/decode.go L1119 contains a TODO about anonymous names.
// TODO(r): anonymous names