Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encoding/gob: decoding fails for structs with anonymous pointer fields that implement GobDecoder interface #31781

Open
skuppa opened this issue May 1, 2019 · 4 comments

Comments

@skuppa
Copy link

commented May 1, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12.1 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/kuppas/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/kuppas/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.12.1/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.12.1/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/1f/s_1g3pg54b123lds0c08xfmw0000gn/T/go-build807164800=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I am using the gob encoder/decoder to serialize and deserialize the strut into a byte array. I ran into an issue when big.Rat embedded into another struct. I am able to successfully encode and decodebig.Rat type directly without any issue. But big.Rat is embedded into another strut then I am able to encode successfully but during the decoding process, it throws panic: runtime error: invalid memory address or nil pointer dereference.

BTW, to solve this issue, I added implemented the GobEncoder/GobDecoder for the custom strut.

Link to go playground https://play.golang.org/p/98PzJDqm0QL

The below code produces the issue and we can uncomment GobEncoder/GobDecoder to successfully run the example.

Code sample:

package main

import (
	"bytes"
	"encoding/gob"
	"fmt"
	"math/big"
)

func main() {
	fmt.Println("Testing Rat")

	rat1 := big.NewRat(10, 5)

	// The Rat works gob encode and decode
	rat1Bytes, err := Serialize(rat1)
	if err != nil {
		fmt.Printf("Error on serializing rat1: %v", err)
	}

	rat2 := &big.Rat{}
	err = Deserialize(rat1Bytes, rat2)
	if err != nil {
		fmt.Printf("Error on deserializing rat1: %v", err)
	}

	if rat2.String() == rat1.String() {
		fmt.Println("rat1 == rat2")
	}

	fmt.Println("Testing CustomRat")

	// CustomRat encode works and decode fails due nil object
	// Uncomment GobEncode and GobDecode to make encode and decode works
	customRat1 := &CustomRat{big.NewRat(10, 5)}

	customRat1Bytes, err := Serialize(customRat1)
	if err != nil {
		fmt.Printf("Error on serializing rat1: %v", err)
	}

	customRat2 := &CustomRat{}
	err = Deserialize(customRat1Bytes, customRat2)
	if err != nil {
		fmt.Printf("Error on deserializing rat1: %v", err)
	}

	if rat2.String() == rat1.String() {
		fmt.Println("customRat1 == customRat2")
	}
}

func Serialize(val interface{}) ([]byte, error) {
	b := new(bytes.Buffer)
	if err := gob.NewEncoder(b).Encode(val); err != nil {
		return nil, err
	}
	return b.Bytes(), nil
}

func Deserialize(data []byte, result interface{}) error {
	return gob.NewDecoder(bytes.NewBuffer(data)).Decode(result)
}

type CustomRat struct {
	*big.Rat
}

// Uncomment the below GobDecode and GobEncode for CustomRat to work.
//func (cr *CustomRat) GobDecode(data []byte) error {
//	cr.Rat = &big.Rat{}
//	return cr.Rat.GobDecode(data)
//}
//
//func (cr *CustomRat) GobEncode() ([]byte, error) {
//	return cr.Rat.GobEncode()
//}

What did you expect to see?

I expect to see successful encoding/decoding when big.Rat whether it is embedded into another strut or not.

What did you see instead?

The big.Rat decoding fails when it is embedded into another strut. Please see the example code above.

@bmkessler

This comment has been minimized.

Copy link
Contributor

commented May 2, 2019

I am not sure the correct behavior here. How is the custom GobDecode implementation on big.Int, big.Rat, big.Float supposed to handle decoding into a nil pointer? The documentation for the GobDecoder interface doesn't specify anything about requiring the pointer to be non-nil, so it seems like implementations should handle that case by allocating a new instance.

For the code above, you can avoid the custom GobEncode/Decode workaround for now by initializing the embedded pointer:

customRat2 := &CustomRat{&big.Rat{}}

@katiehockman katiehockman changed the title Gob encoding/decoding fails when `big.Rat` embedded into another type encoding/gob: decoding fails when `big.Rat` embedded into another type May 2, 2019

@katiehockman

This comment has been minimized.

Copy link
Contributor

commented May 2, 2019

/cc @robpike

@smw1218

This comment has been minimized.

Copy link

commented May 2, 2019

@bmkessler the example was simplified for clarity (I work with @skuppa). In our actual use case, the big.Rat was nested several levels deep inside a more complicated struct necessitating the override of GobDecode.
Given that all the other structs were initialized until we got to the big.Rat, it certainly seems like it should allocate it. Additionally, the big.Rat.GobDecode does initialize the zero value if the length of the buffer is zero: https://golang.org/src/math/big/ratmarsh.go?s=1058:1099#L32

@bmkessler

This comment has been minimized.

Copy link
Contributor

commented May 3, 2019

Yeah, this looks to be a general issue with the gob decode handling anonymous pointer fields that implement the GobDecoder interface. I made a simple reproducer for a custom type that properly initializes the struct if either the pointer field is not anonymous or if the struct doesn't implement the GobDecoder interface.

https://play.golang.org/p/TxD3mjSpI-2

I think the issue title should be updated to something like encoding/gob: decoding fails for structs with anonymous pointer fields that implement GobDecoder interface as the issue is not specific to big.Rat

Note, https://golang.org/src/encoding/gob/decode.go L1119 contains a TODO about anonymous names.
// TODO(r): anonymous names

@katiehockman katiehockman changed the title encoding/gob: decoding fails when `big.Rat` embedded into another type encoding/gob: decoding fails for structs with anonymous pointer fields that implement GobDecoder interface May 3, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.