encoding/json: marshal of custom type not quoted when using ",string" tag field

[aronatkins]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.8.3 linux/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/var/lib/jenkins/go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build418778517=/tmp/go-build"
CXX="g++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

What did you do?

We receive JSON which encodes booleans as numeric strings. Parsing that JSON works just fine with a ",string" tag and a type with its own UnmarshalJSON definition. The quoting is removed and we get the proper boolean value.

Quoting is not added when serializing this type, which means that JSON IN>decode>encode>OUT produces OUT different than IN.

type parsedBool bool

func (basi *parsedBool) UnmarshalJSON(data []byte) error {
	v, err := strconv.ParseBool(string(data))
	if err != nil {
		return err
	}
	*basi = parsedBool(v)
	return nil
}

func (basi parsedBool) MarshalJSON() ([]byte, error) {
	if basi {
		return []byte(`1`), nil
	} else {
		return []byte(`0`), nil
	}
}

https://play.golang.org/p/ccIw4E6UJF

What did you expect to see?

Equivalent support for ",string" for this custom type when both encoding and decoding. I was surprised by the difference in behavior. Both cases should either err (because parsedBool is not bool) or properly handle the quotes.

The json docs state:

The "string" option signals that a field is stored as JSON inside a JSON-encoded string. It applies only to fields of string, floating point, integer, or boolean types.

What did you see instead?

Given:

	wrapped := struct {
		P parsedBool `json:",string"`
		B bool       `json:",string"`
	}{
		P: true,
		B: true,
	}

This is marshaled as:

{"P":1,"B":"true"}

This was unexpected because we properly parse:

{"P":"1","B":"true"}

[OneOfOne]

You are using a custom marshal function, not sure why is that unexpected?

[mvdan]

Perhaps the package docs should be improved to clarify that ,string gets ignored when using a Marshaler/Unmarshaler.

[aronatkins]

@OneOfOne + @mvdan The primary concern is the inconsistency between reading/writing JSON. Note that the quoting is removed when parsing JSON but not added when creating JSON.

The inconsistency can be addressed either by supporting quoting when serializing ",string"-annotated custom types (which are still primitives under the hood) OR by removing support for quote-removal when deserializing ",string"-annotated custom types.

Both options have backwards-compatibility issues, as programs are likely to have already worked around the observed behavior.

[andybons]

Marking for Go2 due to backwards-incompatible changes that would need to be made.

[toby1991]

@OneOfOne + @mvdan The primary concern is the inconsistency between reading/writing JSON. Note that the quoting is removed when parsing JSON but not added when creating JSON.

The inconsistency can be addressed either by supporting quoting when serializing ",string"-annotated custom types (which are still primitives under the hood) OR by removing support for quote-removal when deserializing ",string"-annotated custom types.

Both options have backwards-compatibility issues, as programs are likely to have already worked around the observed behavior.

@aronatkins I've found the solution...

type BigInt struct {
	_bi big.Int
}
func (bi BigInt) String() string {
	return bi._bi.String()
}

// before
//func (bi BigInt) MarshalJSON() ([]byte, error) {
//	return []byte(bi._bi.String()), nil
//}

// now
func (bi BigInt) MarshalJSON() ([]byte, error) {
	return []byte(`"` + bi._bi.String() + `"`), nil
}

You jus need to add the " or ' to quote the string data.. I think that will make the json package consider that the data is a string type

I fixed this issue by using the tricks above. @totoval
https://github.com/totoval/framework/blob/7b3016a86bb1e41c56d90668d63ddd929e3f34a7/model/types/bigint/bigint.go#L67

For your example,
https://play.golang.org/p/7N0c1Q4_UOF

[ianlancetaylor]

CC @dsnet @mvdan Does the encoding/json/v2 proposal address this issue? Thanks.