Add binary marshal

[klauspost]

Fixes #409

(copied)

Allow specifying directives for (external) types that support BinaryMarshaler + BinaryUnmarshaler and optionally BinaryAppender.

• //msgp:binmarshal pkg.Type pkg.Type2 will use BinaryMarshaler/BinaryUnmarshaler.
• //msgp:binappend pkg.Type pkg.Type2 will use BinaryAppender/BinaryUnmarshaler.

Serialized data will be added an bin data, no special header.

I guess we could also add the text interfaces. But the user would probably need to specify the storage type...

• //msgp:textmarshal pkg.Type pkg.Type2 will use TextMarshaler/TextUnmarshaler; data saved as bin (default)
• //msgp:textappend pkg.Type pkg.Type2 will use TextAppender/TextUnmarshaler; data saved as bin (default)
• //msgp:textmarshal as:string pkg.Type pkg.Type2 will use TextMarshaler/TextUnmarshaler; data saved as string.
• //msgp:textappend as:string pkg.Type pkg.Type2 will use TextAppender/TextUnmarshaler; data saved as string.

Example

// BinaryTestType implements encoding.BinaryMarshaler and encoding.BinaryUnmarshaler
type BinaryTestType struct {
	Value string
}

//msgp:binmarshal BinaryTestType

func (t *BinaryTestType) MarshalBinary() ([]byte, error) {
	return []byte(t.Value), nil
}

func (t *BinaryTestType) UnmarshalBinary(data []byte) error {
	t.Value = string(data)
	return nil
}

Implementation notes

Most of these will require a temporary buffer. Exceptions:

• MarshalMsg will encode to the destination when using Append.
• UnmarshalMsg will use a zerocopy to read.

Sizes are not accurate. We assume header + 32 bytes.

[philhofer]

I'm not sure it matters a whole lot, but you could write a (*msgp.Writer).AppendBytes(encoding.BinaryAppender) method that uses the internal write buffer to elide the heap allocation in the same way you're doing it for the marshal path.

[klauspost]

Good idea. This would be the added functions:

// Temporary buffer for reading/writing binary data.
var bytesPool = sync.Pool{New: func() any { return make([]byte, 0, 1024) }}

// WriteBinaryAppender will write the bytes from the given
// encoding.BinaryAppender as a bin array.
func (mw *Writer) WriteBinaryAppender(b encoding.BinaryAppender) error {
	dst := bytesPool.Get().([]byte)
	defer bytesPool.Put(dst)
	dst, err := b.AppendBinary(dst[:0])
	if err != nil {
		return err
	}
	return mw.WriteBytes(dst)
}

func (mw *Writer) WriteTextAppender(b encoding.TextAppender) error
func (mw *Writer) WriteTextAppenderString(b encoding.TextAppender) error

// ReadBinaryUnmarshal reads a binary-encoded object from the reader and unmarshals it into dst.
func (m *Reader) ReadBinaryUnmarshal(dst encoding.BinaryUnmarshaler) error
func (m *Reader) ReadTextUnmarshal(dst encoding.TextUnmarshaler) err
func (m *Reader) ReadTextUnmarshalString(dst encoding.TextUnmarshaler) error

A complication is that we don't know if the Append interface is implemented on a pointer or a value... But it can be done. Ironically I am having the biggest issue implementing it on the Reader - even though that is always a pointer. I will ping when I have an update.

[klauspost]

Cleaned up the codegen and implemented "copy-less" bin8/str8 for Append MarshalMsg. Basic codegen now looks like this:

// DecodeMsg implements msgp.Decodable
func (z *TextAppenderBinValue) DecodeMsg(dc *msgp.Reader) (err error) {
	err = dc.ReadTextUnmarshal(z)
	if err != nil {
		err = msgp.WrapError(err)
		return
	}
	return
}

// EncodeMsg implements msgp.Encodable
func (z *TextAppenderBinValue) EncodeMsg(en *msgp.Writer) (err error) {
	err = en.WriteTextAppender(z)
	if err != nil {
		err = msgp.WrapError(err)
		return
	}
	return
}

// MarshalMsg implements msgp.Marshaler
func (z *TextAppenderBinValue) MarshalMsg(b []byte) (o []byte, err error) {
	o = msgp.Require(b, z.Msgsize())
	o = append(o, 0, 0)
	zb0001 := len(o)
	o, err = z.AppendText(o)
	if err != nil {
		err = msgp.WrapError(err)
		return
	}
	zb0001 = len(o) - zb0001
	o = msgp.AppendBytesTwoPrefixed(o, zb0001)
	return
}

// UnmarshalMsg implements msgp.Unmarshaler
func (z *TextAppenderBinValue) UnmarshalMsg(bts []byte) (o []byte, err error) {
	var zb0001 []byte
	zb0001, bts, err = msgp.ReadBytesZC(bts)
	if err != nil {
		err = msgp.WrapError(err)
		return
	}
	err = z.UnmarshalText(zb0001)
	if err != nil {
		err = msgp.WrapError(err)
		return
	}
	o = bts
	return
}

// Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message
func (z *TextAppenderBinValue) Msgsize() (s int) {
	s = msgp.TextAppenderBinSize
	return
}