Support Weird Number Encodings

[0xcaff]

I've found numbers in network protocols which are 3 or 5 bytes long and decoding them is hard.

[lunixbochs]

Can you link to a protocol document or RFC describing these?

On Feb 5, 2016, at 9:57 PM, Martin Charles notifications@github.com wrote:

I've found numbers in network protocols which are 3 or 5 bytes long and decoding them is hard.

—
Reply to this email directly or view it on GitHub.

[0xcaff]

I'm trying to develop a client for Boom Beach. Information about the protocol used for this game and other games made by supercell can be found here. The length field is 3 bytes long (big endian). I've currently been decoding it by appending an empty byte to the front then converting it to an integer. I would like to use your library to handle all the encoding/decoding code but I can't because of this.

Thanks!

[lunixbochs]

I think the solution here is to support unknown types via custom Pack(), Unpack(), and Sizeof() methods, as per issue #4.

Maybe with signatures like this:

func (t *Type) Pack(buf []byte, opt *struc.Options) error
func (t *Type) Unpack(r io.Reader, opt *struc.Options) error
func (t *Type) Sizeof(opt *struc.Options) int

Then you'd just define something like:

(EDIT: if you're from the future, this type was not the final API. Use Int3 here)

type Int3 int32
func (i *Int3) Pack(buf []byte, opt *struc.Options) error {
    var tmp [4]byte
    binary.BigEndian.PutUint32(tmp[:], *i)
    copy(buf, tmp[:3])
    return nil
}
func (i *Int3) Unpack(r io.Reader, opt *struc.Options) error {
    var tmp [4]byte
    if _, err := r.Read(tmp[3]); err != nil {
        return err
    }
    *i = binary.BigEndian.Uint32(tmp[:])
    return nil
}
func (i *Int3) Sizeof(opt *struc.Options) int {
    return 3
}

Pack uses a []byte because I call Sizeof() first to stack-allocate memory where possible.

struc.Options currently provides the machine's pointer size and Endianness.

Honestly I could also use this on some common types to get nice performance improvements by reducing reflection.

[0xcaff]

This sounds like a good way to make the library more usable and maintainable. I'm looking forward to the implementation.

[lunixbochs]

Try 18d5282 with sample code for Int3 at the top of https://github.com/lunixbochs/struc/blob/master/custom_test.go

[0xcaff]

Thanks, but I can't seem to use the Int3 type with a sizeof. For example, the following does not work, Data is of length zero and empty.

type Message struct {
    Length Int3 `struc:"sizeof=Data"`
    Data []byte `struc:"[]byte"`
}

[lunixbochs]

I added a unit test for Int3 being used as a sizeof field and fixed a possible crash case with custom types as sizes. Should work now.