New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encoding/json: Marshaler/Unmarshaler not stream friendly #12001

Open
lukescott opened this Issue Aug 3, 2015 · 2 comments

Comments

Projects
None yet
4 participants
@lukescott

lukescott commented Aug 3, 2015

The Marshaler/Unmarshaler interface deals with whole []byte slices:

type Marshaler interface {
        MarshalJSON() ([]byte, error)
}
type Unmarshaler interface {
        UnmarshalJSON([]byte) error
}

If you're dealing with a type that encodes to an array that has a large number of objects inside you have to encode all of into a single []byte.

The encoding/xml package is, unlike encoding/json, stream friendly:

type Marshaler interface {
        MarshalXML(e *Encoder, start StartElement) error
}
type Unmarshaler interface {
        UnmarshalXML(d *Decoder, start StartElement) error
}

With MarshalXML you can call e.Encode/e.EncodeElement/e.EncodeToken.

Since encoding/json is gaining Token()/EncodeToken() methods it would be really helpful to have a Marshaler/Unmarshaler interface that can take advantage of that. Perhaps something along the lines of:

type Encoder interface {
        EncodeJSON(e *Encoder) error
}
type Decoder interface {
        DecodeJSON(d *Decoder) error
}

Since Marshaler/Unmarshaler can't be changed. Or something like:

type Marshaler2 interface {
        MarshalJSON(e *Encoder) error
}
type Unmarshaler2 interface {
        UnmarshalJSON(d *Decoder) error
}

@ianlancetaylor ianlancetaylor added this to the Unplanned milestone Aug 3, 2015

@mildred

This comment has been minimized.

mildred commented Oct 30, 2015

This would be really nice to have. I looked a bit into the code and found that it won't be easy to use the Unmarshaler2 interface while just using the json.Unmarshal(...) function, because we don't have a Decoder object.

However, it's more easy to have this implemented when using Decoder.Unmarshal(...)

@JonathanFraser

This comment has been minimized.

JonathanFraser commented Feb 1, 2017

I think it's worth considering looking at the way the yaml package handles custom unmarshallers.

type Unmarshaller interface {
    UnmarshalYAML(unmarshal func(interface{}) error) error
}

type Marshaller interface {
    MarshalYAML() (interface{}, error)
}

While I would normally dislike empty interfaces when it comes to marshalling its kind of unavoidable. What I like about this is it allows the object so simply inject the marshallable type it wishes to use under the hood without exposing a raw byte string. This has a nice side effect of decoupling the encoding from the object. For example, rather than this be a MarshalYAML and UnmarshalYAML interface this could just be Marshal and Unmarshal. This is nice because it breaks dependancies, moreover, the user just needs to specify the underlying marshalling type and it doesn't really care if it's decoded as a stream or a buffer. It's completely agnostic to that.

@vrde vrde referenced this issue Mar 13, 2018

Closed

further devel #1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment