Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encoding/json: Marshaler/Unmarshaler not stream friendly #12001

Open
lukescott opened this issue Aug 3, 2015 · 3 comments
Open

encoding/json: Marshaler/Unmarshaler not stream friendly #12001

lukescott opened this issue Aug 3, 2015 · 3 comments
Milestone

Comments

@lukescott
Copy link

@lukescott lukescott commented Aug 3, 2015

The Marshaler/Unmarshaler interface deals with whole []byte slices:

type Marshaler interface {
        MarshalJSON() ([]byte, error)
}
type Unmarshaler interface {
        UnmarshalJSON([]byte) error
}

If you're dealing with a type that encodes to an array that has a large number of objects inside you have to encode all of into a single []byte.

The encoding/xml package is, unlike encoding/json, stream friendly:

type Marshaler interface {
        MarshalXML(e *Encoder, start StartElement) error
}
type Unmarshaler interface {
        UnmarshalXML(d *Decoder, start StartElement) error
}

With MarshalXML you can call e.Encode/e.EncodeElement/e.EncodeToken.

Since encoding/json is gaining Token()/EncodeToken() methods it would be really helpful to have a Marshaler/Unmarshaler interface that can take advantage of that. Perhaps something along the lines of:

type Encoder interface {
        EncodeJSON(e *Encoder) error
}
type Decoder interface {
        DecodeJSON(d *Decoder) error
}

Since Marshaler/Unmarshaler can't be changed. Or something like:

type Marshaler2 interface {
        MarshalJSON(e *Encoder) error
}
type Unmarshaler2 interface {
        UnmarshalJSON(d *Decoder) error
}
@ianlancetaylor ianlancetaylor added this to the Unplanned milestone Aug 3, 2015
@mildred
Copy link

@mildred mildred commented Oct 30, 2015

This would be really nice to have. I looked a bit into the code and found that it won't be easy to use the Unmarshaler2 interface while just using the json.Unmarshal(...) function, because we don't have a Decoder object.

However, it's more easy to have this implemented when using Decoder.Unmarshal(...)

@JonathanFraser
Copy link

@JonathanFraser JonathanFraser commented Feb 1, 2017

I think it's worth considering looking at the way the yaml package handles custom unmarshallers.

type Unmarshaller interface {
    UnmarshalYAML(unmarshal func(interface{}) error) error
}

type Marshaller interface {
    MarshalYAML() (interface{}, error)
}

While I would normally dislike empty interfaces when it comes to marshalling its kind of unavoidable. What I like about this is it allows the object so simply inject the marshallable type it wishes to use under the hood without exposing a raw byte string. This has a nice side effect of decoupling the encoding from the object. For example, rather than this be a MarshalYAML and UnmarshalYAML interface this could just be Marshal and Unmarshal. This is nice because it breaks dependancies, moreover, the user just needs to specify the underlying marshalling type and it doesn't really care if it's decoded as a stream or a buffer. It's completely agnostic to that.

@smyrman
Copy link

@smyrman smyrman commented Jun 3, 2020

The linked issue #14750 is not dependent on or directly related to this issue. It just raises another potential use-case for the described interface.

The use-case is relevant for the described Decoder interface (ignoring the naming conflict) in particular. By passing along a (sub) json.Decoder instead of []byte to a DecodeJSON method, you could also pass along options set on the original decoder, such as UseNumber, DissalowUnknownFields etc. If it's indeed a sub-decoder/copy, then it should be possible to override these properties without affecting the parrent. E.g.

const jsonExample = `{"a": "foo", "b": "bar", "c": 42}`

type T struct {
   A string
   B string
   C interface{}
}

func (t *T) DecodeJSON(dec json.Decoder) error {
   // Rely on parent decoder option for DissalowUnknownFields.
    dec.UseNumber() // Override UseNumber for sub-decoder only.
    return dec.Decode(t)
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.