Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
GitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
encoding/json: decoding into existing map pointer values unexpectedly reallocates them #31924
When decoding JSON data into an existing map object, an existing key's value is reallocated. I can't determine conclusively whether or not this is expected behavior, but the doc seems to be
The example below should explain this in a better way:
It is unexpected that B and A (as variables) lose coherence after the call to json.Unmarshal. The call reallocates "A.B", creating a copy of it. After that call, the objects are independent, which may be unexpected behavior.
What did you expect to see?
What did you see instead?
Possibly relevant godoc from encoding/json
Specifically, this part:
To me implies that it does not allocate a new value and set the pointer to point to it, but instead uses the existing value pointed at.
If a map is already present, we expect it to populate provided keys but not remove existing keys that weren't overwritten.
If a struct is already present, we sometimes expect it to populate provided keys but not zero out existing keys.
But if a map contains a struct (or pointer to a struct), it appears that the outcome is to replace the key entirely, rather than to populate recursively. Contrast with what you might expect for
And it sort of makes sense to me that, since map values aren't addressable, a map[string]struct would just replace the struct with a new struct, because trying to behave otherwise is actually pretty hard. But when it's a map[string]*struct, it's not insane to think it should act like populating a struct would normally.
Seems to be the samme with map[string]map[string]int, etc., which is to say, the recursion of decoding is not the same as you'd get by recursing yourself. If you have a map key-value pair
Here is what I think is a simpler example: https://play.golang.org/p/gnyc9t1kleB
It shows that a map value is replaced, and thus the original value is left untouched, and the pointers differ. Edit: it's replaced with a zero value, so all other fields are lost.
When a struct is used, the same pointer is reused, so the original value changes.
I tend to agree that both cases should behave the same here, but it's hard to tell if that's a good idea before digging into the code.
I've convinced myself that the current behavior is simply by accident. I've worked on a fix, and no existing json tests fail. I also like the new logic much better, as it's more consistent with structs and the general package behavior.
I've also made non-pointer elements keep their existing values too, by making a copy of the value and assigning it back to its key.
I'm not sure if there's much else to do here for the current
I would definitely add this issue to the bag of design issues to consider in a future json API redesign, though. How should we keep track of those? I've seen some others in the form of proposals in "held" status, but I don't think this issue should be a proposal. It's just a design inconsistency bug.
I don't think adding options for historical design tech debt is a good idea. The problem with that kind of debt is that the API is less consistent and a bit harder to use, but one can usually still write code around the issue. If we add an option, to fix the debt and avoid breaking backwards compatibility, the package is still not consistent (by default) and, arguably, it's even harder to use for someone new to Go.