Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encoding/json: Compact escapes U+2028 and U+2029 #30357

Open
PhilipBorgesen opened this Issue Feb 22, 2019 · 7 comments

Comments

Projects
None yet
4 participants
@PhilipBorgesen
Copy link
Contributor

PhilipBorgesen commented Feb 22, 2019

What version of Go are you using (go version)?

$ go version
go version go1.11 darwin/amd64

Does this issue reproduce with the latest release?

Yes.

What did you do?

https://play.golang.org/p/XvnF676cmhY

What did you expect to see?

I expected json.Compact to only elide insignificant whitespace as documented, in this case outputting the input untouched.

What did you see instead?

json.Compact escaped U+2028 and U+2029 inside the string, causing the output to be less compact than the input.

The cause appears to be on line 31 of indent.go (https://golang.org/src/encoding/json/indent.go?s=736:1005#L11) where the escape parameter is missing as a branch condition.

@dmitshur dmitshur added this to the Go1.13 milestone Feb 22, 2019

@dmitshur

This comment has been minimized.

Copy link
Member

dmitshur commented Feb 22, 2019

This may be related to #5836, a security issue, which was fixed by escaping these 2 characters in commit d754647.

@PhilipBorgesen

This comment has been minimized.

Copy link
Contributor Author

PhilipBorgesen commented Feb 23, 2019

It definitively is related; I will argue that escaping Compact input was unintended though.
If Compact were meant to transform the input in this surprising way I would have expected the Indent function to showcase the same behavior, which is not the case:

https://play.golang.org/p/JAydUaEnVwB

I would also have expected that the documentation for Compact would have been updated as it was for HTMLEscape but that neither happened.

@dmitshur

This comment has been minimized.

Copy link
Member

dmitshur commented Feb 23, 2019

Would you like to reopen this issue, so that it can be investigated?

@PhilipBorgesen

This comment has been minimized.

Copy link
Contributor Author

PhilipBorgesen commented Feb 23, 2019

Hi dmitshur

I think I accidentially tapped “Close and comment” instead of “Comment”. I did not mean to close the issue.

Sorry for the inconvenience, I want the issue to be investigated.

@bcmills

This comment has been minimized.

Copy link
Member

bcmills commented Feb 25, 2019

CC @rsc @dsnet @bradfitz @mvdan for encoding/json.

(Note also https://github.com/tc39/proposal-json-superset, which may mitigate some of the concerns relating to #5836.)

@bradfitz

This comment has been minimized.

Copy link
Member

bradfitz commented Feb 25, 2019

Perhaps we just add a package comment to encoding/json that says, effectively, that "whenever we say JSON we actually mean JSON that's also valid JavaScript". Then Compact's docs would be correct.

Does that work?

Because I think this is really a documentation omission if anything. I don't think we want to change behavior here.

@PhilipBorgesen

This comment has been minimized.

Copy link
Contributor Author

PhilipBorgesen commented Feb 25, 2019

@bradfitz: If we go with such wording as a package comment I would expect Compact to return an error, not to alter the contents silently. To be clear I do not want errors to generated for RFC 7159 compliant JSON, which is what I expect encoding/json to deal with based on the package comment.

Why do we want Compact to perform this additional escaping? Is it not the purpose and responsibility of the HTMLEscape function?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.