encoding/json: No way to avoid HTMLEscape when Marshal()-ing #8592

gopherbot · 2014-08-26T04:15:10Z

by surtri:

Before filing a bug, please check whether it has been fixed since the
latest release. Search the issue tracker and check that you're running the
latest version of Go:

Run "go version" and compare against
http://golang.org/doc/devel/release.html  If a newer version of Go exists,
install it and retry what you did to reproduce the problem.

Thanks.

What does 'go version' print?

What steps reproduce the problem?
If possible, include a link to a program on play.golang.org.

1. http://play.golang.org/p/SQ2j7c2mpQ
2. There used to be a distinction between Marshal() and MarshalForHTML(), but that was
removed because it was deemed that HTMLEscaping < and > would always be better.
For my use-case, I want to keep those characters because:

a) It makes the resulting JSON file smaller
b) It makes the raw JSON more readable

What happened?

"\u003chtml\u003e"

What should have happened instead?

"<html>"

Please provide any additional information below.

I know this issue has been discussed before
(https://golang.org/issue/3127) but I've been unable to find a way
to disable the HTMLEscape() method. Is there any way to get around that? It makes
everything much less readable and bigger (I transfer terabytes of JSON a month so these
small changes add up). I think there would be value in offering a method for opting out
of this automatic HTML escape feature.

bradfitz · 2014-08-26T07:41:04Z

Comment 1:

Perhaps a bool on the Encoder struct.

Labels changed: added repo-main.

gopherbot · 2014-09-16T06:25:46Z

Comment 2 by wangz@google.com:

Automatic HTML-escaping is irrelevant to JSON encoding. Therefore the feature should be
turned off by default.
The "auto-escaping is safer" argument in the original thread
(https://golang.org/issue/3127) is bogus because it all depends on
the context where the serialized data is used.
Before this gets fixed, one quick workaround is to fork the encoding/json module and
remove "&& b != '<' && b != '>' && b != '&' " from the following two functions in
encode.go:
func (e *encodeState) string(s string) (int, error)
func (e *encodeState) stringBytes(s []byte) (int, error)

rsc · 2014-09-16T13:51:00Z

Comment 3:

The result is valid JSON. The details of how it gets encoded are not guaranteed and
don't need to be under user control. The only guarantee is that the result is valid and
semantically correct JSON, which it is. It is not worth making the API more complex to
control whether < > get escaped. If you really care, do a global search and
replace on the output to put them back.

Status changed to WorkingAsIntended.

gopherbot · 2014-09-16T17:29:49Z

Comment 4 by wangz@google.com:

The result is valid json but also unnecessarily inflated in many contexts where '<',
'>', and '&' abound.
Doing string replacement in a second pass is an inefficient and cumbersome workaround.
Please consider Brad's suggestion to add a bool to the Encoder struct.

gopherbot · 2014-10-14T23:28:49Z

Comment 5 by karim.nassar@vervemobile.com:

+1 for flag. In my context, I am generating the html that is in response to an api that,
in some cases, mismanages the escaped text. I can't control them, and I trust myself.
Forking the json pkg seems pretty heavy weight, and string replacement is woefully
inefficient when milliseconds matter.

gopherbot added workingasintended labels Oct 14, 2014

flowbehappy mentioned this issue Jun 2, 2015

flag for controlling HTMLEscape when Marshalling pquerna/ffjson#141

Open

dmclain mentioned this issue Dec 30, 2015

Slack messages can't send urls as markup prometheus/alertmanager#207

Closed

rgarcia mentioned this issue Mar 9, 2016

Ignore transfer-encoding and content-length headers Clever/http-science#9

Merged

golang locked and limited conversation to collaborators Jun 25, 2016

gopherbot added the FrozenDueToAge label Jun 25, 2016

This issue was closed.

golang / go

encoding/json: No way to avoid HTMLEscape when Marshal()-ing #8592

encoding/json: No way to avoid HTMLEscape when Marshal()-ing #8592

gopherbot commented Aug 26, 2014

bradfitz commented Aug 26, 2014

gopherbot commented Sep 16, 2014

rsc commented Sep 16, 2014

gopherbot commented Sep 16, 2014

gopherbot commented Oct 14, 2014

golang / go

encoding/json: No way to avoid HTMLEscape when Marshal()-ing #8592

encoding/json: No way to avoid HTMLEscape when Marshal()-ing #8592

Comments

gopherbot commented Aug 26, 2014

bradfitz commented Aug 26, 2014

gopherbot commented Sep 16, 2014

rsc commented Sep 16, 2014

gopherbot commented Sep 16, 2014

gopherbot commented Oct 14, 2014