html/template: should escape JSON without using \x #33671

earthboundkid · 2019-08-15T17:34:02Z

What version of Go are you using (`go version`)?

go version go1.12.7 darwin/amd64

Does this issue reproduce with the latest release?

Yes.

What did you do?

package main

import (
	"html/template"
	"log"
	"os"
)

func main() {
	const tpl = `<!DOCTYPE html>
<html>
	<head>
		<meta charset="UTF-8">
		<title>{{ .Title }}</title>
	</head>
	<body>
		<script type="application/ld+json">
		{
			"@context": "http://schema.org",
			"@type": "WebPage",
			"name":"{{ .Title }}",
			"description": "{{ .Description }}"
		}
		</script>
	</body>
</html>`

	check := func(err error) {
		if err != nil {
			log.Fatal(err)
		}
	}
	t, err := template.New("webpage").Parse(tpl)
	check(err)

	data := struct {
		Title, Description string
	}{
		Title:       "<My \"Cool\" Page>",
		Description: "A \"Cool\" Page by 'Me'",
	}

	err = t.Execute(os.Stdout, data)
	check(err)
}

What did you expect to see?

An HTML document containing valid JSON.

What did you see instead?

Invalid JSON, as reported by https://search.google.com/structured-data/testing-tool/

Go escapes what is between the <script> tags as though it were JavaScript because of #26053. That's mostly correct, except that there are subtle difference between valid JavaScript and valid JSON. Specifically, https://search.google.com/structured-data/testing-tool/ reports that \x3c style escaping is not correct for LD JSON. It needs to be \u003c instead.

The text was updated successfully, but these errors were encountered:

earthboundkid · 2019-08-15T17:41:23Z

FWIW, http://json.org agrees that \u escapes are valid JSON and \x escapes are not.

ISTM, the simple solution is to tell the JavaScript escaper to always use \u. JS and JSON have other differences around whitespace handing, but it's easy enough to make them compatible.

agnivade · 2019-08-16T03:36:35Z

@mikesamuel @empijei

empijei · 2019-10-16T07:20:58Z

I investigated this a bit and I confirm it is an issue.

( Some context for future reference: https://json-ld.org/ )

@carlmjohnson would you like to work on a CL for this?
@bcmills this doesn't need further investigation

Harumaro · 2020-03-20T11:33:43Z

We are having this exact same problem as of today, is there a deadline or a workaround for this? It's really impacting our SEO.

earthboundkid · 2020-03-20T12:55:56Z

I started working on a fix, but honestly, I'm in over my head because I don't know what is valid or invalid in JS and I don't understand what all the test cases are testing for.

In terms of a workaround, if you're using Hugo, you can do {{ $x | jsonify | safeJS }}. Other Go templating systems should have similar ways of converting to JSON and marking as safe.

Harumaro · 2020-03-20T17:23:51Z

Thanks, if anybody needs a workaround for gin gonic, it's in Chinese, but Google translate is helpful.
https://codus.me/blog/golang-template-json-ld-error-encoding

empijei · 2020-03-23T09:28:07Z

@Harumaro there is no deadline that I am aware of but if you are willing to work on a fix I can do the review and approval to expedite this.

earthboundkid · 2020-03-23T12:51:50Z

@empijei Sent a broken CL with just the \x → \u change. It needs a lot of test fixes.

gopherbot · 2020-03-23T12:54:00Z

Change https://golang.org/cl/224898 mentions this issue: html/template: escape JS/JSON with \u

empijei · 2020-03-23T13:29:44Z

Thanks for working on this. Please let me know when it is ready for review :)

…

On Mon, 23 Mar 2020 at 13:54, GopherBot ***@***.***> wrote: Change https://golang.org/cl/224898 mentions this issue: html/template: escape JS/JSON with \u — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#33671 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJSU4OPUPOPRG4XMGT2DFDRI5LXVANCNFSM4IMAIKBA> .

liuggio · 2020-03-23T14:06:31Z

I love the PR you made we really need it, without the fixed code we are issuing a lot of errors on js console (SEO problem and logging problems our sentry is out of quota for this)

In terms of a workaround, if you're using Hugo, you can do {{ $x | jsonify | safeJS }}

@carlmjohnson the workaround is not working
https://play.golang.org/p/Elcp4dF7NhS

waiting for this, now we are patching go :)

(edited)
the real missing feature is the "raw" content to avoid unnecessary escaped strings.
or custom fn escaper

earthboundkid · 2020-03-23T14:30:59Z

This fixes your snippet without waiting on a language change (months in the best case): https://play.golang.org/p/pNtKGJgTQs0

empijei · 2020-03-27T17:39:25Z

@empijei Sent a broken CL with just the \x → \u change. It needs a lot of test fixes.

I can take a look and see if I can fix it.

gopherbot · 2020-03-27T18:44:08Z

Change https://golang.org/cl/226097 mentions this issue: html/template: switch to unicode escape instea of hex

earthboundkid · 2020-04-16T17:39:59Z

🎉

earthboundkid changed the title ~~html/template should recognize <script type="application/ld+json">~~ html/template should escape JSON without using \x Aug 15, 2019

bcmills changed the title ~~html/template should escape JSON without using \x~~ html/template: should escape JSON without using \x Aug 19, 2019

bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Aug 19, 2019

bcmills added this to the Go1.14 milestone Aug 19, 2019

earthboundkid mentioned this issue Aug 29, 2019

Make --minify able to be set in config.toml gohugoio/hugo#5142

Closed

gcushen mentioned this issue Sep 20, 2019

Transition from Microdata to JSON-LD HugoBlox/hugo-blox-builder#413

Closed

8 tasks

rsc modified the milestones: Go1.14, Backlog Oct 9, 2019

earthboundkid mentioned this issue Mar 23, 2020

html/template: escape JS/JSON with \u #38022

Closed

gopherbot closed this as completed in d4d2980 Apr 16, 2020

moorereason mentioned this issue May 13, 2020

hugo converts "+" into "\x2b", causing ld+json parsing failure gohugoio/hugo#6695

Closed

golang locked and limited conversation to collaborators Apr 16, 2021

gopherbot added the FrozenDueToAge label Apr 16, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

html/template: should escape JSON without using \x #33671

html/template: should escape JSON without using \x #33671

earthboundkid commented Aug 15, 2019

earthboundkid commented Aug 15, 2019

agnivade commented Aug 16, 2019

empijei commented Oct 16, 2019

Harumaro commented Mar 20, 2020

earthboundkid commented Mar 20, 2020

Harumaro commented Mar 20, 2020

empijei commented Mar 23, 2020

earthboundkid commented Mar 23, 2020

gopherbot commented Mar 23, 2020

empijei commented Mar 23, 2020 via email

liuggio commented Mar 23, 2020 •

edited

Loading

earthboundkid commented Mar 23, 2020 •

edited

Loading

empijei commented Mar 27, 2020

gopherbot commented Mar 27, 2020

earthboundkid commented Apr 16, 2020

html/template: should escape JSON without using \x #33671

html/template: should escape JSON without using \x #33671

Comments

earthboundkid commented Aug 15, 2019

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What did you do?

What did you expect to see?

What did you see instead?

earthboundkid commented Aug 15, 2019

agnivade commented Aug 16, 2019

empijei commented Oct 16, 2019

Harumaro commented Mar 20, 2020

earthboundkid commented Mar 20, 2020

Harumaro commented Mar 20, 2020

empijei commented Mar 23, 2020

earthboundkid commented Mar 23, 2020

gopherbot commented Mar 23, 2020

empijei commented Mar 23, 2020 via email

liuggio commented Mar 23, 2020 • edited Loading

earthboundkid commented Mar 23, 2020 • edited Loading

empijei commented Mar 27, 2020

gopherbot commented Mar 27, 2020

earthboundkid commented Apr 16, 2020

What version of Go are you using (`go version`)?

liuggio commented Mar 23, 2020 •

edited

Loading

earthboundkid commented Mar 23, 2020 •

edited

Loading