encoding/json: how to marshal with unicode escape?

[cupen]

What version of Go are you using (go version)?

$ go version
go version go1.14.2 linux/amd64

What did you expect to see?

module json add a API for marshal string with unicode escape would be useful.

unicode escape
https://tools.ietf.org/html/rfc7159#section-7

package main

import (
	"fmt"
	"encoding/json"
)

type Object struct {
	Name string
}

func main() {
	obj := Object{Name:"哇呀呀"}
	line, _ := json.MarshalUnicodeEscape(obj)
	fmt.Println(string(line))
}

{"Name": "\u54c7\u5440\u5440"}

[mvdan]

Before we start talking about new API, we should first talk about why you need to do that in the first place. You shouldn't need to escape non-ASCII letters in json.

[cupen]

@mvdan I have several program environments, go, python2, js runtime(a v8 app). they have different default character encoding, utf-8(go, python2). ucs2 (js runtime). It need to convert utf-8 to usc2 or usc2 to utf-8 when a json text transferred between them, it's a error-prone job.

I think unicode escape is use for this case., it's a ascii-safe and legal json string encoding format.

[mvdan]

That seems like a very narrow edge case, and I don't think it should be the json package's job to support producing ASCII output alone.

You could consider having a named string type that implements MarshalJSON and replaces all non-ASCII characters with escape codes, if you want. That should be under twenty lines of extra Go code.

[cupen]

Yeah, I know your meaning. here is the e.g. : https://play.golang.org/p/YVSQzad2Z2r

type UnicodeEscape string

func (ue UnicodeEscape) MarshalJSON() ([]byte, error) {
	text := strconv.QuoteToASCII(string(ue))
	return []byte(text), nil
}

type Object struct {
	Name UnicodeEscape
}

But I would to think the unicode-escape is a proposal of json spec, not a hack or a monkey-patch only for python, js(v8) or others.

BTW: json.MarshalUnicodeEscape is a ugly name, maybe I can add a new encoder option for it.

[seankhliao]

json spec says unicode https://tools.ietf.org/html/rfc7159#section-8.1

[cupen]

@seankhliao Yes, it says json text character encoding could be utf-8, utf-16, utf-32, and json string character encoding could be utf-8, utf-16, utf-32 or unicode escape. Sorry to sound like a word game. 😄
https://tools.ietf.org/html/rfc7159#section-7

e.g.:

• json string: "\u54c7\u5440\u5440", it's a json value with string type.
• json text: {"Name": "\u54c7\u5440\u5440"}, it contains all of the json elements, field name, field value and ,:"{}.

For a json text {"Name": "\u54c7\u5440\u5440"}, it's ascii safe no matter which UTF be used.

[networkimprov]

Maybe you could consider a `json:"ascii"` tag for this case?

[mvdan]

I think the few lines of code shown in #39137 (comment) are a completely acceptable solution to this. The json API should only cover common issues and needs. Trying to avoid utf-8 altogether in favor of ascii with unicode escapes certainly feels like an edge case that we shouldn't cover, especially given how easy it is to do with MarshalJSON.

[cupen]

ok, I'll do it by no-std library.