Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encoding/json: does not recognise semicolon as a valid field name #39189

Open
kolatat opened this issue May 21, 2020 · 6 comments
Open

encoding/json: does not recognise semicolon as a valid field name #39189

kolatat opened this issue May 21, 2020 · 6 comments
Milestone

Comments

@kolatat
Copy link

@kolatat kolatat commented May 21, 2020

What version of Go are you using (go version)?

go1.14 windows/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

windows amd64

What did you do?

import (
	"encoding/json"
	"fmt"
)

func main() {
	encoded := []byte(`{";": "World!"}`)
	type MyObject struct {
		Hello string `json:";"`
	}
	var decoded MyObject
	if err := json.Unmarshal(encoded, &decoded); err != nil {
		fmt.Println(err)
		return
	}
	fmt.Printf("%+v", decoded)
}

What did you expect to see?

{Hello:World!}

What did you see instead?

{Hello:}

@natebwangsut

@gopherbot
Copy link

@gopherbot gopherbot commented May 21, 2020

Change https://golang.org/cl/234818 mentions this issue: encoding/json: allow semicolon in field key / struct tag

@cagedmantis cagedmantis added this to the Backlog milestone May 21, 2020
@cagedmantis
Copy link
Contributor

@cagedmantis cagedmantis commented May 21, 2020

@mvdan
Copy link
Member

@mvdan mvdan commented May 21, 2020

I don't see a reason why not, but where do we draw the line on which characters are OK and which are not? And why is the semicolon part of the first group?

I'm not familiar with the history of that piece of code, so I really am asking. I think that needs an answer before we review a change.

@natebwangsut
Copy link

@natebwangsut natebwangsut commented May 22, 2020

So pretty much we check JSON spec (RFC-7159) for validity on our "bug" and it seems to us that the spec would treat a semicolon as a normal character.

https://tools.ietf.org/html/rfc7159

@dsnet
Copy link
Member

@dsnet dsnet commented May 22, 2020

I don't see a reason why not, but where do we draw the line on which characters are OK and which are not?

Back in February of 2011, the entirety of the struct tag was used as the JSON key name. It seems that the name syntax was restricted (https://golang.org/cl/4173061) so that the tags could in theory be used for other purposes like protocol buffers (#1520).

Later in June of 2011, a well-defined grammar for application-specific struct tags was defined and formally implemented in the reflect package (https://golang.org/cl/4645069).

It seems to me that the restricted set of valid characters is an artifact from a previous era to work around a limitation that no longer applies today.

The only restriction I can imagine for the character set would be a , since it is used to delimit the set of extra tag attributes that come after the name. In theory we could define a more complex grammar where someone could put a quoted string as to encode any arbitrary name that is valid UTF-8.

If the grammar is opened up to other characters, we'll need to consider how the equalFold logic is supposed to operate.

@seankhliao
Copy link
Contributor

@seankhliao seankhliao commented May 22, 2020

relaxing the restrictions would also fix #22518 and #35287

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants
You can’t perform that action at this time.