Wrong Content-Type leads to stack overflow #1978

k-tipp · 2019-07-05T15:51:01Z

go version: 1.12.6
gin version (or commit ref): v1.4.0-dev
operating system: win10

Description

Stupid people do stupid things. In my case I selected <application/javascript> as Content-Type in Postman instead of <application/json> and was wondering why the heck my code doesn't work as expected.... well, reason found. Anyway an error message which is a little more specific than the following one would have been nice.

Note: I only tested javascript vs. json header, the error is repeatable and it seems that the content of the json message is not relevant.

Relevant code snipped

func(c *gin.Context) {
     org := Organization{}
    if err := c.ShouldBind(&org); err == nil { // within c.ShouldBind() the stack overflow happens
	// not relevant
	c.Header("Location", loc.String())
    } else {
        // Never reached in this case
        log.Println(err)
    }
}

Error message

runtime: goroutine stack exceeds 1000000000-byte limit
fatal error: stack overflow

runtime stack:
runtime.throw(0xd9f020, 0xe)
	c:/go/src/runtime/panic.go:617 +0x79
runtime.newstack()
	c:/go/src/runtime/stack.go:1041 +0x7b7
runtime.morestack()
	c:/go/src/runtime/asm_amd64.s:429 +0x97

goroutine 20 [running]:
github.com/gin-gonic/gin/binding.tryToSetValue(0xc87660, 0xc00307af30, 0x198, 0xc53b26, 0x2, 0x0, 0x0, 0xe7a280, 0xc87660, 0xc53b2a, ...)
	D:/gopath/src/github.com/gin-gonic/gin/binding/form_mapping.go:108 +0x6cc fp=0xc020801458 sp=0xc020801450 pc=0xa5463c
github.com/gin-gonic/gin/binding.mapping(0xc87660, 0xc00307af30, 0x198, 0xc53b26, 0x2, 0x0, 0x0, 0xe7a280, 0xc87660, 0xc53b2a, ...)
	D:/gopath/src/github.com/gin-gonic/gin/binding/form_mapping.go:74 +0x4d1 fp=0xc0208016b0 sp=0xc020801458 pc=0xa53a81
github.com/gin-gonic/gin/binding.mapping(0xcfc580, 0xc00307af30, 0x199, 0xc14a7d, 0x7, 0x0, 0x0, 0xe7a280, 0xcbf9c0, 0x0, ...)
	D:/gopath/src/github.com/gin-gonic/gin/binding/form_mapping.go:92 +0x85b fp=0xc020801908 sp=0xc0208016b0 pc=0xa53e0b
github.com/gin-gonic/gin/binding.mapping(0xcbf9c0, 0xc003076960, 0x196, 0xc14a7d, 0x7, 0x0, 0x0, 0xe7a280, 0xcbf9c0, 0x0, ...)
	D:/gopath/src/github.com/gin-gonic/gin/binding/form_mapping.go:63 +0x2a6 fp=0xc020801b60 sp=0xc020801908 pc=0xa53856
github.com/gin-gonic/gin/binding.mapping(0xd45ba0, 0xc003076960, 0x199, 0xca94d5, 0xa, 0x0, 0x0, 0xe7a280, 0xcbfae0, 0xca94e1, ...)
	D:/gopath/src/github.com/gin-gonic/gin/binding/form_mapping.go:92 +0x85b fp=0xc020801db8 sp=0xc020801b60 pc=0xa53e0b
github.com/gin-gonic/gin/binding.mapping(0xcbfae0, 0xc00306f3a8, 0x196, 0xca94d5, 0xa, 0x0, 0x0, 0xe7a280, 0xcbfae0, 0xca94e1, ...)
	D:/gopath/src/github.com/gin-gonic/gin/binding/form_mapping.go:63 +0x2a6 fp=0xc020802010 sp=0xc020801db8 pc=0xa53856
github.com/gin-gonic/gin/binding.mapping(0xd363a0, 0xc00306f380, 0x199, 0xc89710, 0x8, 0x0, 0x0, 0xe7a280, 0xcbfde0, 0xc8971a, ...)
	D:/gopath/src/github.com/gin-gonic/gin/binding/form_mapping.go:92 +0x85b fp=0xc020802268 sp=0xc020802010 pc=0xa53e0b
github.com/gin-gonic/gin/binding.mapping(0xcbfde0, 0xc003076958, 0x196, 0xc89710, 0x8, 0x0, 0x0, 0xe7a280, 0xcbfde0, 0xc8971a, ...)
........
...additional frames elided...
created by net/http.(*Server).Serve
	c:/go/src/net/http/server.go:2884 +0x7be

Edit 2:

Golang structs:

type Organization struct {
	// *datatype.DomainResource
	// Identifier []*datatype.Identifier      `json:"identifier,omitempty"`
	// Active bool `json:"active,omitempty" dgraph:"org_identifier,omitempty"`
	// Type       []*datatype.CodeableConcept `json:"type,omitempty"`
	// Name string `json:"name,omitempty"`
	// Alias      []string                    `json:"alias,omitempty"`
	// Telecom    []*datatype.ContactPoint    `json:"telecom,omitempty"`
	// Address    []*datatype.Address         `json:"address,omitempty"`
	PartOf *datatype.Reference `json:"partOf,omitempty"`
	// Contact  []*OrganizationContact `json:"contact,omitempty"`
	Endpoint []*datatype.Reference `json:"endpoint,omitempty"`
}

type Reference struct {
	// *Element
	// Ref        string     `json:"reference,omitempty"`
	// Type       string     `json:"type,omitempty"`
	Identifier *Identifier `json:"identifier,omitempty"`
	// Display    string     `json:"display,omitempty"`
}

type Identifier struct {
	// *Element
	// Use    string           `json:"use,omitempty"`
	// Type   *CodeableConcept `json:"type,omitempty"`
	// System string           `json:"system,omitempty"`
	// Value  string           `json:"value,omitempty"`
	// Period *Period          `json:"period,omitempty"`
	Assigner *Reference `json:"assigner,omitempty"`
}

Necessary request to trigger the stack overflow: Any, even an empty request body is enough.

The text was updated successfully, but these errors were encountered:

guonaihong · 2019-07-06T01:41:25Z

What is the definition of the Organization structure?

k-tipp · 2019-07-06T18:08:12Z

It's the FHIR resource "Organization", the definition is here and there are also some examples. I tested various JSON messages, even very simple ones, and the error was there. But I didn't checked, if the Go structure has a influence on the result, I will upload the structure on Monday and check if the behavior changes if I simplify it.

k-tipp · 2019-07-08T13:36:18Z

I narrowed down the problem a little.

The error only happens when Organization.PartOf and Organization.Endpoint aren't commented out.
Looking at my code, using so many pointers wasn't the best idea. When removing the pointers on Reference.Identifier and Identifier.Assigner the IDE shows an error for an invalid recursive type. This is the root of my problem.
It's more a json deserialization problem than a gin problem, sorry for the noise.

k-tipp · 2019-07-08T15:18:52Z

I reopened the issue, because it is a gin problem, I forgot, that the problem is only there when the Content-Type is wrong.

guonaihong · 2019-07-10T01:11:30Z

Can you test this pr to fix the problem?
#1943

k-tipp · 2019-07-10T08:59:45Z

Thank you very much for your work! The problem is still there, but the stack trace changed a little:

runtime stack:
runtime.throw(0xd9d2f0, 0xe)
	c:/go/src/runtime/panic.go:617 +0x79
runtime.newstack()
	c:/go/src/runtime/stack.go:1041 +0x7b7
runtime.morestack()
	c:/go/src/runtime/asm_amd64.s:429 +0x97

goroutine 50 [running]:
runtime.mapaccess2_faststr(0xcb4ce0, 0xc00018a450, 0xc3a654, 0x2, 0x0, 0x0)
	c:/go/src/runtime/map_faststr.go:107 +0x7b0 fp=0xc020401310 sp=0xc020401308 pc=0x415930
github.com/gin-gonic/gin/binding.setByForm(0xc86360, 0xc00279e420, 0x198, 0xc3a654, 0x2, 0x0, 0x0, 0xe78240, 0xc86360, 0xc3a658, ...)
	D:/gopath/src/github.com/gin-gonic/gin/binding/form_mapping.go:140 +0x9a fp=0xc0204014c8 sp=0xc020401310 pc=0xa54a5a
github.com/gin-gonic/gin/binding.formSource.TrySet(0xc00018a450, 0xc86360, 0xc00279e420, 0x198, 0xc3a654, 0x2, 0x0, 0x0, 0xe78240, 0xc86360, ...)
	D:/gopath/src/github.com/gin-gonic/gin/binding/form_mapping.go:45 +0x10b fp=0xc0204015c8 sp=0xc0204014c8 pc=0xa5369b
github.com/gin-gonic/gin/binding.tryToSetValue(0xc86360, 0xc00279e420, 0x198, 0xc3a654, 0x2, 0x0, 0x0, 0xe78240, 0xc86360, 0xc3a658, ...)
	D:/gopath/src/github.com/gin-gonic/gin/binding/form_mapping.go:136 +0x5b6 fp=0xc020401758 sp=0xc0204015c8 pc=0xa54906
github.com/gin-gonic/gin/binding.mapping(0xc86360, 0xc00279e420, 0x198, 0xc3a654, 0x2, 0x0, 0x0, 0xe78240, 0xc86360, 0xc3a658, ...)
	D:/gopath/src/github.com/gin-gonic/gin/binding/form_mapping.go:78 +0x56e fp=0xc0204019c0 sp=0xc020401758 pc=0xa53e5e
github.com/gin-gonic/gin/binding.mapping(0xcfa9c0, 0xc00279e420, 0x199, 0xc12a7d, 0x7, 0x0, 0x0, 0xe78240, 0xcbe080, 0x0, ...)
	D:/gopath/src/github.com/gin-gonic/gin/binding/form_mapping.go:96 +0x8f8 fp=0xc020401c28 sp=0xc0204019c0 pc=0xa541e8
github.com/gin-gonic/gin/binding.mapping(0xcbe080, 0xc00279e3f0, 0x196, 0xc12a7d, 0x7, 0x0, 0x0, 0xe78240, 0xcbe080, 0x0, ...)
	D:/gopath/src/github.com/gin-gonic/gin/binding/form_mapping.go:67 +0x343 fp=0xc020401e90 sp=0xc020401c28 pc=0xa53c33
github.com/gin-gonic/gin/binding.mapping(0xd0f960, 0xc00279e3f0, 0x199, 0xc40b6d, 0x4, 0x0, 0x0, 0xe78240, 0xcbd660, 0xc40b73, ...)
	D:/gopath/src/github.com/gin-gonic/gin/binding/form_mapping.go:96 +0x8f8 fp=0xc0204020f8 sp=0xc020401e90 pc=0xa541e8

To give you some more information about whats happening I added some additional print statements to form_mapping.go. I added the statements after I captured the above stack trace, so the line numbers are correct:

2019/07/10 10:57:53 called github.com/gin-gonic/gin/binding/form_mapping.go#tryToSetValue(value, field, setter, tag) with params: { value: , field: {ID  string json:"id,omitempty" %!s(uintptr=0) [%!s(int=0)] %!s(bool=false)}, setter: map[], tag: form}
2019/07/10 10:57:53 github.com/gin-gonic/gin/binding/form_mapping.go:136, return setter.TrySet(value, field, tagValue, setOpt): value: , field: {ID  string json:"id,omitempty" %!s(uintptr=0) [%!s(int=0)] %!s(bool=false)}, tagValue: ID, setOpt: {%!s(bool=false) }
2019/07/10 10:57:53 called github.com/gin-gonic/gin/binding/form_mapping.go#setByForm(value, field, form, tagValue, opt) with params: {value: , field: {ID  string json:"id,omitempty" %!s(uintptr=0) [%!s(int=0)] %!s(bool=false)}, form: map[], tagValue: ID, opt: {%!s(bool=false) }}
2019/07/10 10:57:54 called github.com/gin-gonic/gin/binding/form_mapping.go#mapping(value, field, setter, tag) with params { value: [], field: {Extension  []*datatype.Extension json:"extension,omitempty" %!s(uintptr=16) [%!s(int=1)] %!s(bool=false)}, setter: map[], tag: form}
2019/07/10 10:57:54 github.com/gin-gonic/gin/binding/form_mapping.go:61 vKind: slice
2019/07/10 10:57:54 called github.com/gin-gonic/gin/binding/form_mapping.go#tryToSetValue(value, field, setter, tag) with params: { value: [], field: {Extension  []*datatype.Extension json:"extension,omitempty" %!s(uintptr=16) [%!s(int=1)] %!s(bool=false)}, setter: map[], tag: form}
2019/07/10 10:57:54 github.com/gin-gonic/gin/binding/form_mapping.go:136, return setter.TrySet(value, field, tagValue, setOpt): value: [], field: {Extension  []*datatype.Extension json:"extension,omitempty" %!s(uintptr=16) [%!s(int=1)] %!s(bool=false)}, tagValue: Extension, setOpt: {%!s(bool=false) }
2019/07/10 10:57:54 called github.com/gin-gonic/gin/binding/form_mapping.go#setByForm(value, field, form, tagValue, opt) with params: {value: [], field: {Extension  []*datatype.Extension json:"extension,omitempty" %!s(uintptr=16) [%!s(int=1)] %!s(bool=false)}, form: map[], tagValue: Extension, opt: {%!s(bool=false) }}
2019/07/10 10:57:55 called github.com/gin-gonic/gin/binding/form_mapping.go#mapping(value, field, setter, tag) with params { value: , field: {Ref  string json:"reference,omitempty" %!s(uintptr=8) [%!s(int=1)] %!s(bool=false)}, setter: map[], tag: form}
2019/07/10 10:57:55 github.com/gin-gonic/gin/binding/form_mapping.go:61 vKind: string
2019/07/10 10:57:55 called github.com/gin-gonic/gin/binding/form_mapping.go#tryToSetValue(value, field, setter, tag) with params: { value: , field: {Ref  string json:"reference,omitempty" %!s(uintptr=8) [%!s(int=1)] %!s(bool=false)}, setter: map[], tag: form}
2019/07/10 10:57:55 github.com/gin-gonic/gin/binding/form_mapping.go:136, return setter.TrySet(value, field, tagValue, setOpt): value: , field: {Ref  string json:"reference,omitempty" %!s(uintptr=8) [%!s(int=1)] %!s(bool=false)}, tagValue: Ref, setOpt: {%!s(bool=false) }
2019/07/10 10:57:55 called github.com/gin-gonic/gin/binding/form_mapping.go#setByForm(value, field, form, tagValue, opt) with params: {value: , field: {Ref  string json:"reference,omitempty" %!s(uintptr=8) [%!s(int=1)] %!s(bool=false)}, form: map[], tagValue: Ref, opt: {%!s(bool=false) }}
2019/07/10 10:57:56 called github.com/gin-gonic/gin/binding/form_mapping.go#mapping(value, field, setter, tag) with params { value: , field: {Type  string json:"type,omitempty" %!s(uintptr=24) [%!s(int=2)] %!s(bool=false)}, setter: map[], tag: form}
2019/07/10 10:57:56 github.com/gin-gonic/gin/binding/form_mapping.go:61 vKind: string
2019/07/10 10:57:56 called github.com/gin-gonic/gin/binding/form_mapping.go#tryToSetValue(value, field, setter, tag) with params: { value: , field: {Type  string json:"type,omitempty" %!s(uintptr=24) [%!s(int=2)] %!s(bool=false)}, setter: map[], tag: form}
2019/07/10 10:57:56 github.com/gin-gonic/gin/binding/form_mapping.go:136, return setter.TrySet(value, field, tagValue, setOpt): value: , field: {Type  string json:"type,omitempty" %!s(uintptr=24) [%!s(int=2)] %!s(bool=false)}, tagValue: Type, setOpt: {%!s(bool=false) }
2019/07/10 10:57:56 called github.com/gin-gonic/gin/binding/form_mapping.go#setByForm(value, field, form, tagValue, opt) with params: {value: , field: {Type  string json:"type,omitempty" %!s(uintptr=24) [%!s(int=2)] %!s(bool=false)}, form: map[], tagValue: Type, opt: {%!s(bool=false) }}

In both cases I used an empty request body.

guonaihong · 2019-07-10T09:13:07Z

Thank you, Can you give me the client test data, Server that can be run, and client code,this question looks very interesting.

k-tipp · 2019-07-10T11:59:46Z

I created a repository with (hopefully) everything you need to reproduce the error. gin-stackoverflow-demo

The tests commented with "Successful" do not end with a fatal error.

In the go, the stack burst will occupy 1G memory, the error message is as follows ```runtime: goroutine stack exceeds 1000000000-byte limit``` If it is a user of gin, the structure of the circular reference is defined on the server side, and the server memory may be used up as long as 100 go processes, and even the server may be down. So I suggest that the mapping function should add a limit on the number of calls, and the number of times can be configured. Of course, I tried several other modifications when I solved the stackoverflow problem described in gin-gonic#1978, but it was not compatible with the existing API, so I did not adopt the solution I tried.

vkd · 2019-07-14T10:47:05Z

About the reasons of this problem:

This problem happens only on "Form" binding, and does not depend on input data.

When you select a Content-Type is <application/json>, the gin binding uses default golang "encoding/json" package for binding a structs, but by default gin uses the custom "Form" binding.

Custom "Form" binding is walking by struct's fields, and if it sees the pointer to another struct then it go deeper (also if it is a nil pointer value). And for current implementation of "Form" binding there is no a detecting for a recurive struct definitions.

In your case, type "Reference" and "Identifier" reference each to another, and "Form" binding is trying to create new empty struct, assign it and going deeper:

type Reference struct {
	Identifier *Identifier `json:"identifier,omitempty"`
}
type Identifier struct {
	Assigner *Reference `json:"assigner,omitempty"`
}

k-tipp · 2019-07-16T06:54:29Z

Thank you very much for the clarification. I leave it to the admins of this repo if they want to close this issue or not. Personally I would prefer an early wrong content type error message over the current behaviour.

k-tipp closed this as completed Jul 8, 2019

k-tipp reopened this Jul 8, 2019

guonaihong mentioned this issue Jul 13, 2019

Improve the problem described in #1978 #1989

Open

JellyZhang mentioned this issue Feb 21, 2020

c.ShouldBind() maybe will stack overflow #2203

Open

thxCode mentioned this issue Apr 28, 2023

refactor: replace a panic with an error when walking too deep on form mapping #3586

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Wrong Content-Type leads to stack overflow #1978

Wrong Content-Type leads to stack overflow #1978

k-tipp commented Jul 5, 2019 •

edited

guonaihong commented Jul 6, 2019

k-tipp commented Jul 6, 2019

k-tipp commented Jul 8, 2019

k-tipp commented Jul 8, 2019

guonaihong commented Jul 10, 2019

k-tipp commented Jul 10, 2019

guonaihong commented Jul 10, 2019 •

edited

k-tipp commented Jul 10, 2019

vkd commented Jul 14, 2019

k-tipp commented Jul 16, 2019

Wrong Content-Type leads to stack overflow #1978

Wrong Content-Type leads to stack overflow #1978

Comments

k-tipp commented Jul 5, 2019 • edited

Description

Relevant code snipped

Error message

Edit 2:

guonaihong commented Jul 6, 2019

k-tipp commented Jul 6, 2019

k-tipp commented Jul 8, 2019

k-tipp commented Jul 8, 2019

guonaihong commented Jul 10, 2019

k-tipp commented Jul 10, 2019

guonaihong commented Jul 10, 2019 • edited

k-tipp commented Jul 10, 2019

vkd commented Jul 14, 2019

k-tipp commented Jul 16, 2019

k-tipp commented Jul 5, 2019 •

edited

guonaihong commented Jul 10, 2019 •

edited