Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

panic in parser #7127

Closed
roidelapluie opened this issue Apr 15, 2020 · 8 comments
Closed

panic in parser #7127

roidelapluie opened this issue Apr 15, 2020 · 8 comments
Labels

Comments

@roidelapluie
Copy link
Member

roidelapluie commented Apr 15, 2020

parser panic: interface conversion: interface {} is *parser.BinaryExpr, not *parser.VectorSelector
goroutine 4130297 [running]:
github.com/prometheus/prometheus/promql/parser.(*parser).recover(0xc010833000, 0xc050374e40)
	/app/promql/parser/parse.go:269 +0x122
panic(0x224d860, 0xc0d7652cc0)
	/usr/local/go/src/runtime/panic.go:679 +0x1b2
github.com/prometheus/prometheus/promql/parser.ParseMetricSelector(0xc26be78ab0, 0x2c, 0x0, 0x0, 0x0, 0x0, 0x0)
	/app/promql/parser/parse.go:156 +0x219
github.com/prometheus/prometheus/web/api/v1.(*API).series(0xc0002d9950, 0xc0d74e0b00, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/app/web/api/v1/api.go:518 +0x7d7
github.com/prometheus/prometheus/web/api/v1.(*API).Register.func1.1(0x2e06680, 0xc0d74edb60, 0xc0d74e0b00)
	/app/web/api/v1/api.go:251 +0x8e
net/http.HandlerFunc.ServeHTTP(0xc00010dd40, 0x2e06680, 0xc0d74edb60, 0xc0d74e0b00)
	/usr/local/go/src/net/http/server.go:2007 +0x44
github.com/prometheus/prometheus/util/httputil.CompressionHandler.ServeHTTP(0x2dced60, 0xc00010dd40, 0x7f4ed5aa1660, 0xc0d74a9f90, 0xc0d74e0b00)
	/app/util/httputil/compression.go:90 +0x7c
github.com/prometheus/prometheus/web.(*Handler).testReady.func1(0x7f4ed5aa1660, 0xc0d74a9f90, 0xc0d74e0b00)
	/app/web/web.go:489 +0x55
net/http.HandlerFunc.ServeHTTP(0xc00010dda0, 0x7f4ed5aa1660, 0xc0d74a9f90, 0xc0d74e0b00)
	/usr/local/go/src/net/http/server.go:2007 +0x44
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerResponseSize.func1(0x7f4ed5aa1660, 0xc0d74a9f40, 0xc0d74e0b00)
	/app/vendor/github.com/prometheus/client_golang/prometheus/promhttp/instrument_server.go:196 +0xe9
net/http.HandlerFunc.ServeHTTP(0xc000822b70, 0x7f4ed5aa1660, 0xc0d74a9f40, 0xc0d74e0b00)
	/usr/local/go/src/net/http/server.go:2007 +0x44
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerDuration.func2(0x7f4ed5aa1660, 0xc0d74a9f40, 0xc0d74e0b00)
	/app/vendor/github.com/prometheus/client_golang/prometheus/promhttp/instrument_server.go:76 +0xb2
net/http.HandlerFunc.ServeHTTP(0xc000822c60, 0x7f4ed5aa1660, 0xc0d74a9f40, 0xc0d74e0b00)
	/usr/local/go/src/net/http/server.go:2007 +0x44
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerCounter.func1(0x2e13e80, 0xc0d74a9ef0, 0xc0d74e0b00)
	/app/vendor/github.com/prometheus/client_golang/prometheus/promhttp/instrument_server.go:100 +0xda
github.com/prometheus/prometheus/web.setPathWithPrefix.func1.1(0x2e13e80, 0xc0d74a9ef0, 0xc0d74e0a00)
	/app/web/web.go:1122 +0x1c6
github.com/prometheus/common/route.(*Router).handle.func1(0x2e13e80, 0xc0d74a9ef0, 0xc0d74e0900, 0x0, 0x0, 0x0)
	/app/vendor/github.com/prometheus/common/route/route.go:83 +0x296
github.com/julienschmidt/httprouter.(*Router).ServeHTTP(0xc000112700, 0x2e13e80, 0xc0d74a9ef0, 0xc0d74e0900)
	/app/vendor/github.com/julienschmidt/httprouter/router.go:334 +0x95e
github.com/prometheus/common/route.(*Router).ServeHTTP(0xc00010c800, 0x2e13e80, 0xc0d74a9ef0, 0xc0d74e0900)
	/app/vendor/github.com/prometheus/common/route/route.go:121 +0x4c
net/http.StripPrefix.func1(0x2e13e80, 0xc0d74a9ef0, 0xc0d74e0800)
	/usr/local/go/src/net/http/server.go:2046 +0x18a
net/http.HandlerFunc.ServeHTTP(0xc00075f920, 0x2e13e80, 0xc0d74a9ef0, 0xc0d74e0800)
	/usr/local/go/src/net/http/server.go:2007 +0x44
net/http.(*ServeMux).ServeHTTP(0xc000112680, 0x2e13e80, 0xc0d74a9ef0, 0xc0d74e0800)
	/usr/local/go/src/net/http/server.go:2387 +0x1bd
github.com/opentracing-contrib/go-stdlib/nethttp.MiddlewareFunc.func5(0x2e12480, 0xc0d74e8700, 0xc0d74e0700)
	/app/vendor/github.com/opentracing-contrib/go-stdlib/nethttp/server.go:140 +0x5fb
net/http.HandlerFunc.ServeHTTP(0xc00075f9e0, 0x2e12480, 0xc0d74e8700, 0xc0d74e0700)
	/usr/local/go/src/net/http/server.go:2007 +0x44
github.com/prometheus/prometheus/web.withStackTracer.func1(0x2e12480, 0xc0d74e8700, 0xc0d74e0700)
	/app/web/web.go:103 +0xb3
net/http.HandlerFunc.ServeHTTP(0xc00075fa10, 0x2e12480, 0xc0d74e8700, 0xc0d74e0700)
	/usr/local/go/src/net/http/server.go:2007 +0x44
net/http.serverHandler.ServeHTTP(0xc00076e000, 0x2e12480, 0xc0d74e8700, 0xc0d74e0700)
	/usr/local/go/src/net/http/server.go:2802 +0xa4
net/http.(*conn).serve(0xc0d15bab40, 0x2e1c480, 0xc0f7028a00)
	/usr/local/go/src/net/http/server.go:1890 +0x875
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:2928 +0x384

Spotted in #7120

apparently 2.17.1

@roidelapluie
Copy link
Member Author

ping @slrtbtfs

@slrtbtfs
Copy link
Contributor

Is the query that caused this crash known?

@roidelapluie
Copy link
Member Author

No, I did not ask. It is an /api/series query.

@roidelapluie
Copy link
Member Author

cc @vsliouniaev do you have an idea about it? this comes from your logs. Thanks.

@slrtbtfs
Copy link
Contributor

Ah, looks like someone put something like a+b as API argument, where only a simple selector would be allowed.

The panic seems to be neither dramatic, nor hard to fix.

I can do a PR tomorrow.

@roidelapluie
Copy link
Member Author

Except that does not panic:

curl -d match[]=a+b -g 'http://127.0.0.1:9090/api/v1/admin/tsdb/delete_series'
{"status":"error","errorType":"bad_data","error":"1:3: parse error: unexpected identifier "b""}

@slrtbtfs
Copy link
Contributor

Hmmm... I'll have a closer look tomorrow.

roidelapluie added a commit to roidelapluie/prometheus that referenced this issue Apr 15, 2020
Reusing the same generatedParserResult ends up in strange panics:
See prometheus#7131 and prometheus#7127.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
@roidelapluie
Copy link
Member Author

closed by #7132

roidelapluie added a commit to roidelapluie/prometheus that referenced this issue Apr 17, 2020
Reusing the same generatedParserResult ends up in strange panics:
See prometheus#7131 and prometheus#7127.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
@prometheus prometheus locked as resolved and limited conversation to collaborators Nov 27, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

2 participants