New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

text/scanner: panic: runtime error: slice bounds out of range #29723

Closed
en-vee opened this Issue Jan 14, 2019 · 4 comments

Comments

Projects
None yet
5 participants
@en-vee
Copy link

en-vee commented Jan 14, 2019

What version of Go are you using (go version)?

$ go version
go version go1.11 windows/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\neeraj.vaidya\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\Users\neeraj.vaidya\go
set GOPROXY=
set GORACE=
set GOROOT=C:\Go
set GOTMPDIR=
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\NEERAJ~1.VAI\AppData\Local\Temp\go-build083329666=/tmp/go-build -gno-record-gcc-switches

What did you do?

  • I am using the text/scanner package to tokenize a UTF-8 encoded text file.
  • The text file (say input.conf) contains the following
propertyKey1 = "unTerminatedValue
propertyKey2 = "validValue"
  • In order to trap the error arising from the unterminated string literal, I create an error Handler function and set it into the scan.Error field
  • But when I call scan.TokenText() method to get the most recently scanned Rune, inside the error handler, I get slice index out of bounds panic/error when attempting this on the unterminated string literal.
  • Upon debugging, I can see that the scan.TokenText() method tries to return a slice from a byte array
// text/scanner/scanner.go
func (s *Scanner) TokenText() string {
...
...
if s.tokBuf.Len() == 0 {
		// common case: the entire token text is still in srcBuf
		return string(s.srcBuf[s.tokPos:s.tokEnd])
	}
  • And the value of s.tokPos is greater-than s.tokEnd.
  • This probably causes the "index out of bounds" error.

Go PlayGround Link : https://play.golang.org/p/uIUPfT0L2Dx

What did you expect to see?

I expected that the scan.TokenText() method inside the error handler would return the last scanned token which is the unterminated literal.

What did you see instead?

I saw a panic message saying "slice bounds out of range"

panic: runtime error: slice bounds out of range

goroutine 1 [running]:
text/scanner.(*Scanner).TokenText(0x452000, 0x41c660, 0xa89e0, 0x1007)
	/usr/local/go/src/text/scanner/scanner.go:678 +0x260
main.main.func1(0x452000, 0x102196, 0x16, 0x0)
	/tmp/sandbox248235930/main.go:17 +0x40
text/scanner.(*Scanner).error(0x452000, 0x102196, 0x16, 0x1007)
	/usr/local/go/src/text/scanner/scanner.go:327 +0x280
text/scanner.(*Scanner).scanString(0x452000, 0x22, 0x0, 0xef180)
	/usr/local/go/src/text/scanner/scanner.go:476 +0xe0
text/scanner.(*Scanner).Scan(0x452000, 0x13, 0x41c7a8, 0x1)
	/usr/local/go/src/text/scanner/scanner.go:593 +0x440
main.main()
	/tmp/sandbox248235930/main.go:20 +0x280

@agnivade agnivade changed the title text/scanner package unexpected panic text/scanner: panic: runtime error: slice bounds out of range Jan 14, 2019

@agnivade agnivade added the NeedsFix label Jan 14, 2019

@agnivade agnivade added this to the Go1.13 milestone Jan 14, 2019

@Gnouc

This comment has been minimized.

Copy link
Contributor

Gnouc commented Jan 14, 2019

The document said s.TokenText() only valid after calling s.Scan(). At the time s.Error is called, s.Scan() was not finished, so s.TokenText() is not valid.

@agnivade

This comment has been minimized.

Copy link
Member

agnivade commented Jan 14, 2019

/cc @griesemer - Is the panic intentional ? Then it should be certainly documented.

@griesemer

This comment has been minimized.

Copy link
Contributor

griesemer commented Jan 14, 2019

@agnivade This looks like a bug.

@griesemer griesemer self-assigned this Jan 14, 2019

@griesemer griesemer added the Suggested label Jan 14, 2019

@gopherbot

This comment has been minimized.

Copy link

gopherbot commented Jan 14, 2019

Change https://golang.org/cl/157819 mentions this issue: text/scanner: don't crash when calling TokenText in error handler

@gopherbot gopherbot closed this in 85c1798 Feb 11, 2019

nebulabox added a commit to nebulabox/go that referenced this issue Feb 18, 2019

text/scanner: don't crash when calling TokenText in error handler
Make sure Scanner.tokEnd is set before we call Scanner.Error
and update documentation accordingly.
(Until now tokEnd was only set before returning from Scan,
so a call to TokenText during error handling may have crashed.)

While at it, tighten a check in Scanner.TokenText to ensure
Scanner.tokEnd >= Scanner.tokPos if we have a token.

Also, silence error messages to Stderr in unrelated TestIllegalExponent.

Fixes golang#29723.

Change-Id: Ia97beeae91eaf9e0ed3dada0a806f1f7122461cc
Reviewed-on: https://go-review.googlesource.com/c/157819
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

nebulabox added a commit to nebulabox/go that referenced this issue Feb 20, 2019

text/scanner: don't crash when calling TokenText in error handler
Make sure Scanner.tokEnd is set before we call Scanner.Error
and update documentation accordingly.
(Until now tokEnd was only set before returning from Scan,
so a call to TokenText during error handling may have crashed.)

While at it, tighten a check in Scanner.TokenText to ensure
Scanner.tokEnd >= Scanner.tokPos if we have a token.

Also, silence error messages to Stderr in unrelated TestIllegalExponent.

Fixes golang#29723.

Change-Id: Ia97beeae91eaf9e0ed3dada0a806f1f7122461cc
Reviewed-on: https://go-review.googlesource.com/c/157819
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment