strconv: "invalid array index" compiler error was reported after changing reloc type

[erifan]

What version of Go are you using (go version)?

$ go version
go version devel +4ae31dc8c5 Fri May 10 00:14:40 2019 +0000 linux/arm64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="arm64"
GOBIN=""
GOCACHE="/home/hostname/.cache/go-build"
GOENV="/home/hostname/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/hostname/gopath/"
GOPROXY="direct"
GOROOT="/home/hostname/go"
GOSUMDB="off"
GOTMPDIR=""
GOTOOLDIR="/home/hostname/go/pkg/tool/linux_arm64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/hostname/go/src/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build652146777=/tmp/go-build -gno-record-gcc-switches"

What did you do?

A small change:
$ cd go/src
$ git diff

diff --git a/src/cmd/internal/objabi/reloctype.go b/src/cmd/internal/objabi/reloctype.go
index a28b48ba50..5dc9356fe1 100644
--- a/src/cmd/internal/objabi/reloctype.go
+++ b/src/cmd/internal/objabi/reloctype.go
@@ -149,6 +149,9 @@ const (
        // R_ARM64_LDST64 sets a LD/ST immediate value to bits [11:3] of a local address.
        R_ARM64_LDST64

+       // R_ARM64_LDST128 sets a LD/ST immediate value to bits [11:4] of a local address.
+       R_ARM64_LDST128
+
        // PPC64.

 // R_POWER_TLS_LE is used to implement the "local exec" model for tls

$ cd cmd/internal/objabi/
$ stringer -type RelocType
$ git diff

diff --git a/src/cmd/internal/objabi/reloctype.go b/src/cmd/internal/objabi/reloctype.go
index a28b48ba50..5dc9356fe1 100644
--- a/src/cmd/internal/objabi/reloctype.go
+++ b/src/cmd/internal/objabi/reloctype.go
@@ -149,6 +149,9 @@ const (
        // R_ARM64_LDST64 sets a LD/ST immediate value to bits [11:3] of a local address.
        R_ARM64_LDST64

+       // R_ARM64_LDST128 sets a LD/ST immediate value to bits [11:4] of a local address.
+       R_ARM64_LDST128
+
        // PPC64.

// R_POWER_TLS_LE is used to implement the "local exec" model for tls
diff --git a/src/cmd/internal/objabi/reloctype_string.go b/src/cmd/internal/objabi/reloctype_string.go
index bd6d285617..6a5bdea1ab 100644
--- a/src/cmd/internal/objabi/reloctype_string.go
+++ b/src/cmd/internal/objabi/reloctype_string.go
@@ -4,6 +4,63 @@ package objabi

 import "strconv"

+func _() {
+       // An "invalid array index" compiler error signifies that the constant values have changed.
+       // Re-run the stringer command to generate them again.
+       var x [1]struct{}
+       _ = x[R_ADDR-1]
+       _ = x[R_ADDRPOWER-2]
+       _ = x[R_ADDRARM64-3]
+       _ = x[R_ADDRMIPS-4]
+       _ = x[R_ADDROFF-5]
+       _ = x[R_WEAKADDROFF-6]
+       _ = x[R_SIZE-7]
+       _ = x[R_CALL-8]
+       _ = x[R_CALLARM-9]
+       _ = x[R_CALLARM64-10]
+       _ = x[R_CALLIND-11]
+       _ = x[R_CALLPOWER-12]
+       _ = x[R_CALLMIPS-13]
+       _ = x[R_CONST-14]
+       _ = x[R_PCREL-15]
+       _ = x[R_TLS_LE-16]
+       _ = x[R_TLS_IE-17]
+       _ = x[R_GOTOFF-18]
+       _ = x[R_PLT0-19]
+       _ = x[R_PLT1-20]
+       _ = x[R_PLT2-21]
+       _ = x[R_USEFIELD-22]
+       _ = x[R_USETYPE-23]
+       _ = x[R_METHODOFF-24]
+       _ = x[R_POWER_TOC-25]
+       _ = x[R_GOTPCREL-26]
+       _ = x[R_JMPMIPS-27]
+       _ = x[R_DWARFSECREF-28]
+       _ = x[R_DWARFFILEREF-29]
+       _ = x[R_ARM64_TLS_LE-30]
+       _ = x[R_ARM64_TLS_IE-31]
+       _ = x[R_ARM64_GOTPCREL-32]
+       _ = x[R_ARM64_GOT-33]
+       _ = x[R_ARM64_PCREL-34]
+       _ = x[R_ARM64_LDST8-35]
+       _ = x[R_ARM64_LDST32-36]
+       _ = x[R_ARM64_LDST64-37]
+       _ = x[R_POWER_TLS_LE-38]
+       _ = x[R_POWER_TLS_IE-39]
+       _ = x[R_POWER_TLS-40]
+       _ = x[R_ADDRPOWER_DS-41]
+       _ = x[R_ADDRPOWER_GOT-42]
+       _ = x[R_ADDRPOWER_PCREL-43]
+       _ = x[R_ADDRPOWER_TOCREL-44]
+       _ = x[R_ADDRPOWER_TOCREL_DS-45]
+       _ = x[R_PCRELDBL-46]
+       _ = x[R_ADDRMIPSU-47]
+       _ = x[R_ADDRMIPSTLS-48]
+       _ = x[R_ADDRCUOFF-49]
+       _ = x[R_WASMIMPORT-50]
+       _ = x[R_XCOFFREF-51]
+}
+
 const _RelocType_name = "R_ADDRR_ADDRPOWERR_ADDRARM64R_ADDRMIPSR_ADDROFFR_WEAKADDROFFR_SIZER_CALLR_CALLARMR_CALLARM64R_CALLINDR_CALLPOWERR_CALLMIPSR_CONSTR_PCRELR_TLS_LER_TLS_IER_GOTOFFR_PLT0R_PLT1R_PLT2R_USEFIELDR_USETYPER_METHODOFFR_POWER_TOCR_GOTPCRELR_JMPMIPSR_DWARFSECREFR_DWARFFILEREFR_ARM64_TLS_LER_ARM64_TLS_IER_ARM64_GOTPCRELR_ARM64_GOTR_ARM64_PCRELR_ARM64_LDST8R_ARM64_LDST32R_ARM64_LDST64R_POWER_TLS_LER_POWER_TLS_IER_POWER_TLSR_ADDRPOWER_DSR_ADDRPOWER_GOTR_ADDRPOWER_PCRELR_ADDRPOWER_TOCRELR_ADDRPOWER_TOCREL_DSR_PCRELDBLR_ADDRMIPSUR_ADDRMIPSTLSR_ADDRCUOFFR_WASMIMPORTR_XCOFFREF"

 var _RelocType_index = [...]uint16{0, 6, 17, 28, 38, 47, 60, 66, 72, 81, 92, 101, 112, 122, 129, 136, 144, 152, 160, 166, 172, 178, 188, 197, 208, 219, 229, 238, 251, 265, 279, 293, 309, 320, 333, 346, 360, 374, 388, 402, 413, 427, 442, 459, 477, 498, 508, 519, 532, 543, 555, 565}

$ cd -
$ ./make.bash

PS: the stringer is the latest version

What did you expect to see?

Go was built and installed successfully.

What did you see instead?

Building Go cmd/dist using /home/erifan01/go-master.
Building Go toolchain1 using /home/erifan01/go-master.
# bootstrap/cmd/internal/objabi
/home/erifan01/go-for-ldst128/src/cmd/internal/objabi/reloctype_string.go:48: invalid array index R_POWER_TLS_LE - 38 (out of bounds for 1-element array)
/home/erifan01/go-for-ldst128/src/cmd/internal/objabi/reloctype_string.go:49: invalid array index R_POWER_TLS_IE - 39 (out of bounds for 1-element array)
/home/erifan01/go-for-ldst128/src/cmd/internal/objabi/reloctype_string.go:50: invalid array index R_POWER_TLS - 40 (out of bounds for 1-element array)
/home/erifan01/go-for-ldst128/src/cmd/internal/objabi/reloctype_string.go:51: invalid array index R_ADDRPOWER_DS - 41 (out of bounds for 1-element array)
/home/erifan01/go-for-ldst128/src/cmd/internal/objabi/reloctype_string.go:52: invalid array index R_ADDRPOWER_GOT - 42 (out of bounds for 1-element array)
/home/erifan01/go-for-ldst128/src/cmd/internal/objabi/reloctype_string.go:53: invalid array index R_ADDRPOWER_PCREL - 43 (out of bounds for 1-element array)
/home/erifan01/go-for-ldst128/src/cmd/internal/objabi/reloctype_string.go:54: invalid array index R_ADDRPOWER_TOCREL - 44 (out of bounds for 1-element array)
/home/erifan01/go-for-ldst128/src/cmd/internal/objabi/reloctype_string.go:55: invalid array index R_ADDRPOWER_TOCREL_DS - 45 (out of bounds for 1-element array)
/home/erifan01/go-for-ldst128/src/cmd/internal/objabi/reloctype_string.go:56: invalid array index R_PCRELDBL - 46 (out of bounds for 1-element array)
/home/erifan01/go-for-ldst128/src/cmd/internal/objabi/reloctype_string.go:57: invalid array index R_ADDRMIPSU - 47 (out of bounds for 1-element array)
/home/erifan01/go-for-ldst128/src/cmd/internal/objabi/reloctype_string.go:57: too many errors
go tool dist: FAILED: /home/erifan01/go-master/bin/go install -gcflags=-l -tags=math_big_pure_go compiler_bootstrap bootstrap/cmd/...: exit status 2

[erifan]

I haven't yet tracked down the failure, but it's clear that function "_()" was not introduced by my change. In addition, the newly added relocation type R_ARM64_LDST128 has not been correctly identified.

[ianlancetaylor]

As the comment says,"Re-run the stringer command to generate them again." When you introduce a new reloc you need to run go generate to regenerate the strings (and you may need to run `go install golang.org/x/tools/cmd/stringer to make the stringer program is available).

[erifan]

Hi @ianlancetaylor I installed Stringer and re-run the stringer command to generate the strings.

[erifan]

Sorry I made a stupid mistake, I didn't use a lower version of go to build Go.

[erifan]

I don't know if I got something wrong, but the problem still exists here. What I did:
1, install stringer with go1.12
2, git clone go master tip (do nothing) and build the new go with go1.12. Success!
3, add a reloc in the newgo/src/cmd/internal/objabi/reloctype.go
4, run command "stringer -type RelocType" under newgo/src/cmd/internal/objabi/
5, rebuild the new go with go1.12. Failed and the error is the same as above.

[ianlancetaylor]

I advise running go generate cmd/internal/objabi rather than running stringer yourself. Then make sure that "reloctype_string.go" has changed in your repo.

[erifan]

It still doesn't work.
If I execute go generate cmd/internal/objabi with go1.12, "reloctype_string.go" won't change, this is obviously wrong although the new Go can be built successfully.
If I execute ../bin/go generate cmd/internal/objabi with the newly built Go, "reloctype_string.go" and "symkind_string.go" hava some changes, but the new Go can't be rebuilt successfully, the error is the same as above.
Even if I don't add a new reloc type (without making any changes) and then execute ../bin/go generate cmd / internal/objabi, "reloctype_string.go" and "symkind_string.go" will still be modified, then the same error will occur when we rebuild Go. This situation also exists on x86.

[ianlancetaylor]

OK, my instructions were probably bad. When you run go generate cmd/internal/objabi with Go 1.12, that will run the generator in the Go 1.12 sources, not in the sources that you are using.

But running ../bin/go generate ought to work, and if the files are changing then something happened. What changed?

By the way I think this change is being made in https://golang.org/cl/176697 anyhow, perhaps looking at the CL will suggest something about what is happening with your code.

[erifan]

https://golang.org/cl/176697 is right, the changes in reloctype.go and reloctype_string.go are what we expect. I don't know which version of stringer he used.

In a clean Go project, running ../bin/go generate cmd/internal/objabi produces the following changes:

$ git diff
diff --git a/src/cmd/internal/objabi/reloctype_string.go b/src/cmd/internal/objabi/reloctype_string.go
index bd6d285617..3c169ffeea 100644
--- a/src/cmd/internal/objabi/reloctype_string.go
+++ b/src/cmd/internal/objabi/reloctype_string.go
@@ -1,12 +1,63 @@
-// Code generated by "stringer -type RelocType"; DO NOT EDIT.
+// Code generated by "stringer -type=RelocType"; DO NOT EDIT.

 package objabi

 import "strconv"

-const _RelocType_name = "R_ADDRR_ADDRPOWERR_ADDRARM64R_ADDRMIPSR_ADDROFFR_WEAKADDROFFR_SIZER_CALLR_CALLARMR_CALLARM64R_CALL                               INDR_CALLPOWERR_CALLMIPSR_CONSTR_PCRELR_TLS_LER_TLS_IER_GOTOFFR_PLT0R_PLT1R_PLT2R_USEFIELDR_USETYPER_METHODOFFR_POWER_TOCR_G                               OTPCRELR_JMPMIPSR_DWARFSECREFR_DWARFFILEREFR_ARM64_TLS_LER_ARM64_TLS_IER_ARM64_GOTPCRELR_ARM64_GOTR_ARM64_PCRELR_ARM64_LDST8                               R_ARM64_LDST32R_ARM64_LDST64R_POWER_TLS_LER_POWER_TLS_IER_POWER_TLSR_ADDRPOWER_DSR_ADDRPOWER_GOTR_ADDRPOWER_PCRELR_ADDRPOWER                               _TOCRELR_ADDRPOWER_TOCREL_DSR_PCRELDBLR_ADDRMIPSUR_ADDRMIPSTLSR_ADDRCUOFFR_WASMIMPORTR_XCOFFREF"
+func _() {
+       // An "invalid array index" compiler error signifies that the constant values have changed.
+       // Re-run the stringer command to generate them again.
+       var x [1]struct{}
+       _ = x[R_ADDR-1]
+       _ = x[R_ADDRPOWER-2]
+       _ = x[R_ADDRARM64-3]
+       _ = x[R_ADDRMIPS-4]
+       _ = x[R_ADDROFF-5]
+       _ = x[R_WEAKADDROFF-6]
+       _ = x[R_SIZE-7]
+       _ = x[R_CALL-8]
+       _ = x[R_CALLARM-9]
+       _ = x[R_CALLARM64-10]
+       _ = x[R_CALLIND-11]
+       _ = x[R_CALLPOWER-12]
+       _ = x[R_CALLMIPS-13]
+       _ = x[R_CONST-14]
+       _ = x[R_PCREL-15]
+       _ = x[R_TLS_LE-16]
+       _ = x[R_TLS_IE-17]
+       _ = x[R_GOTOFF-18]
+       _ = x[R_PLT0-19]
+       _ = x[R_PLT1-20]
+       _ = x[R_PLT2-21]
+       _ = x[R_USEFIELD-22]
+       _ = x[R_USETYPE-23]
+       _ = x[R_METHODOFF-24]
+       _ = x[R_POWER_TOC-25]
+       _ = x[R_GOTPCREL-26]
+       _ = x[R_JMPMIPS-27]
+       _ = x[R_DWARFSECREF-28]
+       _ = x[R_DWARFFILEREF-29]
+       _ = x[R_ARM64_TLS_LE-30]
+       _ = x[R_ARM64_TLS_IE-31]
+       _ = x[R_ARM64_GOTPCREL-32]
+       _ = x[R_POWER_TLS_LE-33]
+       _ = x[R_POWER_TLS_IE-34]
+       _ = x[R_POWER_TLS-35]
+       _ = x[R_ADDRPOWER_DS-36]
+       _ = x[R_ADDRPOWER_GOT-37]
+       _ = x[R_ADDRPOWER_PCREL-38]
+       _ = x[R_ADDRPOWER_TOCREL-39]
+       _ = x[R_ADDRPOWER_TOCREL_DS-40]
+       _ = x[R_PCRELDBL-41]
+       _ = x[R_ADDRMIPSU-42]
+       _ = x[R_ADDRMIPSTLS-43]
+       _ = x[R_ADDRCUOFF-44]
+       _ = x[R_WASMIMPORT-45]
+}
+
+const _RelocType_name = "R_ADDRR_ADDRPOWERR_ADDRARM64R_ADDRMIPSR_ADDROFFR_WEAKADDROFFR_SIZER_CALLR_CALLARMR_CALLARM64R_CALL                               INDR_CALLPOWERR_CALLMIPSR_CONSTR_PCRELR_TLS_LER_TLS_IER_GOTOFFR_PLT0R_PLT1R_PLT2R_USEFIELDR_USETYPER_METHODOFFR_POWER_TOCR_G                               OTPCRELR_JMPMIPSR_DWARFSECREFR_DWARFFILEREFR_ARM64_TLS_LER_ARM64_TLS_IER_ARM64_GOTPCRELR_POWER_TLS_LER_POWER_TLS_IER_POWER_T                               LSR_ADDRPOWER_DSR_ADDRPOWER_GOTR_ADDRPOWER_PCRELR_ADDRPOWER_TOCRELR_ADDRPOWER_TOCREL_DSR_PCRELDBLR_ADDRMIPSUR_ADDRMIPSTLSR_A                               DDRCUOFFR_WASMIMPORT"

-var _RelocType_index = [...]uint16{0, 6, 17, 28, 38, 47, 60, 66, 72, 81, 92, 101, 112, 122, 129, 136, 144, 152, 160, 166, 1                               72, 178, 188, 197, 208, 219, 229, 238, 251, 265, 279, 293, 309, 320, 333, 346, 360, 374, 388, 402, 413, 427, 442, 459, 477,                                498, 508, 519, 532, 543, 555, 565}
+var _RelocType_index = [...]uint16{0, 6, 17, 28, 38, 47, 60, 66, 72, 81, 92, 101, 112, 122, 129, 136, 144, 152, 160, 166, 1                               72, 178, 188, 197, 208, 219, 229, 238, 251, 265, 279, 293, 309, 323, 337, 348, 362, 377, 394, 412, 433, 443, 454, 467, 478,                                490}

 func (i RelocType) String() string {
        i -= 1
diff --git a/src/cmd/internal/objabi/symkind_string.go b/src/cmd/internal/objabi/symkind_string.go
index 2b9a9080e8..399f9b09dc 100644
--- a/src/cmd/internal/objabi/symkind_string.go
+++ b/src/cmd/internal/objabi/symkind_string.go
@@ -4,6 +4,25 @@ package objabi

 import "strconv"

+func _() {
+       // An "invalid array index" compiler error signifies that the constant values have changed.
+       // Re-run the stringer command to generate them again.
+       var x [1]struct{}
+       _ = x[Sxxx-0]
+       _ = x[STEXT-1]
+       _ = x[SRODATA-2]
+       _ = x[SNOPTRDATA-3]
+       _ = x[SDATA-4]
+       _ = x[SBSS-5]
+       _ = x[SNOPTRBSS-6]
+       _ = x[STLSBSS-7]
+       _ = x[SDWARFINFO-8]
+       _ = x[SDWARFRANGE-9]
+       _ = x[SDWARFLOC-10]
+       _ = x[SDWARFMISC-11]
+       _ = x[SABIALIAS-12]
+}
+
 const _SymKind_name = "SxxxSTEXTSRODATASNOPTRDATASDATASBSSSNOPTRBSSSTLSBSSSDWARFINFOSDWARFRANGESDWARFLOCSDWARFMISCSABIALIAS                               "

 var _SymKind_index = [...]uint8{0, 4, 9, 16, 26, 31, 35, 44, 51, 61, 72, 81, 91, 100}

[ianlancetaylor]

There is something wrong with your diff. What happened to R_XCOFFREF? It should be defined in cmd/internal/objabi/reloctype.go, and it should show up in your generated strings.

[erifan]

Oh, I found the cause. If we are going to use stringer in go-xxx, we should export the root directory of go-xxx as GOROOT and install stringer with go-xxx. The right steps are as follow:

1, git clone <go-url> /tmp/go
2, cd /tmp/go/src
# build Go
3, ./make.bash
4, export GOROOT=/tmp/go
5, export GOPATH=/tmp/gopath
# Install stringer with the Go just built.
6, /tmp/go/bin/go get -u golang.org/x/tools/... 
# Do Some Changes here ... 
# run /tmp/go generate
7, /tmp/go/bin/go generate cmd/internal/objabi/

Then we can rebuild /tmp/go successfully. But even if I did nothing, reloctype_string.go and symkind_string.go would still be modified. I don't know if someone changed some code but didn't run "go generate". In addition, this will lead to a lot of file update in vendor/golang.org/x. I think we should update these vendor files.

Thank you for your patient guidance. @ianlancetaylor