Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: concatstring unexpected fault address #70154

Closed
aimuz opened this issue Nov 1, 2024 · 5 comments
Closed

runtime: concatstring unexpected fault address #70154

aimuz opened this issue Nov 1, 2024 · 5 comments
Labels
compiler/runtime Issues related to the Go compiler and/or runtime.

Comments

@aimuz
Copy link
Contributor

aimuz commented Nov 1, 2024

Go version

go version go1.23.2 darwin/arm64

Output of go env in your module/workspace:

GO111MODULE='on'
GOARCH='arm64'
GOBIN=''
GOCACHE='/Users/aimuz/Library/Caches/go-build'
GOENV='/Users/aimuz/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/aimuz/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/aimuz/go'
GOPRIVATE=''
GOPROXY='https://goproxy.cn,direct'
GOROOT='/opt/homebrew/opt/go/libexec'
GOSUMDB='sum.golang.google.cn'
GOTMPDIR=''
GOTOOLCHAIN='local'
GOTOOLDIR='/opt/homebrew/opt/go/libexec/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.23.2'
GODEBUG=''
GOTELEMETRY='local'
GOTELEMETRYDIR='/Users/aimuz/Library/Application Support/go/telemetry'
GCCGO='gccgo'
GOARM64='v8.0'
AR='ar'
CC='cc'
CXX='c++'
CGO_ENABLED='1'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/hj/5x_j6nwj56sdkyyxnznp2rb40000gn/T/go-build3393140508=/tmp/go-build -gno-record-gcc-switches -fno-common'

What did you do?

type imageService struct{}
func (i *imageService) GetImageURL(_ context.Context, image string) string {
	if strings.Compare(image, "dynamic_content_ban.jpg") == 0 {
		return image
	}
	image += "~tplv-kp7y9lmlrj-dynamic-m-thumb.image"
	return image
}

When I run a pressure test, I will get the following error,When I remove this line of code, everything will be fine image += "~tplv-kp7y9lmlrj-dynamic-m-thumb.image"

What did you see happen?

unexpected fault address 0x2246000
fatal error: fault
unexpected fault address 0x2246000
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x1 addr=0x2246000 pc=0x47e079]

goroutine 270969 gp=0xc00511ea80 m=12 mp=0xc000700008 [running]:
runtime.throw({0x1541d18?, 0xc000263108?})
	runtime/panic.go:1067 +0x48 fp=0xc002c8a588 sp=0xc002c8a558 pc=0x4749a8
runtime.sigpanic()
	runtime/signal_unix.go:914 +0x26c fp=0xc002c8a5e8 sp=0xc002c8a588 pc=0x4766ec
runtime.memmove()
	runtime/memmove_amd64.s:427 +0x4f9 fp=0xc002c8a5f0 sp=0xc002c8a5e8 pc=0x47e079
runtime.concatstrings(0x0?, {0xc002c8a680?, 0x2, 0x0?})
	runtime/string.go:53 +0x178 fp=0xc002c8a660 sp=0xc002c8a5f0 pc=0x458938
runtime.concatstring2(0x0?, {0x127aa00?, 0x0?}, {0x156a736?, 0xc002c8ab88?})
	runtime/string.go:60 +0x4a fp=0xc002c8a6b0 sp=0xc002c8a660 pc=0x458a2a

What did you expect to see?

lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
Address sizes:       40 bits physical, 57 bits virtual
CPU(s):              16
On-line CPU(s) list: 0-15
Thread(s) per core:  2
Core(s) per socket:  8
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               106
Model name:          Intel(R) Xeon(R) Platinum 8362 CPU @ 2.80GHz
Stepping:            6
CPU MHz:             2800.000
BogoMIPS:            5600.00
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           48K
L1i cache:           32K
L2 cache:            1280K
L3 cache:            49152K
NUMA node0 CPU(s):   0-15
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves wbnoinvd arat avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid md_clear arch_capabilities
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Nov 1, 2024
@seankhliao seankhliao changed the title runtime: map unexpected fault address runtime: concatstring unexpected fault address Nov 1, 2024
@zigo101
Copy link

zigo101 commented Nov 1, 2024

How is the image parameter passed? Is the argument created in an unsafe way?

@aimuz
Copy link
Contributor Author

aimuz commented Nov 2, 2024

In my code, there is no use of unsafe anywhere, and I'm trying to get a minimal reproduction of the code out there

@randall77
Copy link
Contributor

It definitely looks like it walked off into unmapped memory. The fault address is at the start of a page.

	VMOVDQU	(SI), Y0
	VMOVDQU	0x20(SI), Y1
	VMOVDQU	0x40(SI), Y2     <- didn't fault here!
	VMOVDQU	0x60(SI), Y3     <- fault is here

Presumably this is because the length of one of the input strings got corrupted somehow. The fault address looks like it walked past the end of the data section, not the end of the heap (which is at much higher addresses).
Corrupting the length of a constant string would be hard to do. It's a hardcoded constant in the instruction stream, and it lives in memory only very briefly in the slice passed from concatstring2 to concatstrings.
So probably it is the length of image that is corrupted (and it came originally from a constant string).

@aimuz
Copy link
Contributor Author

aimuz commented Nov 4, 2024

Thank you all, I think I should know the reason now. In my code, I used singleflight, which in some cases causes singleflight to return the same Content.Image.

The following is a sample code. If get() is called concurrently, different goroutines may receive different Content. Since Content is passed by value, it will perform a simple shallow copy, and at this point, Content.Image will all point to the same Images slice. If GetImageURL is simultaneously performed on Images, there is a chance that this issue will occur.

After I added a deep copy method, the problem no longer occurs. Thank you all

type Content struct {
	ID     uint32
	Images []string
}

func get() Content {
   singleflight.Do("key", func() (any,error){
     return Content{
        Images: []string{
            "https://example.com/image1.jpg",
        }
     }
   })
}

@aimuz aimuz closed this as completed Nov 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/runtime Issues related to the Go compiler and/or runtime.
Projects
None yet
Development

No branches or pull requests

5 participants