Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reflect: can set map elem with string key of a different string type #52379

Closed
kortschak opened this issue Apr 16, 2022 · 7 comments
Closed

reflect: can set map elem with string key of a different string type #52379

kortschak opened this issue Apr 16, 2022 · 7 comments
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. release-blocker
Milestone

Comments

@kortschak
Copy link
Contributor

kortschak commented Apr 16, 2022

Original report here.

What version of Go are you using (go version)?

$ go version
go version go1.18.1 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN="/home/user/bin"
GOCACHE="/home/user/.cache/go-build"
GOENV="/home/user/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/user/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/user"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/user/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/user/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.18.1"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/user/go/src/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2354676463=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Run https://go.dev/play/p/b7kheLRQpLL on go1.18

What did you expect to see?

Something like this

panic: reflect.Value.SetMapIndex: value of type string is not assignable to type main.CustomString

goroutine 1 [running]:
reflect.Value.assignTo({0x4701c0, 0xc000010050, 0x7f34d3c33108}, {0x47f0b5, 0x19}, 0x46fd40, 0x0)
	/usr/local/go-faketime/src/reflect/value.go:2810 +0x2ad
reflect.Value.SetMapIndex({0x472460, 0xc000012120, 0x0}, {0x4701c0, 0xc000010050, 0x98}, {0x46fb40, 0x494e88, 0x82})
	/usr/local/go-faketime/src/reflect/value.go:2045 +0xf6
main.main()
	/tmp/sandbox3220972237/prog.go:9 +0x218

Program exited.

What did you see instead?

Program exited.

Additional information

The same behaviour not-surprisingly exists for map access (https://go.dev/play/p/3KtoQGNVvgR).

Bisected to

23832ba2e2fb396cda1dacf3e8afcb38ec36dcba is the first bad commit
commit 23832ba2e2fb396cda1dacf3e8afcb38ec36dcba
Author: Joe Tsai <joetsai@digital-static.net>
Date:   Thu Aug 26 19:13:22 2021 -0700

    reflect: optimize for maps with string keys
    
    Over 80% of all Go map types use a string as the key.
    The Go runtime already has a specialized implementation for such maps
    in runtime/map_faststr.go. However, the Go reflection implementation
    has not historically made use of that implementation.
    
    This CL plumbs the appropriate logic to be accessible from Go reflection
    so that it can benefit as well.
    
        name                            old time/op    new time/op    delta
        Map/StringKeys/MapIndex-4       4.65us ± 5%    2.95us ± 3%  -36.50%  (p=0.016 n=4+5)
        Map/StringKeys/SetMapIndex-4    7.47us ± 5%    5.27us ± 2%  -29.40%  (p=0.008 n=5+5)
        Map/Uint64Keys/MapIndex-4       3.79us ± 3%    3.75us ± 2%     ~     (p=0.548 n=5+5)
        Map/Uint64Keys/SetMapIndex-4    6.13us ± 3%    6.09us ± 1%     ~     (p=0.746 n=5+5)
    
    Change-Id: I5495d48948d8caf2d004a03ae1820ab5f8729670
    Reviewed-on: https://go-review.googlesource.com/c/go/+/345486
    Trust: Joe Tsai <joetsai@digital-static.net>
    Run-TryBot: Joe Tsai <joetsai@digital-static.net>
    TryBot-Result: Go Bot <gobot@golang.org>
    Reviewed-by: Keith Randall <khr@golang.org>

:040000 040000 bcb02e8bffb89cc2119b282573cc327ac59a8b34 ef1bf8cdefa674b136a3360601c8985b915b4582 M	src

It looks like this change does not include a check for assignability.

@kortschak
Copy link
Contributor Author

kortschak commented Apr 16, 2022

Replacing the kind equality with an assignment check doesn't completely wipe out the gains from the previous change.

diff --git a/src/reflect/all_test.go b/src/reflect/all_test.go
index 5a35d98b51..87caca3300 100644
--- a/src/reflect/all_test.go
+++ b/src/reflect/all_test.go
@@ -1496,6 +1496,10 @@ func TestMap(t *testing.T) {
        if m != nil {
                t.Errorf("mv.Set(nil) failed")
        }
+
+       type S string
+       shouldPanic("not assignable", func() { mv.MapIndex(ValueOf(S("key"))) })
+       shouldPanic("not assignable", func() { mv.SetMapIndex(ValueOf(S("key")), ValueOf(0)) })
 }
 
 func TestNilMap(t *testing.T) {
diff --git a/src/reflect/value.go b/src/reflect/value.go
index 2496cbe463..5608bcd7ad 100644
--- a/src/reflect/value.go
+++ b/src/reflect/value.go
@@ -1616,7 +1616,7 @@ func (v Value) MapIndex(key Value) Value {
        // of unexported fields.
 
        var e unsafe.Pointer
-       if key.kind() == String && tt.key.Kind() == String && tt.elem.size <= maxValSize {
+       if key.kind() == String && directlyAssignable(key.typ, tt.key) && tt.elem.size <= maxValSize {
                k := *(*string)(key.ptr)
                e = mapaccess_faststr(v.typ, v.pointer(), k)
        } else {
@@ -2232,7 +2232,7 @@ func (v Value) SetMapIndex(key, elem Value) {
        key.mustBeExported()
        tt := (*mapType)(unsafe.Pointer(v.typ))
 
-       if key.kind() == String && tt.key.Kind() == String && tt.elem.size <= maxValSize {
+       if key.kind() == String && directlyAssignable(key.typ, tt.key) && tt.elem.size <= maxValSize {
                k := *(*string)(key.ptr)
                if elem.typ == nil {
                        mapdelete_faststr(v.typ, v.pointer(), k)
name                          old time/op    new time/op    delta
Map/StringKeys/MapIndex-8       3.42µs ± 1%    3.95µs ± 2%  +15.53%  (p=0.000 n=9+10)
Map/StringKeys/SetMapIndex-8    6.59µs ± 1%    6.82µs ± 1%   +3.49%  (p=0.000 n=10+10)
Map/Uint64Keys/MapIndex-8       4.71µs ± 2%    4.49µs ± 2%   -4.84%  (p=0.000 n=9+10)
Map/Uint64Keys/SetMapIndex-8    7.51µs ± 2%    7.34µs ± 1%   -2.23%  (p=0.000 n=10+10)
MapIterNext-8                    141ns ± 2%     140ns ± 1%   -1.15%  (p=0.008 n=10+9)

I'm happy to send this change unless there are better options.

@gopherbot
Copy link

Change https://go.dev/cl/400635 mentions this issue: reflect: ensure map keys match key type in MapIndex and SetMapIndex

@ianlancetaylor
Copy link
Contributor

@gopherbot Please open backport to 1.18.

The reflect package incorrectly accepts this case. This should be fixed so that we don't have programs that work with the current release and fail with the next release.

Earlier releases correctly panic.

@gopherbot
Copy link

Backport issue(s) opened: #52386 (for 1.18).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

@ianlancetaylor ianlancetaylor added this to the Go1.19 milestone Apr 16, 2022
@aclements
Copy link
Member

@kortschak , would you mind preparing a back-port CL to apply to the release-branch.go.1.18 branch?

@kortschak
Copy link
Contributor Author

No worries. Happy to do that.

@gopherbot
Copy link

Change https://go.dev/cl/404000 mentions this issue: reflect: ensure map keys match key type in MapIndex and SetMapIndex

gopherbot pushed a commit that referenced this issue May 9, 2022
…pIndex and SetMapIndex

name                          old time/op    new time/op    delta
Map/StringKeys/MapIndex-8           2.36µs ± 5%    2.55µs ±11%  +7.98%  (p=0.006 n=10+9)
Map/StringKeys/SetMapIndex-8        4.86µs ± 7%    4.77µs ± 1%    ~     (p=0.211 n=10+9)
Map/StringKindKeys/MapIndex-8       2.29µs ± 3%    2.28µs ± 4%    ~     (p=0.631 n=10+10)
Map/StringKindKeys/SetMapIndex-8    4.44µs ± 3%    4.61µs ± 1%  +3.78%  (p=0.000 n=10+10)
Map/Uint64Keys/MapIndex-8           3.42µs ± 9%    3.11µs ± 2%  -9.20%  (p=0.000 n=10+9)
Map/Uint64Keys/SetMapIndex-8        5.17µs ± 3%    5.00µs ± 1%  -3.23%  (p=0.000 n=9+10)

For #52379
Fixes #52386

Change-Id: I545c71ea3145280828ca4186aad036a6c02016ed
Reviewed-on: https://go-review.googlesource.com/c/go/+/400635
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
(cherry picked from commit 11a650b)
Reviewed-on: https://go-review.googlesource.com/c/go/+/404000
Reviewed-by: David Chase <drchase@google.com>
@golang golang locked and limited conversation to collaborators May 4, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. release-blocker
Projects
None yet
Development

No branches or pull requests

4 participants