Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go2go: decide what unsafe.Sizeof/Alignof mean when applied to values of type parameter type #40301

Open
AndrewWPhillips opened this issue Jul 20, 2020 · 17 comments
Assignees
Labels
Milestone

Comments

@AndrewWPhillips
Copy link

@AndrewWPhillips AndrewWPhillips commented Jul 20, 2020

What version of Go are you using (go version)?

$ go version
go version devel +893c5ec17b Thu Jul 16 21:30:46 2020 +0000 windows/amd64

Does this issue reproduce with the latest release?

N/A

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\Andre\AppData\Local\go-build
set GOENV=C:\Users\Andre\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\Andre\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\Andre\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Users\Andre\goroot
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Users\Andre\goroot\pkg\tool\windows_amd64
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\Andrew\AppData\Local\Temp\go-build443956446=/tmp/go-build -gno-record-g
cc-switches

What did you do?

package main

import "unsafe"

type HasZero interface {
	type int, int8 // etc
}

func f(type T HasZero)() {
	const sz = unsafe.Sizeof(T(0))
}

$ go tool go2go build

What did you expect to see?

build succeeds (or perhaps build error message)

What did you see instead?

panic: Sizeof unimplemented for type sum [recovered]
panic: Sizeof unimplemented for type sum

goroutine 1 [running]:
go/types.(*Checker).handleBailout(0xc00009e3c0, 0xc0000cb9f8)
/path/goroot/src/go/types/check.go:252 +0xa5
panic(0xe21f40, 0xeb74e0)
/path/goroot/src/runtime/panic.go:969 +0x176
go/types.(*StdSizes).Sizeof(0xc0000a2280, 0xec3700, 0xc0000f0600, 0xc0000f06c0)
/path/goroot/src/go/types/sizes.go:152 +0x218
go/types.(*Config).sizeof(0xc0000f0500, 0xec3700, 0xc0000f0600, 0x0)
/path/goroot/src/go/types/sizes.go:258 +0xa2
go/types.(*Checker).builtin(0xc00009e3c0, 0xc0000f0680, 0xc0000f0440, 0x11, 0xc000020800)
/path/goroot/src/go/types/builtins.go:642 +0x3997
go/types.(*Checker).call(0xc00009e3c0, 0xc0000f0680, 0xc0000f0440, 0x30)
/path/goroot/src/go/types/call.go:63 +0xdb0
go/types.(*Checker).exprInternal(0xc00009e3c0, 0xc0000f0680, 0xebf680, 0xc0000f0440, 0x0, 0x0, 0x2b)
/path/goroot/src/go/types/expr.go:1603 +0x1df0
go/types.(*Checker).rawExpr(0xc00009e3c0, 0xc0000f0680, 0xebf680, 0xc0000f0440, 0x0, 0x0, 0x0)
/path/goroot/src/go/types/expr.go:1033 +0xc7
go/types.(*Checker).expr(0xc00009e3c0, 0xc0000f0680, 0xebf680, 0xc0000f0440)
/path/goroot/src/go/types/expr.go:1726 +0x5c
go/types.(*Checker).constDecl(0xc00009e3c0, 0xc0000d2480, 0x0, 0x0, 0xebf680, 0xc0000f0440)
/path/goroot/src/go/types/decl.go:419 +0x192
go/types.(*Checker).declStmt(0xc00009e3c0, 0xebfa00, 0xc0000f0480)
/path/goroot/src/go/types/decl.go:832 +0xe5
go/types.(*Checker).stmt(0xc00009e3c0, 0x0, 0xebf7c0, 0xc0000885d0)
/path/goroot/src/go/types/stmt.go:319 +0x3886
go/types.(*Checker).stmtList(0xc00009e3c0, 0x0, 0xc0000885e0, 0x1, 0x1)
/path/goroot/src/go/types/stmt.go:125 +0xd6
go/types.(*Checker).funcBody(0xc00009e3c0, 0xc0000d23c0, 0xfefe10, 0x1, 0xc0000d2420, 0xc0000b8e10, 0x0, 0x0)
/path/goroot/src/go/types/stmt.go:42 +0x268
go/types.(*Checker).funcDecl.func1()
/path/goroot/src/go/types/decl.go:792 +0x6e
go/types.(*Checker).processDelayed(0xc00009e3c0, 0x0)
/path/goroot/src/go/types/check.go:327 +0x45
go/types.(*Checker).checkFiles(0xc00009e3c0, 0xc0000cc0a0, 0x1, 0x1, 0x0, 0x0)
/path/goroot/src/go/types/check.go:295 +0x20d
go/types.(*Checker).Files(...)
/path/goroot/src/go/types/check.go:257
go/types.(*Config).Check(0xc0000f0500, 0xc0000a2528, 0x4, 0xc0000f0240, 0xc0000cc0a0, 0x1, 0x1, 0xc0000d5310, 0x1, 0x1, ...)
/path/goroot/src/go/types/api.go:387 +0x188
go/go2go.rewriteFilesInPath(0xc0000d2240, 0x0, 0x0, 0xe6fac3, 0x1, 0xc0000b8c60, 0x1, 0x3, 0x0, 0x0, ...)
/path/goroot/src/go/go2go/go2go.go:93 +0x4f0
go/go2go.rewriteToPkgs(0xc0000d2240, 0x0, 0x0, 0xe6fac3, 0x1, 0xc0000985c0, 0xc0000b8ab0, 0xc0000b8a80, 0xc0000b8a50, 0xc0000b8a20)
/path/goroot/src/go/go2go/go2go.go:46 +0x16e
go/go2go.Rewrite(...)
/path/goroot/src/go/go2go/go2go.go:30
main.translate(0xc0000d2240, 0xe6fac3, 0x1)
/path/goroot/src/cmd/go2go/translate.go:15 +0x4e
main.main()
/path/goroot/src/cmd/go2go/main.go:78 +0xa0c

@AndrewWPhillips
Copy link
Author

@AndrewWPhillips AndrewWPhillips commented Jul 20, 2020

BTW I think that having unsafe.Sizeof() work on parameter types is important. It worked (see code below) when I tried go2go a couple of months ago - using "contracts" for constraints.

$ go version
go version devel +af2b592260 Wed Apr 22 14:12:34 2020 -0700 Linux/amd64

contract (
	Element(T) {
		T int8, int16, int32, int64, int, uint8, uint16, uint32, uint64, uint //, uintptr
	}
)

func minInt(type T Element)() T {
	zero := T(0)
	if zero - 1 > 0 {
		return zero
	}
	// Handle signed types - may need fix if new types added to Go (like int128)
	switch unsafe.Sizeof(T(0)) {
	case 1:
		v := math.MinInt8
		return T(v)
	case 2:
		v := math.MinInt16
		return T(v)
	case 4:
		v := math.MinInt32
		return T(v)
	case 8:
		v := math.MinInt64
		return T(v)
	}
	panic("unknown type")
}
@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jul 23, 2020

CC @griesemer

Looks like a bug in the type checker. Perhaps unsafe.Sizeof just hasn't been implemented for type lists yet.

@griesemer
Copy link
Contributor

@griesemer griesemer commented Jul 23, 2020

@AndrewWPhillips It didn't "work" before - it just may not have crashed.

The question is: What does that unsafe.Sizeof(x) where x is of a type parameter type even mean? unsafe.Sizeof is a compile time constant, it must be determined when the function that contains it is compiled. I can see how we could make this work when the associated constraint contains a type list, but if there's more than one type in the type list (or they have different sizes) it's going to be very hard to maintain that "compile-time" constant aspect. We don't have any facility for something like this in the compiler at the moment, and it's not clear (to me) that we need it.

@griesemer
Copy link
Contributor

@griesemer griesemer commented Jul 24, 2020

I will "fix this" for now by reporting an error. We can revisit if we have a better idea of what the right approach is.

@gopherbot
Copy link

@gopherbot gopherbot commented Jul 24, 2020

Change https://golang.org/cl/244622 mentions this issue: [dev.go2go] go/types: don't crash with unsafe.Alignof/Sizeof on a type parameter value

@griesemer griesemer closed this Jul 24, 2020
gopherbot pushed a commit that referenced this issue Jul 24, 2020
…e parameter value

Report an error instead for now until we have a better idea.
(It's unclear what these operations should do as they are
defined to return a compile-time constant which we can't
know in general.)

Fixes #40301.

Change-Id: I22a991311de117bc00d52b67b4ce862ea09d855a
Reviewed-on: https://go-review.googlesource.com/c/go/+/244622
Reviewed-by: Robert Griesemer <gri@golang.org>
@griesemer
Copy link
Contributor

@griesemer griesemer commented Jul 24, 2020

@ianlancetaylor pointed out that one could define unsafe.Sizeof such that it would not return a compile-time constant if the argument is of type parameter type. That would certainly be trivial from a type-checker's point of view but would break an assumption we currently have for unsafe.Sizeof (but not permitting it would also break an assumption, so perhaps that's the way to go after all).

@griesemer
Copy link
Contributor

@griesemer griesemer commented Jul 24, 2020

Reopened and retitled for decision making.

@griesemer griesemer reopened this Jul 24, 2020
@griesemer griesemer changed the title cmd/go2go: panic taking size of parameter type cmd/go2go: decide what unsafe.Sizeof/Alignof mean when applied to values of type parameter type Jul 24, 2020
@AndrewWPhillips
Copy link
Author

@AndrewWPhillips AndrewWPhillips commented Jul 25, 2020

It didn't "work" before - it just may not have crashed.

@griesemer sorry to disagree but the code from my above comment (5 days ago) was tested and working.
I created a simpler example below which prints 0 1 2 8 16 24 using go2go from go version devel +af2b592260 Wed Apr 22 14:12:34 2020 -0700 linux/amd64

package main

import "unsafe"

func main() {
	f(struct{})()
	f(byte)()
	f(int16)()
	f(uint64)()
	f(string)()
	f(time.Time)()
}

func f(type T)() {
	var v T
	println(unsafe.Sizeof(v))
}
@griesemer
Copy link
Contributor

@griesemer griesemer commented Jul 25, 2020

@AndrewWPhillips Indeed! I apologize for the unqualified rushed judgement.

I've checked out that older version and looked at the type-checker and the generated code. It turns out the unsafe.Sizeof wasn't implemented for type of type parameter type (as suspected), but there was also no panic call to remind me that it was not yet implemented, and thus it didn't crash. Instead it used the fallback value, which happens to be the word size (== 8). So, while type-checking was not crashing, it definitively was incorrect. The translator portion of the go2go tool doesn't consider the fact that unsafe.Sizeof(T(0)) is deemed a constant value by the type-checker and always generates code to call unsafe.Sizeof(T(0)) rather than always producing the constant 8. Because the tool instantiates (== generates) the function f for each possible type, approximately resulting in a f_int, f_byte, and f_uint64 version of f, it happened to work after all. One might even say it worked because we hadn't implemented it yet...

Anyway, mystery solved. Thanks for pointing this out.

@AndrewWPhillips
Copy link
Author

@AndrewWPhillips AndrewWPhillips commented Jul 26, 2020

Thanks @griesemer for tracking that down. I hope my feedback was of some use and not a distraction from the great work you are doing!

BTW I had a strong feeling that unsafe.Sizeof on values of type parameter types was important. However, I have tried to find examples to prove my point to no avail, except for the above example of obtaining the smallest value of signed integer types. Their is probably a better way than my clumsy use of Sizeof; I would appreciate if anyone could provide help with that. [My initial thought was to shift ~0 (ie all bits on) one bit right - but for signed ints that just sign extends - if I could cast from an unsigned type to equiv. signed type I could do it that way.]

@AndrewWPhillips
Copy link
Author

@AndrewWPhillips AndrewWPhillips commented Jul 29, 2020

I updated the code in my comment of Jul 25 (w/o changing the meaning) to show that in go version devel +af2b592260 Wed Apr 22 14:12:34 2020 -0700 linux/amd64 unsafe.Sizeof seems to work with any "type" of type parameter type (ie no contract used).

Also I still haven't found a good way to work out the minimum value of any integer type that does not used unsafe.Sizeof. The code below does not build due to use of unsafe.Sizeof in the 2nd last line. (It builds and work perfectly in the abovementioned Apr 22 version of go2go using a "contract".)

type Element interface {
	type int, int8 // etc
}

var signedIntMin = map[uintptr]int64{
	1: math.MinInt8,
	2: math.MinInt16,
	4: math.MinInt32,
	8: math.MinInt64,
}

// minInt returns the smallest allowed integer for an element (signed/unsigned integer)
func minInt(type T Element)() T {
	zero := T(0)
	if zero - 1 > 0 {
		// unsigned integer min has all bits off (ie zero)
		return zero
	}
	return T(signedIntMin[unsafe.Sizeof(T(0))])
}
@griesemer
Copy link
Contributor

@griesemer griesemer commented Jul 29, 2020

This code implements minInt in two ways, both without using unsafe.

@AndrewWPhillips
Copy link
Author

@AndrewWPhillips AndrewWPhillips commented Jul 29, 2020

Thanks, just what I needed. BTW I believe the first solution is not portable (int can be 32 bits, can it not)?

@griesemer
Copy link
Contributor

@griesemer griesemer commented Jul 29, 2020

Correct. One could convert to a T and check if it's not 0, but then one might just as well choose the 2nd solution. There's a loop, but it iterates only 4x at most. It could be unrolled easily if speed mattered. It may be faster than the type switch.

@bcmills
Copy link
Member

@bcmills bcmills commented Jul 30, 2020

Just to add a data point: the change to report an error broke my implementation of unsafeslice.Convert (#38203) in https://github.com/bcmills/go2go/blob/6d205890ae92524c687a32d2f29f6d0fd24eb709/unsafeslice/unsafeslice.go2.

@bcmills
Copy link
Member

@bcmills bcmills commented Jul 30, 2020

(And I don't see a way to implement unsafeslice.Convert generically without either using unsafe.Sizeof or falling back to reflect.)

@AndrewWPhillips
Copy link
Author

@AndrewWPhillips AndrewWPhillips commented Aug 17, 2020

.. It could be unrolled easily if speed mattered. ...

@griesemer FYI I benchmarked your minInt funcs (thanks again) and the type switch one was almost twice as fast as the loop one. So I unrolled the loop and that was almost 20X faster, as we expected. [And the speed increase was irrespective of size of type parameter (8/16/32/64) so there must be some code elimination happening but I have not checked the generated code.]

See code here in generic "range set" container I created.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants
You can’t perform that action at this time.