Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
proposal: Go 2: spec: add integer types with explicit overflow behavior, and remove unchecked operations on built-in integers #30209
This proposal is intended to accomplish the same goals as #19624, but in a way that complies with the migration strategy outlined in https://github.com/golang/proposal/blob/master/design/28221-go2-transitions.md#language-redefinitions.
See #19624 for background.
The three new packages each provide a set of integer types:
// Package checked defines integer types whose arithmetic operations and conversions // panic on overflow. // Bitwise operations and logical shifts will not trigger a panic. package checked type Uint8 <builtin> type Uint16 <builtin> […] type Int8 <builtin> […] type Byte = Uint8 type Rune <builtin> type Int <builtin> type Uint <builtin> type Uintptr <builtin>
// Package wrapped defines integer types whose arithmetic operations and conversions wrap // using two's-complement arithmetic. package wrapped type Uint8 <builtin> […] type Int8 <builtin> […] type Byte = Uint8 type Rune <builtin> type Int <builtin> type Uint <builtin> type Uintptr <builtin>
// Package unbounded defines an arbitrary-precision integer type with unbounded range. // Unbounded types do not support bitwise XOR, complement, or clear operations. package unbounded type Int <builtin>
Defined types that have
Defined types that have builtin integer types as their underlying type have the same behavior as the underlying type, except that they are not mutually-assignable with
A checked assignment uses the form
Bitwise operations and logical shifts do not overflow, and therefore do not set the
Unchecked arithmetic operations on builtin integer types are a compile-time error.
var x int32 = 1<<31 - 1 y := x + 1 // compile-time error: `x + 1` is not checked for overflow
var x int32 = 1<<31 - 1 y, ok := x + 1 // y = -2147483648; ok = false
var x checked.Int32 = 1<<31 - 1 y := x + 1 // run-time panic: `x + 1` overflows
var x checked.Int32 = 1<<31 - 1 y, ok := x + 1 // y = -2147483648; ok = false
var x wrapped.Int32 = 1<<31 - 1 y := x + 1 // y = -2147483648
var x wrapped.Int32 = 1<<31 - 1 y, ok := x + 1 // y = -2147483648; ok = false
var x int32 = 1<<30 y, ok := x<<1 // y = -2147483648; ok = false // signed shift is arithmetic, and shifting into the sign bit overflows
var x uint32 = 1<<31 y, ok := x<<1 // y = 0; ok = true // unsigned shift is logical, and by definition cannot overflow
Any integer type can be explicitly converted to a
var x checked.Int32 = 1<<31 - 1 var y = int32(x) // y = 2147483647
var x checked.Uint32 = 1<<31 - 1 var y = int32(x) // compile-time error: conversion from checked.Uint32 may overflow int32
var x checked.Uint32 = 1<<31 y, _ = int32(x) // y = -2147483648
An unchecked conversion to a
var x int64 = 1<<31 y := checked.Int32(x) // run-time panic: `x` overflows checked.Int32
var x int64 = 1<<31 y, ok := checked.Int32(x) // y = -2147483648; ok = false
A conversion to a
var x checked.Int64 = 1<<31 y := wrapped.Int32(x) // y = -2147483648
A conversion to
var x int64 = 1<<31 y := unbounded.Int(x) // y = 2147483648
var x checked.Int32 = 1<<31-1 y := unbounded.Int(x+1) // run-time panic: `x+1` overflows checked.Int32
var x wrapped.Int32 = 1<<31-1 y := unbounded.Int(x+1) // y = -2147483648
var x checked.Int32 = 1<<31-1 y, ok := unbounded.Int(x+1) // y = -2147483648; ok = false
Each sized type in the
var x wrapped.Int32 = 1<<31 - 1 var y int32 […] y = x + 1 // y = -2147483648
var x checked.Int32 = 1<<31-1 var y wrapped.Int32 […] y = x + 1 // compile-time error: checked.Int32 is not assignable to wrapped.Int32
type MyInt32 wrapped.Int32 var x wrapped.Int32 = 1<<31 - 1 var y MyInt32 […] y = x + 1 // compile-time error: wrapped.Int32 is not assignable to MyInt32
type MyInt32 int32 var x wrapped.Int32 = 1<<31 - 1 var y MyInt32 […] y = x + 1 // compile-time error: wrapped.Int32 is not assignable to MyInt32
var f func() int32 var g func(int32) var x checked.Int32 x = f() g(x + 1) // ok: x+1 is checked, then passed to g as an int32
This allows functions to perform operations on
The type of an arithmetic expression depends on its operands. If both operands are of
If both operands are of a builtin integer type or a defined type with a builtin integer as its underlying type, the expression must be a checked assignment, bitwise operator, or logical shift, and its result is the same type as the operands.
An untyped integer constant can be assigned to any
Unfortunately, in order to comply with https://github.com/golang/proposal/blob/master/design/28221-go2-transitions.md#language-changes, the inferred type of a variable initialized from an untyped constant must remain the built-in
changed the title
proposal: spec: add integer types with explicit overflow behavior and remove unchecked operations on built-in integers
Feb 13, 2019
Note that part (1) would be trivial if we had operator overloading, and part (3) could really just be a
The assignability rule could perhaps be eliminated from the proposal without causing too much damage. That would make the code much more verbose, but not fundamentally alter its structure.
Similarly, the conversion check might be implementable using generic constructor functions, and can be implemented (inefficiently and without type-safety) using reflection.
IMO, the interesting part of this proposal is part (2), the aggregation of overflow checks to the whole expression. That would otherwise require constructing an overflow-tracking object, using its methods in place of the existing arithmetic expressions, and remembering to check its result at the end — a lot of syntax, and much more error-prone than a regular old
@ericlagergren An unbounded signed integer type can represent all of the unsigned values too, and is closed under subtraction.
In contrast, an unsigned type carries the risk of overflow: what would the behavior be for
I’ll admit I haven’t read this too carefully, but I hope you’ll permit me some questions anyway:
Indeed. I don't feel strongly about that either way — I'm happy to go with whatever makes the proposal more palatable to the folks making the final decision.
I'm sorry I haven't read everything carefully but how exactly is this backwards compatible?
EDIT: I guess it's not intended to be? Is there any idea of how much existing code this would break?
If I'm reading this correctly essentially every existing Go program would become invalid, because they all have arithmetic operations on builtin integer types. That seems like a very heavy lift.
I'd prefer to see plain
Even in the latter case, I feel like this proposal would make bounded arithmetic code overly verbose.
As an aside, why disallow XOR and complement on unbounded ints? Assuming they're signed, aren't both operations well-defined and guaranteed to yield a finite result?
Probably every program, but not every package: in particular, packages that use
Also note that it would be possible to adopt parts (1) and (2) of this proposal — the new packages and checked assignment — without the breaking change of part (3). That would provide a much smaller benefit, but at a much smaller corresponding cost.
Note that that proposal does not comply with the migration strategy outlined in https://github.com/golang/proposal/blob/master/design/28221-go2-transitions.md#language-redefinitions.
Interesting question. Mostly I think it's weird to apply two's-complement arithmetic when the sign bit is infinitely far away, but I suppose it is still well-defined. Perhaps they would be ok after all.
I am aginst part 3 of the proposal, since that would take us too far.
// Package checked defines integer types whose arithmetic operations and conversions // panic on overflow. package checked type Uint8 = range uint8[0:255] type Uint16 = range uint16[0:65535] type Int8 = range int8[-127:127] type Int16 = range int16[-32767..32768]
So, with my proposal, that will include the
@beoran I like the generalization to ranged types, but note that there is one subtle mismatch between this proposal and yours. Your proposal requires:
This proposal explicitly does not: the overflow check in this proposal is always dynamic, because that's much easier to implement portably, and it is possible that the part of the code in which the overflow occurs may not actually be reachable (for example, because it is on a code path that is only reachable on an architecture with a different-sized
I see, ease of implementation is also an important element. I am willing to change my proposal to allow the compiler to make the check always dynamic for an assignment to a variable in a function. For an assignment to a constant, the range check should always be performed at compile time. I not sure how the range check should be done for package level variables. Sure, a panic is possible but might be confusing.
I amended the proposal. Now I propose the range check is on compile type for constants, and run time for variables, although the compiler may still do the range checking at compile time (implementation defined). I also detailed the spec of len(range), worked out other details more. I also disallowed complex underlying types for range types and for range boundaries since for complex64 and complex128, the meaning of "range" becomes quite complex, indeed.
I would argue that the comma-ok idea suggested here, part 2 of the proposal, should not be done initially. I argue this on two points. First, I think that few people will actually want that feature. Second, for those people who really do want it, they can already write it by using a deferred function that calls
I would be ok with omitting the
I think its major use-cases are for down-casting
This is why I feel ranged types are an important feature of Ada and other Wirth-like languages: they help writing correct programs without overflow. As it stands now, in Go, like other C inspired languages, it is still too easy to accidentally cause overflow. Go would do well to adopt range types to prevent this.