New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: permit blank (_) separator in integer number literals #28493

Open
griesemer opened this Issue Oct 30, 2018 · 75 comments

Comments

Projects
None yet
@griesemer
Copy link
Contributor

griesemer commented Oct 30, 2018

This has come up before, specifically during discussions of #19308 and #28256. I am writing this down so we have a place for discussing this independently. I have no strong feelings about this proposal either way.

Proposal

This is a fully backward-compatible proposal for permitting the use of a blank (_) as a separator in number literals. Specifically, we change the integer literal syntax such that we also allow a "_" after the first digit (the change is the extra "_"):

int_lit = decimal_lit | octal_lit | hex_lit .
decimal_lit = ( "1" … "9" ) { decimal_digit | "_" } .
octal_lit = "0" { octal_digit | "_" } .
hex_lit = "0" ( "x" | "X" ) hex_digit { hex_digit | "_" } .

And we change the floating-point number literal syntax correspondingly (the change is the extra "_"):

float_lit = decimals "." [ decimals ] [ exponent ] | decimals exponent | "." decimals [ exponent ] .
decimals = decimal_digit { decimal_digit | "_" } .
exponent = ( "e" | "E" ) [ "+" | "-" ] decimals .

For complex number literals the change is implied by the change to floating-point literals.

Examples:

123_456_789
1_000_000_000
0b0110_1101_1101_0010  // if we had binary integer literals
0xdead_beef
3.1415_9265

but also:

0___         // 0
0_77         // 77 decimal, or perhaps not permitted (TBD)
01_10        // 0110 (octal)
1_2__3___    // 123
0x1_23_0     // 0x1230
0__.0__e-0__ // 0.0e-0

Discussion

The notation follows more or less the syntax used in other languages (e.g., Swift).
The implementation is straight-forward (a minor change to the number scanner). As the examples show, the separator, if judiciously used, may improve readability; or it may degrade it significantly.
A tool (go vet) might impose restrictions on how the separator is used for readability.

@ericlagergren

This comment has been minimized.

Copy link
Contributor

ericlagergren commented Oct 30, 2018

What would be the reason for allowing the underscore to appear anywhere instead of limiting it to (multiples of) commonly used places? E.g.,

1_000_000_000
0xDEAD_BEEF
0b0101_1111_0001

ISTM being too permissive only degrades readability.

@dgryski

This comment has been minimized.

Copy link
Contributor

dgryski commented Oct 30, 2018

@ericlagergren It seems more Go-like to leave the grammar more free-form with regards to placement of _, but leave the enforcement of style (how many digits in each group) to tooling and code review.

@ericlagergren

This comment has been minimized.

Copy link
Contributor

ericlagergren commented Oct 30, 2018

@griesemer

This comment has been minimized.

Copy link
Contributor Author

griesemer commented Oct 30, 2018

@ericlagergren What's a commonly used placement of _? That's really difficult to impossible to answer and shouldn't be decided by the language. Also, note that other languages also don't try to pin that down.

@ericlagergren

This comment has been minimized.

Copy link
Contributor

ericlagergren commented Oct 30, 2018

@alanfo

This comment has been minimized.

Copy link

alanfo commented Oct 31, 2018

I think this is an excellent idea if we are to have binary literals (which otherwise become unreadable after one or two bytes) and also helps with the readability of other long numbers.

Several other languages have a similar feature.

As far as placement is concerned, we'll just have to leave that to the good taste of gophers :)

@davecheney

This comment has been minimized.

Copy link
Contributor

davecheney commented Oct 31, 2018

I don’t think this improves readability. For example, which is more readable?

x := 100000000
y := 100_000_000
const million = 1000000
z := 100 * million

The latter is, IMO, the more descriptive, leverages the qualities of untyped constants, the lineage of the time package, and exists today without new syntax.

@alanfo

This comment has been minimized.

Copy link

alanfo commented Oct 31, 2018

Well, exact powers of 10 can always be dealt with in some other manner. One could also do this:

z2 := int(1e8)

But what about these:

// binary literals
a := 0b0101110110011011
b := 0b01011101_10011011 // bytes
c := 0b0101_1101_1001_1011 // nybbles

// decimal literals
d := 2861945612983
e := 2_861_945_612_983 // thousand separators

To my eyes, the underscored literals are much easier to read.

@mundaym

This comment has been minimized.

Copy link
Member

mundaym commented Oct 31, 2018

Are there any concrete examples of existing code where this would help readability significantly? Adding a separator does make sense for big numbers but are they common enough to warrant changing the language for?

I've spent a bit of time working with Swift code and in my limited experience I've seen that underscores tend to get used inconsistently. I think they end up making it harder (for me anyway) to parse the code.

Go already has quite a few ways to write constants, I'm not sure we need more.

@alanfo

This comment has been minimized.

Copy link

alanfo commented Oct 31, 2018

When programming in other languages which have digit separators (C#, Java, Kotlin), I generally split binary literals into nybbles and add thousand separators for decimal literals >= 1,000. That more or less mimics how I'd write them down on paper (with space and comma as respective separators).

Of course, that doesn't mean I find longer numbers without separators unreadable - I'm OK up to about twice those lengths - but, when you have something, you tend to use it.

Very long decimal literals don't crop up very often (except perhaps powers of 10) but when they do it really is good to have the digit separator in the toolbox :)

If this proposal is accepted, then no doubt there will be people who abuse it or use it inconsistently but I don't really think it is practical to limit placement in some way. Other languages don't seem to bother either.

@deanveloper

This comment has been minimized.

Copy link

deanveloper commented Oct 31, 2018

@RalphCorderoy

This comment has been minimized.

Copy link

RalphCorderoy commented Oct 31, 2018

I think it would be better to rule out adjacent or trailing underscores, and to always want at least one digit between the base indicator and the first underscore, e.g. 0_7 is invalid, effectively ruling out a leading underscore on the value's digits.

int_lit = decimal_lit | octal_lit | hex_lit .
decimal_lit = ( "1" … "9" ) { decimal_digit | "_" decimal_digit } .
octal_lit = "0" [ octal_digit { octal_digit | "_" octal_digit } ] .
hex_lit = "0" ( "x" | "X" ) hex_digit { hex_digit | "_" hex_digit } .
@griesemer

This comment has been minimized.

Copy link
Contributor Author

griesemer commented Oct 31, 2018

@RalphCorderoy Note that the proposed syntax already requires at least one digit after the base indicator before the first blank (_). Based on that rule, 0_7 would be decimal 7; but for this specific case of octals, I agree it should be invalid.

I'm not convinced about ruling out adjacent separators. For one, other languages permit it, and not permitting it would truly make the literal syntax quite a bit more complex. As is, it simply follows the same pattern we already have for identifiers.

Also, consider as a counter-example: 0b0110_0111_1010_1100__0011_0100_0101_1100 .

@RalphCorderoy

This comment has been minimized.

Copy link

RalphCorderoy commented Oct 31, 2018

Hi @griesemer, the double underscore for a 16-bit boundary is a good example of why it should be permitted.

With leading underscores, including octal, ruled out, is there a reason to permit trailing ones? It's allowed in identifiers, but this change is about separating digits for readability. The lexing of 42_____x can complain about the trailing _ on peeking the x because lexing as 42 and _____x instead, after much backtracking, wouldn't be valid for later parsing anyway AFAIK.

@deanveloper

This comment has been minimized.

Copy link

deanveloper commented Oct 31, 2018

0_77 // 77 decimal, or perhaps not permitted (TBD)

Definitely should not be permitted. It looks like the proposed syntax prevents this though.

[...] is there a reason to permit trailing ones? It's allowed in identifiers [...]

I can't think of any reasons. Also, identifiers allow leading underscores as well, and we've ruled those out so I'm not sure if that logic applies

@cznic

This comment has been minimized.

Copy link
Contributor

cznic commented Oct 31, 2018

This is a fully backward-compatible proposal for permitting the use of a blank (_) as a separator in number literals.

I don't think this is the case. Currently 123_456 forms three valid tokens while under the proposal it would become just one.

I'm aware the compatibility guarantee is narrowed to "continues to compile", but that's not the same as "fully backwards compatible", as that should IMHO include not breaking also any of the existing tools.

@magical

This comment has been minimized.

Copy link
Contributor

magical commented Oct 31, 2018

@cznic As you say, the compatibility guarantee only says that valid code will continue to work; it makes no particular guarantees about invalid code.

The cost of updating tools to support new language features is something that has to be considered with any language change, but that is a separate issue from backward compatibility.

@rasky

This comment has been minimized.

Copy link
Member

rasky commented Nov 1, 2018

While doing embedded programming, I often use hexadecimal constants and I feel that 0x0f00_0000 is a big readability improvement over 0x0f000000, as it's too easy to miss a digit.

@seaskyways

This comment has been minimized.

Copy link

seaskyways commented Nov 1, 2018

1+
This feature has been implemented lately in Kotlin and has been of great help. Would love to see it in Golang!

@deanveloper

This comment has been minimized.

Copy link

deanveloper commented Nov 1, 2018

I don’t think this improves readability. For example, which is more readable?

x := 100000000
y := 100_000_000
const million = 1000000
z := 100 * million

The latter is, IMO, the more descriptive, leverages the qualities of untyped constants, the lineage of the time package, and exists today without new syntax.
@davecheney

For simple, small(er) numbers, this argument holds. But it starts breaking down once we start talking about numbers that are larger than a billion. I'd rather not have to look back and count 9 zeroes just to verify that 100000000 is actually a billion so that I can do 5 * billion.

The argument also starts to break down when we start to talk about numbers which aren't as simple as 100 million, for instance, a large prime number 10107476689

x := 10_107_476_689

// alternatively
const thousand = 1000
const million = thousand * 1000;
const billion = million * 1000;
y := 10 * billion + 107 * million + 476 * thousand + 689

The argument also breaks down when you're not talking about decimal notation, which has already been brought up so I won't go into depth, but it's very useful to be able to break up binary/hex into nybbles/bytes.

PS - the 100000000 in the first paragraph only contains 8 zeroes, which was done to illustrate that your solution starts to break down a bit once you have larger numbers

@tomvanwoow

This comment has been minimized.

Copy link

tomvanwoow commented Nov 1, 2018

I don't believe it is a good idea to enforce restrictions on where the underscores must go, as I can imagine some people preferring 1234_567_890 to 1_234_567_890. It should be down the coder to write nice looking code.

@theodesp

This comment has been minimized.

Copy link

theodesp commented Nov 5, 2018

It looks like this proposal is very similar to Python's PEP-515

Ideally, we would also like to have rules for fmt also. For example:

a := 10107476689
fmt.Printf("%#_d", a) // 10_107_476_689
fmt.Printf("%#_x", a) // '2_5a73_dad1'
@deanveloper

This comment has been minimized.

Copy link

deanveloper commented Nov 5, 2018

@theodesp again I bring up international number formatting

Is there any reason to put it in fmt? We do not have comma separators in fmt today, and I think for good reason. Number formats are localized, so I'm not sure if it's a good idea for fmt. Maybe it'd be good for the text package, but we already have comma separation there, so I don't see much use

Also, the # parameter is already taken by "alternate form". For instance, it adds a leading "0x" for "%x". Not sure if you knew this or not. If you did, you forgot to put it in your outputs, and there is no "%#d". If you didn't, well I hope you learned something new 😃

@theodesp

This comment has been minimized.

Copy link

theodesp commented Nov 5, 2018

I knew it, this was just an example extension to the existing flags. I use fmt because its the most obvious choice and I don't think the underscores are part of any sort of internationalization as the scope of this proposal is totally different.

@deanveloper

This comment has been minimized.

Copy link

deanveloper commented Nov 5, 2018

The main point that I was making was that separating by 3s is localized - the Indian number format does not do this. Go forces the "." separator in fmt because it's part of the language, but separating numbers by 3s is not part of the language, so I'm not sure if that's in the scope of fmt

@rsc rsc modified the milestones: Proposal, Go1.13 Jan 30, 2019

@rsc

This comment has been minimized.

Copy link
Contributor

rsc commented Jan 30, 2019

As a reminder, we introduced a new process for these Go 2-related language changes in our blog post blog.golang.org/go2-here-we-come. We are going to tentatively accept a proposal, land changes at the start of a cycle, get experience using it, and then make the final acceptance decision three months later, at the freeze. For Go 1.13, this would mean landing a change when the tree opens February, and making the final decision when the tree freezes in May.

We are going to tentatively accept this proposal for Go 1.13 and plan to land its implementation when the tree opens. The issue state for "tentative accept" will be marked Proposal-Accepted but left open and milestoned to the Go release (Go1.13 here). At the freeze we will revisit the issue and close it if it is finally accepted.

@griesemer

This comment has been minimized.

Copy link
Contributor Author

griesemer commented Feb 3, 2019

After rethinking my initial implementation a bit, I need to revise my #28493 (comment): My initial implementation was trying overly hard to only accept valid literals and stop tokenization early.

In retrospect this was clearly the wrong approach and led to an overly complex solution. It's much better to accept a more lenient number literal syntax (separators everywhere, etc.) and check after the fact (basically what https://go-review.googlesource.com/c/go/+/160243 is doing for strconv). The resulting implementation is massively simpler (simpler in fact than what was there before and more reliably testable) and also leads to better error messages from the compiler. For instance, instead of tokenizing 123__45_ as 123, __45, _ per the spec's tokenization rule it's better to accept 123_45_ as a single token and complain about the incorrect '_''s (either way the program is not correct, so it's ok to slightly deviate from how tokens are split up).

In short, separators are actually straight-forward to handle correctly.

@gopherbot

This comment has been minimized.

Copy link

gopherbot commented Feb 4, 2019

Change https://golang.org/cl/161098 mentions this issue: spec: document new Go2 number literals

@gopherbot

This comment has been minimized.

Copy link

gopherbot commented Feb 5, 2019

Change https://golang.org/cl/161199 mentions this issue: text/scanner: accept new Go2 number literals

gopherbot pushed a commit that referenced this issue Feb 11, 2019

cmd/compile: accept new Go2 number literals
This CL introduces compiler support for the new binary and octal integer
literals, hexadecimal floats, and digit separators for all number literals.

The new Go 2 number literal scanner accepts the following liberal format:

number   = [ prefix ] digits [ "." digits ] [ exponent ] [ "i" ] .
prefix   = "0" [ "b" |"B" | "o" | "O" | "x" | "X" ] .
digits   = { digit | "_" } .
exponent = ( "e" | "E" | "p" | "P" ) [ "+" | "-" ] digits .

If the number starts with "0x" or "0X", digit is any hexadecimal digit;
otherwise, digit is any decimal digit. If the accepted number is not valid,
errors are reported accordingly.

See the new test cases in scanner_test.go for a selection of valid and
invalid numbers and the respective error messages.

R=Go1.13

Updates #12711.
Updates #19308.
Updates #28493.
Updates #29008.

Change-Id: Ic8febc7bd4dc5186b16a8c8897691e81125cf0ca
Reviewed-on: https://go-review.googlesource.com/c/157677
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

gopherbot pushed a commit that referenced this issue Feb 11, 2019

go/scanner: accept new Go2 number literals
This CL introduces go/scanner support for the new binary and octal integer
literals, hexadecimal floats, and digit separators for all number literals.
The new code is closely mirroring the respective code for number literals in
cmd/compile/internal/syntax/scanner.go.

R=Go1.13

Updates #12711.
Updates #19308.
Updates #28493.
Updates #29008.

Change-Id: I5315c6aaa7cfc41a618296be20e3acd5114d6b3c
Reviewed-on: https://go-review.googlesource.com/c/159997
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

gopherbot pushed a commit that referenced this issue Feb 11, 2019

cmd/gofmt: test that Go 2 number literals can be formatted
R=Go1.13

Updates #12711.
Updates #19308.
Updates #28493.
Updates #29008.

Change-Id: Icd25aa7f6e18ed671ea6cf2b1b292899daf4b1a5
Reviewed-on: https://go-review.googlesource.com/c/160018
Reviewed-by: Russ Cox <rsc@golang.org>

gopherbot pushed a commit that referenced this issue Feb 11, 2019

go/constant: accept new Go2 number literals
This CL introduces go/constant support for the new binary and octal integer
literals, hexadecimal floats, and digit separators for all number literals.

R=Go1.13

Updates #12711.
Updates #19308.
Updates #28493.
Updates #29008.

Change-Id: I7a55f91b8b6373ae6d98ba923b626d33c5552946
Reviewed-on: https://go-review.googlesource.com/c/160239
Reviewed-by: Russ Cox <rsc@golang.org>

gopherbot pushed a commit that referenced this issue Feb 11, 2019

go/types: add tests for new Go 2 number literals
This CL ensures that go/types can now handle the new
Go 2 number literals. The relevant changes enabling
them in go/types were made in go/constant in the CL
https://golang.org/cl/160239.

R=Go1.13

Updates #12711.
Updates #19308.
Updates #28493.
Updates #29008.

Change-Id: I45c1387198fac94769ac59c5301d86b4e1a1ff98
Reviewed-on: https://go-review.googlesource.com/c/160240
Reviewed-by: Russ Cox <rsc@golang.org>

gopherbot pushed a commit that referenced this issue Feb 11, 2019

text/scanner: accept new Go2 number literals
This CL introduces text/scanner support for the new binary and octal integer
literals, hexadecimal floats, and digit separators for all number literals.
The new code is closely mirroring the respective code for number literals in
cmd/compile/internal/syntax/scanner.go.

Uniformly use the term "invalid" rather than "illegal" in error messages
to match the respective error messages in the other scanners directly.

R=Go1.13

Updates #12711.
Updates #19308.
Updates #28493.
Updates #29008.

Change-Id: I2f291de13ba5afc0e530cd8326e6bf4c3858ebac
Reviewed-on: https://go-review.googlesource.com/c/161199
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

gopherbot pushed a commit that referenced this issue Feb 12, 2019

strconv: accept underscores in ParseInt, ParseUint, ParseFloat
This CL modifies ParseInt, ParseUint, and ParseFloat
to accept digit-separating underscores in their arguments.
For ParseInt and ParseUint, the underscores are only
allowed when base == 0.

See golang.org/design/19308-number-literals for background.

For #28493.

Change-Id: I057ca2539d89314643f591ba8144c3ea7126651c
Reviewed-on: https://go-review.googlesource.com/c/160243
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>

nebulabox added a commit to nebulabox/go that referenced this issue Feb 18, 2019

cmd/compile: accept new Go2 number literals
This CL introduces compiler support for the new binary and octal integer
literals, hexadecimal floats, and digit separators for all number literals.

The new Go 2 number literal scanner accepts the following liberal format:

number   = [ prefix ] digits [ "." digits ] [ exponent ] [ "i" ] .
prefix   = "0" [ "b" |"B" | "o" | "O" | "x" | "X" ] .
digits   = { digit | "_" } .
exponent = ( "e" | "E" | "p" | "P" ) [ "+" | "-" ] digits .

If the number starts with "0x" or "0X", digit is any hexadecimal digit;
otherwise, digit is any decimal digit. If the accepted number is not valid,
errors are reported accordingly.

See the new test cases in scanner_test.go for a selection of valid and
invalid numbers and the respective error messages.

R=Go1.13

Updates golang#12711.
Updates golang#19308.
Updates golang#28493.
Updates golang#29008.

Change-Id: Ic8febc7bd4dc5186b16a8c8897691e81125cf0ca
Reviewed-on: https://go-review.googlesource.com/c/157677
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

nebulabox added a commit to nebulabox/go that referenced this issue Feb 18, 2019

go/scanner: accept new Go2 number literals
This CL introduces go/scanner support for the new binary and octal integer
literals, hexadecimal floats, and digit separators for all number literals.
The new code is closely mirroring the respective code for number literals in
cmd/compile/internal/syntax/scanner.go.

R=Go1.13

Updates golang#12711.
Updates golang#19308.
Updates golang#28493.
Updates golang#29008.

Change-Id: I5315c6aaa7cfc41a618296be20e3acd5114d6b3c
Reviewed-on: https://go-review.googlesource.com/c/159997
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

nebulabox added a commit to nebulabox/go that referenced this issue Feb 18, 2019

cmd/gofmt: test that Go 2 number literals can be formatted
R=Go1.13

Updates golang#12711.
Updates golang#19308.
Updates golang#28493.
Updates golang#29008.

Change-Id: Icd25aa7f6e18ed671ea6cf2b1b292899daf4b1a5
Reviewed-on: https://go-review.googlesource.com/c/160018
Reviewed-by: Russ Cox <rsc@golang.org>

nebulabox added a commit to nebulabox/go that referenced this issue Feb 18, 2019

go/constant: accept new Go2 number literals
This CL introduces go/constant support for the new binary and octal integer
literals, hexadecimal floats, and digit separators for all number literals.

R=Go1.13

Updates golang#12711.
Updates golang#19308.
Updates golang#28493.
Updates golang#29008.

Change-Id: I7a55f91b8b6373ae6d98ba923b626d33c5552946
Reviewed-on: https://go-review.googlesource.com/c/160239
Reviewed-by: Russ Cox <rsc@golang.org>

nebulabox added a commit to nebulabox/go that referenced this issue Feb 18, 2019

go/types: add tests for new Go 2 number literals
This CL ensures that go/types can now handle the new
Go 2 number literals. The relevant changes enabling
them in go/types were made in go/constant in the CL
https://golang.org/cl/160239.

R=Go1.13

Updates golang#12711.
Updates golang#19308.
Updates golang#28493.
Updates golang#29008.

Change-Id: I45c1387198fac94769ac59c5301d86b4e1a1ff98
Reviewed-on: https://go-review.googlesource.com/c/160240
Reviewed-by: Russ Cox <rsc@golang.org>

nebulabox added a commit to nebulabox/go that referenced this issue Feb 18, 2019

text/scanner: accept new Go2 number literals
This CL introduces text/scanner support for the new binary and octal integer
literals, hexadecimal floats, and digit separators for all number literals.
The new code is closely mirroring the respective code for number literals in
cmd/compile/internal/syntax/scanner.go.

Uniformly use the term "invalid" rather than "illegal" in error messages
to match the respective error messages in the other scanners directly.

R=Go1.13

Updates golang#12711.
Updates golang#19308.
Updates golang#28493.
Updates golang#29008.

Change-Id: I2f291de13ba5afc0e530cd8326e6bf4c3858ebac
Reviewed-on: https://go-review.googlesource.com/c/161199
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

nebulabox added a commit to nebulabox/go that referenced this issue Feb 18, 2019

strconv: accept underscores in ParseInt, ParseUint, ParseFloat
This CL modifies ParseInt, ParseUint, and ParseFloat
to accept digit-separating underscores in their arguments.
For ParseInt and ParseUint, the underscores are only
allowed when base == 0.

See golang.org/design/19308-number-literals for background.

For golang#28493.

Change-Id: I057ca2539d89314643f591ba8144c3ea7126651c
Reviewed-on: https://go-review.googlesource.com/c/160243
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>

nebulabox added a commit to nebulabox/go that referenced this issue Feb 20, 2019

cmd/compile: accept new Go2 number literals
This CL introduces compiler support for the new binary and octal integer
literals, hexadecimal floats, and digit separators for all number literals.

The new Go 2 number literal scanner accepts the following liberal format:

number   = [ prefix ] digits [ "." digits ] [ exponent ] [ "i" ] .
prefix   = "0" [ "b" |"B" | "o" | "O" | "x" | "X" ] .
digits   = { digit | "_" } .
exponent = ( "e" | "E" | "p" | "P" ) [ "+" | "-" ] digits .

If the number starts with "0x" or "0X", digit is any hexadecimal digit;
otherwise, digit is any decimal digit. If the accepted number is not valid,
errors are reported accordingly.

See the new test cases in scanner_test.go for a selection of valid and
invalid numbers and the respective error messages.

R=Go1.13

Updates golang#12711.
Updates golang#19308.
Updates golang#28493.
Updates golang#29008.

Change-Id: Ic8febc7bd4dc5186b16a8c8897691e81125cf0ca
Reviewed-on: https://go-review.googlesource.com/c/157677
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

nebulabox added a commit to nebulabox/go that referenced this issue Feb 20, 2019

go/scanner: accept new Go2 number literals
This CL introduces go/scanner support for the new binary and octal integer
literals, hexadecimal floats, and digit separators for all number literals.
The new code is closely mirroring the respective code for number literals in
cmd/compile/internal/syntax/scanner.go.

R=Go1.13

Updates golang#12711.
Updates golang#19308.
Updates golang#28493.
Updates golang#29008.

Change-Id: I5315c6aaa7cfc41a618296be20e3acd5114d6b3c
Reviewed-on: https://go-review.googlesource.com/c/159997
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

nebulabox added a commit to nebulabox/go that referenced this issue Feb 20, 2019

cmd/gofmt: test that Go 2 number literals can be formatted
R=Go1.13

Updates golang#12711.
Updates golang#19308.
Updates golang#28493.
Updates golang#29008.

Change-Id: Icd25aa7f6e18ed671ea6cf2b1b292899daf4b1a5
Reviewed-on: https://go-review.googlesource.com/c/160018
Reviewed-by: Russ Cox <rsc@golang.org>

nebulabox added a commit to nebulabox/go that referenced this issue Feb 20, 2019

go/constant: accept new Go2 number literals
This CL introduces go/constant support for the new binary and octal integer
literals, hexadecimal floats, and digit separators for all number literals.

R=Go1.13

Updates golang#12711.
Updates golang#19308.
Updates golang#28493.
Updates golang#29008.

Change-Id: I7a55f91b8b6373ae6d98ba923b626d33c5552946
Reviewed-on: https://go-review.googlesource.com/c/160239
Reviewed-by: Russ Cox <rsc@golang.org>

nebulabox added a commit to nebulabox/go that referenced this issue Feb 20, 2019

go/types: add tests for new Go 2 number literals
This CL ensures that go/types can now handle the new
Go 2 number literals. The relevant changes enabling
them in go/types were made in go/constant in the CL
https://golang.org/cl/160239.

R=Go1.13

Updates golang#12711.
Updates golang#19308.
Updates golang#28493.
Updates golang#29008.

Change-Id: I45c1387198fac94769ac59c5301d86b4e1a1ff98
Reviewed-on: https://go-review.googlesource.com/c/160240
Reviewed-by: Russ Cox <rsc@golang.org>

nebulabox added a commit to nebulabox/go that referenced this issue Feb 20, 2019

text/scanner: accept new Go2 number literals
This CL introduces text/scanner support for the new binary and octal integer
literals, hexadecimal floats, and digit separators for all number literals.
The new code is closely mirroring the respective code for number literals in
cmd/compile/internal/syntax/scanner.go.

Uniformly use the term "invalid" rather than "illegal" in error messages
to match the respective error messages in the other scanners directly.

R=Go1.13

Updates golang#12711.
Updates golang#19308.
Updates golang#28493.
Updates golang#29008.

Change-Id: I2f291de13ba5afc0e530cd8326e6bf4c3858ebac
Reviewed-on: https://go-review.googlesource.com/c/161199
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

nebulabox added a commit to nebulabox/go that referenced this issue Feb 20, 2019

strconv: accept underscores in ParseInt, ParseUint, ParseFloat
This CL modifies ParseInt, ParseUint, and ParseFloat
to accept digit-separating underscores in their arguments.
For ParseInt and ParseUint, the underscores are only
allowed when base == 0.

See golang.org/design/19308-number-literals for background.

For golang#28493.

Change-Id: I057ca2539d89314643f591ba8144c3ea7126651c
Reviewed-on: https://go-review.googlesource.com/c/160243
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
@gopherbot

This comment has been minimized.

Copy link

gopherbot commented Feb 20, 2019

Change https://golang.org/cl/163079 mentions this issue: text/scanner: don't liberally consume (invalid) floats or underbars

gopherbot pushed a commit that referenced this issue Feb 20, 2019

text/scanner: don't liberally consume (invalid) floats or underbars
This is a follow-up on https://golang.org/cl/161199 which introduced
the new Go 2 number literals to text/scanner.

That change introduced a bug by allowing decimal and hexadecimal floats
to be consumed even if the scanner was not configured to accept floats.

This CL changes the code to not consume a radix dot '.' or exponent
unless the scanner is configured to accept floats.

This CL also introduces a new mode "AllowNumberbars" which controls
whether underbars '_' are permitted as digit separators in numbers
or not.

There is a possibility that we may need to refine text/scanner
further (e.g., the Float mode now includes hexadecimal floats
which it didn't recognize before). We're very early in the cycle,
so let's see how it goes.

RELNOTE=yes

Updates #12711.
Updates #19308.
Updates #28493.
Updates #29008.

Fixes #30320.

Change-Id: I6481d314f0384e09ef6803ffad38dc529b1e89a3
Reviewed-on: https://go-review.googlesource.com/c/163079
Reviewed-by: Ian Lance Taylor <iant@golang.org>

nebulabox added a commit to nebulabox/go that referenced this issue Feb 21, 2019

text/scanner: don't liberally consume (invalid) floats or underbars
This is a follow-up on https://golang.org/cl/161199 which introduced
the new Go 2 number literals to text/scanner.

That change introduced a bug by allowing decimal and hexadecimal floats
to be consumed even if the scanner was not configured to accept floats.

This CL changes the code to not consume a radix dot '.' or exponent
unless the scanner is configured to accept floats.

This CL also introduces a new mode "AllowNumberbars" which controls
whether underbars '_' are permitted as digit separators in numbers
or not.

There is a possibility that we may need to refine text/scanner
further (e.g., the Float mode now includes hexadecimal floats
which it didn't recognize before). We're very early in the cycle,
so let's see how it goes.

RELNOTE=yes

Updates golang#12711.
Updates golang#19308.
Updates golang#28493.
Updates golang#29008.

Fixes golang#30320.

Change-Id: I6481d314f0384e09ef6803ffad38dc529b1e89a3
Reviewed-on: https://go-review.googlesource.com/c/163079
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment