Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix #1

Merged
merged 546 commits into from May 16, 2019

Conversation

Projects
None yet
@Kiku-Reise
Copy link
Owner

commented May 16, 2019

This PR will be imported into Gerrit with the title and first
comment (this text) used to generate the subject and body of
the Gerrit change.

Please ensure you adhere to every item in this list.

More info can be found at https://github.com/golang/go/wiki/CommitMessage

  • The PR title is formatted as follows: net/http: frob the quux before blarfing
    • The package name goes before the colon
    • The part after the colon uses the verb tense + phrase that completes the blank in,
      "This change modifies Go to ___________"
    • Lowercase verb after the colon
    • No trailing period
    • Keep the title as short as possible. ideally under 76 characters or shorter
  • No Markdown
  • The first PR comment (this one) is wrapped at 76 characters, unless it's
    really needed (ASCII art, table, or long link)
  • If there is a corresponding issue, add either Fixes #1234 or Updates #1234
    (the latter if this is not a complete fix) to this comment
  • If referring to a repo other than golang/go you can use the
    owner/repo#issue_number syntax: Fixes golang/tools#1234
  • We do not use Signed-off-by lines in Go. Please don't add them.
    Our Gerrit server & GitHub bots enforce CLA compliance instead.
  • Delete these instructions once you have read and applied them

bmkessler and others added some commits Mar 10, 2019

cmd/compile: add unsigned divisibility rules
"Division by invariant integers using multiplication" paper
by Granlund and Montgomery contains a method for directly computing
divisibility (x%c == 0 for c constant) by means of the modular inverse.
The method is further elaborated in "Hacker's Delight" by Warren Section 10-17

This general rule can compute divisibilty by one multiplication and a compare
for odd divisors and an additional rotate for even divisors.

To apply the divisibility rule, we must take into account
the rules to rewrite x%c = x-((x/c)*c) and (x/c) for c constant on the first
optimization pass "opt".  This complicates the matching as we want to match
only in the cases where the result of (x/c) is not also available.
So, we must match on the expanded form of (x/c) in the expression x == c*(x/c)
in the "late opt" pass after common subexpresion elimination.

Note, that if there is an intermediate opt pass introduced in the future we
could simplify these rules by delaying the magic division rewrite to "late opt"
and matching directly on (x/c) in the intermediate opt pass.

Additional rules to lower the generic RotateLeft* ops were also applied.

On amd64, the divisibility check is 25-50% faster.

name                     old time/op  new time/op  delta
DivconstI64-4            2.08ns ± 0%  2.08ns ± 1%     ~     (p=0.881 n=5+5)
DivisibleconstI64-4      2.67ns ± 0%  2.67ns ± 1%     ~     (p=1.000 n=5+5)
DivisibleWDivconstI64-4  2.67ns ± 0%  2.67ns ± 0%     ~     (p=0.683 n=5+5)
DivconstU64-4            2.08ns ± 1%  2.08ns ± 1%     ~     (p=1.000 n=5+5)
DivisibleconstU64-4      2.77ns ± 1%  1.55ns ± 2%  -43.90%  (p=0.008 n=5+5)
DivisibleWDivconstU64-4  2.99ns ± 1%  2.99ns ± 1%     ~     (p=1.000 n=5+5)
DivconstI32-4            1.53ns ± 2%  1.53ns ± 0%     ~     (p=1.000 n=5+5)
DivisibleconstI32-4      2.23ns ± 0%  2.25ns ± 3%     ~     (p=0.167 n=5+5)
DivisibleWDivconstI32-4  2.27ns ± 1%  2.27ns ± 1%     ~     (p=0.429 n=5+5)
DivconstU32-4            1.78ns ± 0%  1.78ns ± 1%     ~     (p=1.000 n=4+5)
DivisibleconstU32-4      2.52ns ± 2%  1.26ns ± 0%  -49.96%  (p=0.000 n=5+4)
DivisibleWDivconstU32-4  2.63ns ± 0%  2.85ns ±10%   +8.29%  (p=0.016 n=4+5)
DivconstI16-4            1.54ns ± 0%  1.54ns ± 0%     ~     (p=0.333 n=4+5)
DivisibleconstI16-4      2.10ns ± 0%  2.10ns ± 1%     ~     (p=0.571 n=4+5)
DivisibleWDivconstI16-4  2.22ns ± 0%  2.23ns ± 1%     ~     (p=0.556 n=4+5)
DivconstU16-4            1.09ns ± 0%  1.01ns ± 1%   -7.74%  (p=0.000 n=4+5)
DivisibleconstU16-4      1.83ns ± 0%  1.26ns ± 0%  -31.52%  (p=0.008 n=5+5)
DivisibleWDivconstU16-4  1.88ns ± 0%  1.89ns ± 1%     ~     (p=0.365 n=5+5)
DivconstI8-4             1.54ns ± 1%  1.54ns ± 1%     ~     (p=1.000 n=5+5)
DivisibleconstI8-4       2.10ns ± 0%  2.11ns ± 0%     ~     (p=0.238 n=5+4)
DivisibleWDivconstI8-4   2.22ns ± 0%  2.23ns ± 2%     ~     (p=0.762 n=5+5)
DivconstU8-4             0.92ns ± 1%  0.94ns ± 1%   +2.65%  (p=0.008 n=5+5)
DivisibleconstU8-4       1.66ns ± 0%  1.26ns ± 1%  -24.28%  (p=0.008 n=5+5)
DivisibleWDivconstU8-4   1.79ns ± 0%  1.80ns ± 1%     ~     (p=0.079 n=4+5)

A follow-up change will address the signed division case.

Updates golang#30282

Change-Id: I7e995f167179aa5c76bb10fbcbeb49c520943403
Reviewed-on: https://go-review.googlesource.com/c/go/+/168037
Run-TryBot: Brian Kessler <brian.m.kessler@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
cmd/link/internal/ld: consolidate macho platform setup
Determine the macho platform once and use that the two places that
need it. This makes it easier to add a third platform check for a
follow-up change.

Updates golang#31447

Change-Id: I522a5fface647ab8e608f816c5832d531534df7a
Reviewed-on: https://go-review.googlesource.com/c/go/+/174198
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
cmd/link/internal/ld,syscall: drop $INODE64 suffixes on simulators
Some libc functions are suffixed with "$INODE64" on macOS.
Unfortunately, the iOS simulator doesn't have the suffixes, so we can't
use GOARCH to distinguish the two platform.

Add linker support for adding the suffix, using the macho platform
to determine whether it is needed.

While here, add the correct suffix for fdopendir on 386. It's
"$INODE64$UNIX2003", believe it or not. Without the suffix,

GOARCH=386 go test -short syscall

crashes on my Mojave machine.

Fixes golang#31447

Change-Id: I9bd3de40ece7df62f744bc24cd00909e56b00b78
Reviewed-on: https://go-review.googlesource.com/c/go/+/174199
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
cmd/link/internal/ld,syscall: replace getfsstat64 with getfsstat
getfsstat64 is deprecated but not yet caught by the App Store checks.
Use the supported getfsstat$INODE64 form instead to ensure forward
compatibility.

Change-Id: I0d97e8a8b254debb3de1cfcb3778dbed3702c249
Reviewed-on: https://go-review.googlesource.com/c/go/+/174200
Run-TryBot: Elias Naur <mail@eliasnaur.com>
Reviewed-by: Keith Randall <khr@golang.org>
cmd/go/internal/renameio: use ERROR_ACCESS_DENIED to check for errors
CL 172418 added code to check for "Access is denied" error.
But "Access is denied" error will be spelled differently on
non-English version of Windows.

Check if error is ERROR_ACCESS_DENIED instead.

Updates golang#31247

Change-Id: I7b1633013d563f7c06c1f12a9be75122106834f9
Reviewed-on: https://go-review.googlesource.com/c/go/+/174123
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
syscall: allow setting security attributes on processes
This allows creating processes that can only be debugged/accessed by
certain tokens, according to a particular security descriptor. We
already had everything ready for this but just neglected to pass through
the value from the user-accessible SysProcAttr.

Change-Id: I4a3fcc9f5078aa0058b26c103355c984093ae03f
Reviewed-on: https://go-review.googlesource.com/c/go/+/174197
Run-TryBot: Jason Donenfeld <Jason@zx2c4.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
runtime: remove spurious register loads for openbsd/amd64 kqueue
The kqueue system call takes no arguments, hence there should be no need
to zero the registers used for the first syscall arguments.

Change-Id: Ia79b2d4f4d568bb6795cb885e1464cf1fc2bf7c7
Reviewed-on: https://go-review.googlesource.com/c/go/+/174128
Run-TryBot: Benny Siegert <bsiegert@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
cmd/compile: intrinsify math/bits.Add64 for ppc64x
This change creates an intrinsic for Add64 for ppc64x and adds a
testcase for it.

name               old time/op  new time/op  delta
Add64-160          1.90ns ±40%  2.29ns ± 0%     ~     (p=0.119 n=5+5)
Add64multiple-160  6.69ns ± 2%  2.45ns ± 4%  -63.47%  (p=0.016 n=4+5)

Change-Id: I9abe6fb023fdf62eea3c9b46a1820f60bb0a7f97
Reviewed-on: https://go-review.googlesource.com/c/go/+/173758
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
Run-TryBot: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com>
runtime: whitelist debugCall32..debugCall65536 in debugCallCheck
Whitelists functions debugCall32 through debugCall65536 in
runtime.debugCallCheck so that any instruction inside those functions
is considered a safe point.
This is useful for implementing nested function calls.

For example when evaluating:

	f(g(x))

The debugger should:

1. initiate the call to 'f' until the entry point of 'f',
2. complete the call to 'g(x)'
3. copy the return value of 'g(x)' in the arguments of 'f'
4. complete the call to 'f'

Similarly for:

	f().amethod()

The debugger should initiate the call to '.amethod()', then initiate
and complete the call to f(), copy the return value to the arguments
of '.amethod()' and finish its call.
However in this example, unlike the other example, it may be
impossible to determine the entry point of '.amethod()' until after
'f()' is evaluated, which means that the call to 'f()' needs to be
initiated while stopped inside a debugCall... function.

Change-Id: I575c23542709cedb1a171d63576f7e11069c7674
Reviewed-on: https://go-review.googlesource.com/c/go/+/161137
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
strconv: Document ParseFloat's special cases
Updates golang#30990

Change-Id: I968fb13251ab3796328089046a3f0fc5c7eb9df9
Reviewed-on: https://go-review.googlesource.com/c/go/+/174204
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Run-TryBot: Benny Siegert <bsiegert@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
cmd/go: implement Go checksum database support
This CL adds support for consulting the Go checksum database
when downloading a module that is not already listed in go.sum.
The overall system is described at golang.org/design/25530-sumdb,
and this CL implements the functionality described specifically in
golang.org/design/25530-sumdb#command-client.

Although the eventual plan is to set GOPROXY and GOSUMDB to
default to a Google-run proxy serving the public Go ecosystem,
this CL leaves them off by default.

Fixes golang#30601.

Change-Id: Ie46140f93c6cc2d85573fbce0878a258819ff44d
Reviewed-on: https://go-review.googlesource.com/c/go/+/173951
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
encoding/json: add a Fuzz function
Adds a sample Fuzz test function to package encoding/json following the
guidelines defined in golang#31309, based on
https://github.com/dvyukov/go-fuzz-corpus/blob/master/json/json.go

Fixes golang#31309
Updates golang#19109

Change-Id: I5fe04d9a5f41c0de339f8518dae30896ec14e356
Reviewed-on: https://go-review.googlesource.com/c/go/+/174058
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
all: remove a few unused parameters
I recently modified tabwriter to reduce the number of defers due to
flush calls. However, I forgot to notice that the new function
flushNoDefers can no longer return an error, due to the lack of the
defer.

In crypto/tls, hashForServerKeyExchange never returned a non-nil error,
so simplify the code.

Finally, in go/types and net we can find a few trivially unused
parameters, so remove them.

Change-Id: I54c8de83fbc944df432453b55c93008d7e810e61
Reviewed-on: https://go-review.googlesource.com/c/go/+/174131
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
net/http: remove "number:" from Response.Status string
The behavior of Value.String method on non-string JavaScript types has
changed after CL 169757.

Update the implementation of Transport.RoundTrip method to construct the
Response.Status string without relying on result.Get("status").String(),
since that now returns strings like "<number: 200>" instead of "200".

Fixes golang#31736

Change-Id: I27b3e6cc95aa65fd1918b1400e88478a154aad12
Reviewed-on: https://go-review.googlesource.com/c/go/+/174218
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Richard Musiol <neelance@gmail.com>
cmd/link/internal/s390x: fix s390x build
Fix breakage from CL 173437

Change-Id: If218ffaa1259fbdee641143ffbe4b38030c373b9
Reviewed-on: https://go-review.googlesource.com/c/go/+/174278
Reviewed-by: Michael Munday <mike.munday@ibm.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
net: correct docs of KeepAlive field in Dialer type
KeepAlive field used to report the wording "keep-alive period"
which may be misleading. This field does not represent the whole
TCP keepalive time, that is the inactivity period upon which one
endpoint starts probing the other end. But it acctually specifies
the keepalive interval, that is the time between two keepalive
probes.

Fixes golang#29089

Change-Id: If99b38ba108830d0e5fe527171a2f5c96a3bcde7
Reviewed-on: https://go-review.googlesource.com/c/go/+/155960
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
misc/wasm: fix command line arguments containing multi-byte characters
Command line arguments containing multi-byte characters were causing
go_js_wasm_exec to crash (RangeError: Source is too large), because
their byte length was not handled correctly. This change fixes the bug.

Fixes golang#31645.

Change-Id: I7860ebf5b12da37d9d0f43d4b6a22d326a90edaf
Reviewed-on: https://go-review.googlesource.com/c/go/+/173877
Run-TryBot: Richard Musiol <neelance@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
runtime: initialise cpu.HWCap on openbsd/arm64
OpenBSD does not provide auxv, however we still need to initialise cpu.HWCap.
For now initialise it to the bare minimum, until some form of CPU capability
detection is implemented or becomes available - see issue golang#31746.

Updates golang#31656

Change-Id: I68c3c069319fe60dc873f46def2a67c9f3d937d5
Reviewed-on: https://go-review.googlesource.com/c/go/+/174129
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
runtime: support all as parameter in gdb goroutine commands.
For example, can use `goroutine all bt` to dump all goroutines'
information.

Change-Id: I51b547c2b837913e4bdabf0f45b28f09250a3e34
GitHub-Last-Rev: d04dcd4
GitHub-Pull-Request: golang#26283
Reviewed-on: https://go-review.googlesource.com/c/go/+/122589
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
os/exec: always set SYSTEMROOT on Windows if not listed in Cmd.Env
Fixes golang#25210

Change-Id: If27b61776154dae9b9b67bf4e4f5faa785d98105
Reviewed-on: https://go-review.googlesource.com/c/go/+/174318
Reviewed-by: Ian Lance Taylor <iant@golang.org>
runtime/cgo: ignore missing Info.plist files on iOS
When running Go programs on Corellium virtual iPhones, the Info.plist
files might not exist. Ignore the error.

Updates golang#31722

Change-Id: Id2e315c09346b69dda9e10cf29fb5dba6743aac4
Reviewed-on: https://go-review.googlesource.com/c/go/+/174202
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
syscall: don't return EINVAL on zero Chmod mode on Windows
Fixes golang#20858

Change-Id: I45c397795426aaa276b20f5cbeb80270c95b920c
Reviewed-on: https://go-review.googlesource.com/c/go/+/174320
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
testing: delay flag registration; move to an Init function
Any code that imports the testing package forces the testing flags to be
defined, even in non-test binaries. People work around this today by
defining a copy of the testing.TB interface just to avoid importing
testing.

Fix this by moving flag registration into a new function, testing.Init.
Delay calling Init until the testing binary begins to run, in
testing.MainStart.

Init is exported for cases where users need the testing flags to be
defined outside of a "go test" context. In particular, this may be
needed where testing.Benchmark is called outside of a test.

Fixes golang#21051

Change-Id: Ib7e02459e693c26ae1ba71bbae7d455a91118ee3
Reviewed-on: https://go-review.googlesource.com/c/go/+/173722
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
runtime: make mmap return 0 instead of -1 on aix/ppc64
Most of the platforms are returning 0 instead of -1 when mmap syscalls
is failing. This patch corrects it for AIX in order to fix
TestMmapErrorSign and to improve AIX compatibility.

Change-Id: I1dad88d0e69163ad55c504b2b4a997892fd876cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/174297
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
cmd/go: add XCOFF format handler for go version
Change-Id: Ib102ae95acfd89fc3c9942a4ec82c74362f62045
Reviewed-on: https://go-review.googlesource.com/c/go/+/174299
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
runtime: account for callbacks in checkdead on Windows
When a callback runs on a different thread in Windows, as in the
runtime package test TestCallbackInAnotherThread, it will use the
extra M. That can cause the test in checkdead to fail incorrectly.
Check whether there actually is an extra M before expecting it.

I think this is a general problem unrelated to timers. I think the test
was passing previously because the timer goroutine was using an M.
But I haven't proved that. This change seems correct, and it avoids
the test failure when using the new timers on Windows.

Updates golang#27707

Change-Id: Ieb31c04ff0354d6fae7e173b59bcfadb8b0464cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/174037
Reviewed-by: Keith Randall <khr@golang.org>
cmd,runtime: enable cgo for openbsd/arm64
Updates golang#31656.

Change-Id: Ide6f829282fcdf20c67998b766a201a6a92c3035
Reviewed-on: https://go-review.googlesource.com/c/go/+/174132
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
cmd/compile: evaluate map initializers incrementally
For the code:

m := map[int]int {
  a(): b(),
  c(): d(),
  e(): f(),
}

We used to do:

t1 := a()
t2 := b()
t3 := c()
t4 := d()
t5 := e()
t6 := f()
m := map[int]int{}
m[t1] = t2
m[t3] = t4
m[t5] = t6

After this CL we do:

m := map[int]int{}
t1 := a()
t2 := b()
m[t1] = t2
t3 := c()
t4 := d()
m[t3] = t4
t5 := e()
t6 := f()
m[t5] = t6

Ordering the initialization this way limits the lifetime of the
temporaries involved.  In particular, for large maps the number of
simultaneously live temporaries goes from ~2*len(m) to ~2. This change
makes the compiler (regalloc, mostly) a lot happier. The compiler runs
faster and uses a lot less memory.

For golang#26546, changes compile time of a big map from 8 sec to 0.5 sec.

Fixes golang#26552

Update golang#26546

Change-Id: Ib7d202dead3feaf493a464779fd9611c63fcc25f
Reviewed-on: https://go-review.googlesource.com/c/go/+/174417
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
cmd/go: add test of $GONOPROXY, $GONOSUMDB behavior
Change-Id: I8a4917ce14ea22d5991226e485d43a9c9312950e
Reviewed-on: https://go-review.googlesource.com/c/go/+/174219
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
encoding/json: fix Unmarshal hang on recursive pointers
indirect walks down v until it gets to a non-pointer. But it does not
handle the case when v is a pointer to itself, like in:

	var v interface{}
	v = &v
	Unmarshal(b, v)

So just stop immediately if we see v is a pointer to itself.

Fixes golang#31740

Change-Id: Ie396264119e24d70284cd9bf76dcb2050babb069
Reviewed-on: https://go-review.googlesource.com/c/go/+/174337
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

randall77 and others added some commits May 13, 2019

cmd/compile: make sure to initialize static entries of slices
If a slice's entries are sparse, we decide to initialize it dynamically
instead of statically. That's CL 151319.

But if we do initialize it dynamically, we still need to initialize
the static entries. Typically we do that, but the bug fixed here is
that we don't if the entry's value is itself an array or struct.

To fix, use initKindLocalCode to ensure that both static and
dynamic entries are initialized via code.

Fixes golang#31987

Change-Id: I1192ffdbfb5cd50445c1206c4a3d8253295201dd
Reviewed-on: https://go-review.googlesource.com/c/go/+/176904
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
cmd/compile: index line number tables by source file to improve sparsity
This reduces allocations and also resolves some
lurking inliner/inlinee line-number match problems.
However, it does add about 1.5% to compile time.

This fixes compiler OOMs seen compiling some large protobuf-
derived inputs.  For compiling the compiler itself,

compilebench -pkg cmd/compile/internal/ssa -memprofile withcl.prof

the numberlines-related memory consumption is reduced from 129MB
to 29MB (about a 5% overall reduction in allocation).

Additionally modified after going over changes with Austin
to remove unused code (nobody called size()) and correct
the cache-clearing code.

I've attempted to speed this up by not using maps, and have
not succeeded.  I'd rather get correct code in now, speed it
up later if I can.

Updates golang#27739.
Fixes golang#29279.

Change-Id: I098005de4e45196a5f5b10c0886a49f88e9f8fd5
Reviewed-on: https://go-review.googlesource.com/c/go/+/154617
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
cmd/internal/obj/x86: fix oclass tests for Hsolaris
Use objabi.Hlinux for now.

Fixes golang#32028

Change-Id: If9745f72c0ee4444ea2a2faa50813d2e1ac2bf97
Reviewed-on: https://go-review.googlesource.com/c/go/+/177077
Reviewed-by: Cherry Zhang <cherryyz@google.com>
cmd/dist: make GOROOT unwritable on builders
Updates golang#30316

Change-Id: I57e489f6bbe4a3b39c907dabe5ac41fb9939cdb4
Reviewed-on: https://go-review.googlesource.com/c/go/+/163477
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
spec: clarify that slice a expression shares underlying array with op…
…erand

The spec was not very precise as to what happens with respect to sharing
if a sliced operand is (a pointer to) an array. Added a small clarification
and a supporting example.

Fixes golang#31689.

Change-Id: Ic49351bec2033abd3f5428154ec3e9a7c2c9eaa5
Reviewed-on: https://go-review.googlesource.com/c/go/+/177139
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
cmd/go/internal/renameio: allow write file with the specified permiss…
…ions

Now renameio package creates file use ioutil.TempFile, which calls
OpenFile with mode 0600, we should support creates a file with given
permission bits.

Fixes golang#31871

Change-Id: I0436e9f7081f2fce18bf9f3b14d55b02d4d995fb
Reviewed-on: https://go-review.googlesource.com/c/go/+/175958
Run-TryBot: Baokun Lee <nototon@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
regexp: optimize for provably too short inputs
For many patterns we can compute the minimum input length at compile time.
If the input is shorter, we can return early and get a huge speedup.

As pointed out by Damian Gryski, Perl's regex engine contains a number of
these kinds of fail-fast optimizations:
https://perldoc.perl.org/perlreguts.html#Peep-hole-Optimisation-and-Analysis

Benchmarks: (including new ones for compile time)

name               old time/op    new time/op    delta
Compile/Onepass-8    4.39µs ± 1%    4.40µs ± 0%  +0.34%  (p=0.029 n=9+8)
Compile/Medium-8     9.80µs ± 0%    9.91µs ± 0%  +1.17%  (p=0.000 n=10+10)
Compile/Hard-8       72.7µs ± 0%    73.5µs ± 0%  +1.10%  (p=0.000 n=9+10)

name                       old time/op    new time/op      delta
Match/Easy0/16-8             52.6ns ± 5%       4.9ns ± 0%     -90.68%  (p=0.000 n=10+9)
Match/Easy0/32-8             64.1ns ±10%      61.4ns ± 1%        ~     (p=0.188 n=10+9)
Match/Easy0/1K-8              280ns ± 1%       277ns ± 2%      -0.97%  (p=0.004 n=10+10)
Match/Easy0/32K-8            4.61µs ± 1%      4.55µs ± 1%      -1.49%  (p=0.000 n=9+10)
Match/Easy0/1M-8              229µs ± 0%       226µs ± 1%      -1.29%  (p=0.000 n=8+10)
Match/Easy0/32M-8            7.50ms ± 1%      7.47ms ± 1%        ~     (p=0.165 n=10+10)
Match/Easy0i/16-8             533ns ± 1%         5ns ± 2%     -99.07%  (p=0.000 n=10+10)
Match/Easy0i/32-8             950ns ± 0%       950ns ± 1%        ~     (p=0.920 n=10+9)
Match/Easy0i/1K-8            27.5µs ± 1%      27.5µs ± 0%        ~     (p=0.739 n=10+10)
Match/Easy0i/32K-8           1.13ms ± 0%      1.13ms ± 1%        ~     (p=0.079 n=9+10)
Match/Easy0i/1M-8            36.7ms ± 2%      36.1ms ± 0%      -1.64%  (p=0.000 n=10+9)
Match/Easy0i/32M-8            1.17s ± 0%       1.16s ± 1%      -0.80%  (p=0.004 n=8+9)
Match/Easy1/16-8             55.5ns ± 6%       4.9ns ± 1%     -91.19%  (p=0.000 n=10+9)
Match/Easy1/32-8             58.3ns ± 8%      56.6ns ± 1%        ~     (p=0.449 n=10+8)
Match/Easy1/1K-8              750ns ± 0%       748ns ± 1%        ~     (p=0.072 n=8+10)
Match/Easy1/32K-8            31.8µs ± 0%      31.6µs ± 1%      -0.50%  (p=0.035 n=10+9)
Match/Easy1/1M-8             1.10ms ± 1%      1.09ms ± 0%      -0.95%  (p=0.000 n=10+9)
Match/Easy1/32M-8            35.5ms ± 0%      35.2ms ± 1%      -1.05%  (p=0.000 n=9+10)
Match/Medium/16-8             442ns ± 2%         5ns ± 1%     -98.89%  (p=0.000 n=10+10)
Match/Medium/32-8             875ns ± 0%       878ns ± 1%        ~     (p=0.071 n=9+10)
Match/Medium/1K-8            26.1µs ± 0%      25.9µs ± 0%      -0.64%  (p=0.000 n=10+10)
Match/Medium/32K-8           1.09ms ± 1%      1.08ms ± 0%      -0.84%  (p=0.000 n=10+9)
Match/Medium/1M-8            34.9ms ± 0%      34.6ms ± 1%      -0.98%  (p=0.000 n=9+10)
Match/Medium/32M-8            1.12s ± 1%       1.11s ± 1%      -0.98%  (p=0.000 n=10+9)
Match/Hard/16-8               721ns ± 1%         5ns ± 0%     -99.32%  (p=0.000 n=10+9)
Match/Hard/32-8              1.32µs ± 1%      1.31µs ± 0%      -0.71%  (p=0.000 n=9+9)
Match/Hard/1K-8              39.8µs ± 1%      39.7µs ± 1%        ~     (p=0.165 n=10+10)
Match/Hard/32K-8             1.57ms ± 0%      1.56ms ± 0%      -0.70%  (p=0.000 n=10+9)
Match/Hard/1M-8              50.4ms ± 1%      50.1ms ± 1%      -0.57%  (p=0.007 n=10+10)
Match/Hard/32M-8              1.62s ± 1%       1.60s ± 0%      -0.98%  (p=0.000 n=10+10)
Match/Hard1/16-8             3.88µs ± 1%      3.86µs ± 0%        ~     (p=0.118 n=10+10)
Match/Hard1/32-8             7.44µs ± 1%      7.46µs ± 1%        ~     (p=0.109 n=10+10)
Match/Hard1/1K-8              232µs ± 1%       229µs ± 1%      -1.31%  (p=0.000 n=10+9)
Match/Hard1/32K-8            7.41ms ± 2%      7.41ms ± 0%        ~     (p=0.237 n=10+8)
Match/Hard1/1M-8              238ms ± 1%       238ms ± 0%        ~     (p=0.481 n=10+10)
Match/Hard1/32M-8             7.69s ± 1%       7.61s ± 0%      -1.00%  (p=0.000 n=10+10)

Fixes golang#31329

Change-Id: I04640e8c59178ec8b3106e13ace9b109b6bdbc25
Reviewed-on: https://go-review.googlesource.com/c/go/+/171023
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
cmd/vendor: go get -u golang.org/x/tools && go mod vendor
Picks up vet fix from CL 176357.

Change-Id: Ia77cd4a582c4edfbe59bbc311e6ce14046df0e83
Reviewed-on: https://go-review.googlesource.com/c/go/+/177137
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
cmd/go: do not build test packages unnecessarily during go vet
Vet needs export data for the imports of the package it is analyzing.
Vet does not need export data for the package itself, since vet will
do its own type checking. Assuming that vet is just as good as the compiler
at detecting invalid programs, don't run the compiler unnecessarily.

This especially matters for tests without external test files or for
which the external test files do not import the test-augmented original
package. In that case, the test-augmented original package need not
be compiled at all.

Cuts time for 'go clean -cache && go vet -x cmd/compile/internal/ssa'
from 7.6r 24.3u 2.8s to 3.5r 8.5u 1.9s, by not running the compiler
on the augmented test package.

There is still more to be done here - if we do need to build a
test-augmented package, we rerun cgo unnecessarily.
But this is a big help.

Cuts time for 'go vet std cmd' by about 30%.

For golang#31916.

Change-Id: If6136b4d384f1da77aed90b43f1a6b95f09b5d86
Reviewed-on: https://go-review.googlesource.com/c/go/+/176438
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
syscall: implement rawVforkSyscall for linux/ppc64x and linux/s390x
This allows the use of CLONE_VFORK and CLONE_VM for fork/exec, preventing
"fork/exec ...: cannot allocate memory" failures from occuring when attempting
to execute commands from a Go process that has a large memory footprint.
Additionally, this should reduce the latency of fork/exec on these platforms.

The same problem was addressed on linux/amd64 via issue golang#5838.

Updates golang#31936

Change-Id: I7ae0fbbeaa29cab944a49a11272a380d497eb2d0
Reviewed-on: https://go-review.googlesource.com/c/go/+/175697
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
crypto/x509: add support for Ed25519 certificates and keys
Based on RFC 8410.

Updates golang#25355

Change-Id: If7abb7eeb0ede10a9bb3d2004f2116e587c6207a
Reviewed-on: https://go-review.googlesource.com/c/go/+/175478
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
errors, fmt: revert rejected changes for Go 1.13
Reverts the following changes:

  https://go.googlesource.com/go/+/1f90d081391d4f5911960fd28d81d7ea5e554a8f
  https://go.googlesource.com/go/+/8bf18b56a47a98b9dd2fa03beb358312237a8c76
  https://go.googlesource.com/go/+/5402854c3557f87fa2741a52ffc15dfb1ef333cc
  https://go.googlesource.com/go/+/37f84817247d3b8e687a701ccb0d6bc7ffe3cb78
  https://go.googlesource.com/go/+/6be6f114e0d483a233101a67c9644cd72bd3ae7a

Partially reverts the followinng change, removing the errors.Opaque
function and the errors.Wrapper type definition:

  https://go.googlesource.com/go/+/62f5e8156ef56fa61e6af56f4ccc633bde1a9120

Updates documentation referencing the Wrapper type.

Change-Id: Ia622883e39cafb06809853e3fd90b21441124534
Reviewed-on: https://go-review.googlesource.com/c/go/+/176997
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
fmt: support %w
When fmt.Errorf is provided with a %w verb with an error operand,
return an error implementing an Unwrap method returning that operand.

It is invalid to use %w with other formatting functions, to use %w
multiple times in a format string, or to use %w with a non-error
operand. When the Errorf format string contains an invalid use of %w,
the returned error does not implement Unwrap.

Change-Id: I534e20d3b163ab22c2b137b1c9095906dc243221
Reviewed-on: https://go-review.googlesource.com/c/go/+/176998
Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
runtime: fix 'go vet -race runtime'
This updates the Go function declarations to match race_amd64.s.

Change-Id: I2b541a6b335ce732f4c31652aa615240ce7bb1c3
Reviewed-on: https://go-review.googlesource.com/c/go/+/177397
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
cmd/go: run full 'go vet' during 'go test' for packages in GOROOT
Now that the main tree complies with 'go vet', enable all vet checks
during 'go test' in the main tree. This helps surface helpful errors
while developing, instead of having to wait for the misc-vet-vetall builder.

During 'go test', the additional vet checks are essentially free:
the vet invocations themselves take only 8 seconds total for the entire tree.

Also update buildall.bash (used by the misc-compile builders)
to run 'go vet std cmd' for each GOOS/GOARCH pair.
This is not as free, since in general it can require recompiling
packages with their tests included before invoking vet.
(That compilation was going on anyway in the 'go test' case.)

On my Mac laptop, ./buildall.bash freebsd used to take
68+16+17+18 = 119 seconds for make.bash and then
the builds of the three freebsd architectures.
Now it takes 68+16+23+17+23+18+24 = 189 seconds, 60% longer.
Some of this is spent doing unnecessary cgo work.
Still, this lets us shard the vet checks and match all.bash.

Fixes golang#20119.
For golang#31916.

Change-Id: I6b0c40bac47708a688463c7fca12c0fc23ab2751
Reviewed-on: https://go-review.googlesource.com/c/go/+/176439
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
cmd/vet/all: delete
The work of running full vet on std and cmd during local development
has moved to go test, which of course runs during all.bash.

For errors in other GOOS/GOARCH combinations, the misc-compile
builders (running buildall.bash) also now run go vet std cmd.

The vetall builder need not do anything anymore.
Make it a no-op until it can be retired, and remove
cmd/vet/all and its special case in the go command.

Fixes golang#31916.

Change-Id: I8f30d184c382ea7c2c8f520e5618f680db633968
Reviewed-on: https://go-review.googlesource.com/c/go/+/176440
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
cmd/go: add more information to -debug-actiongraph
Add information about action and build IDs,
commands run, time spent in commands,
and overall time to -debug-actiongraph output.

Also avoid a terrible outcome in:

	go build -debug-actiongraph x.go

Change-Id: I7a3a638f4848d75f2bdc997517f4ab23656da4f1
Reviewed-on: https://go-review.googlesource.com/c/go/+/177138
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
cmd/go: rename renamed testing import
_go_testing is equally unlikely and much less concerning
than testing_xxxxxxxxxxxx if it appears in an error message
(as it does, for example, in https://storage.googleapis.com/go-build-log/0d543f89/linux-amd64_3467a10e.log).

Change-Id: I45dc3ebe2d3b6c9e53273cd21782ee11a53f5edb
Reviewed-on: https://go-review.googlesource.com/c/go/+/177197
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
cmd/go: cut 'go test -short cmd/go' time by about half
Was 50 seconds on unloaded Mac laptop; now 27.
Still longer than I would like, but every little bit helps.

For golang#26473.

Change-Id: Id4be016ee1555cbc3512eca0ae10236d7f06bd02
Reviewed-on: https://go-review.googlesource.com/c/go/+/177398
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
runtime: disable some tests in -quick mode
Speeds up the "go test runtime -cpu=1,2,4 -short -quick" phase of all.bash.

For golang#26473.

Change-Id: I090f5a5aa754462b3253a2156dc31fa67ce7af2a
Reviewed-on: https://go-review.googlesource.com/c/go/+/177399
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
misc/cgo: disable testcarchive, testplugin during -short test (includ…
…ing all.bash)

These tests take 20 seconds each to rebuild the entire world
in their respective modes. That's too much for short mode,
including all.bash on developer machines.

Keep doing it on builders and if someone runs 'go test' by hand
in that directory.

For golang#26473.

Change-Id: I3dc6955bc3aa7a20fd170efcde72a7d19b37bdbf
Reviewed-on: https://go-review.googlesource.com/c/go/+/177417
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
cmd/dist: say 'go test .' instead of 'go test' in a few places for cl…
…eaner output

This just makes all.bash a bit less chatty.

Change-Id: I7d2ecabf0c7d8df2065d7052718f611bb2907801
Reviewed-on: https://go-review.googlesource.com/c/go/+/177418
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
testing: panic on calls to Short/Verbose before Parse
CL 121936 added this diagnostic to avoid a panic accessing *short.
(Hence the "This shouldn't really be a panic" comment.)

That CL was right to produce a clearer error than a plain memory fault,
but I think wrong to print+exit instead of panicking. I just ran into
one of these in a real program, and there is no indication anywhere
of how the program reached this point. The panic will show that.
So change print+exit to a panic with a helpful message, in contrast
to the original panic with an unhelpful message and the current
helpful message without stack trace.

Change-Id: Ib2bae1dead4ccde92f00fa3a34c05241ff7690c6
Reviewed-on: https://go-review.googlesource.com/c/go/+/177419
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
testing: shorten go test -short testing
This cuts the time for 'go test -short testing' from 0.9s to < 0.1s.

Change-Id: Ib8402f80239e1e96ea5221dfd5cd0db08170d85b
Reviewed-on: https://go-review.googlesource.com/c/go/+/177420
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
cmd/compile: add debugging and stats output to numberlines
This is useful for debugging line number assignment and
also for making sense of pathological line number inputs.

Activated with
-gcflags=-d=ssa/number_lines/stats=1 (the bit matters)
-gcflags=-d=ssa/number_lines/debug

Stats:
"SUM_LINE_RANGE",
   SUM for f in files {MAX line in f {line}-MIN line in f {line}}
"MAXMIN_LINE_RANGE",
   MAX for f in files {MAX line in f {line}} -
   MIN for f in files {MIN line in f {line}}
"MAXFILE", maxfile,
   MAX for f in files {f}
"NFILES", len(entries)
   | files |

Change-Id: I8a7336e6370452fe2e3a62de17606db9bd6a6fd3
Reviewed-on: https://go-review.googlesource.com/c/go/+/174947
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
os: pass correct environment when creating Windows processes
This is CVE-2019-11888.

Previously, passing a nil environment but a non-nil token would result
in the new potentially unprivileged process inheriting the parent
potentially privileged environment, or would result in the new
potentially privileged process inheriting the parent potentially
unprivileged environment. Either way, it's bad. In the former case, it's
an infoleak. In the latter case, it's a possible EoP, since things like
PATH could be overwritten.

Not specifying an environment currently means, "use the existing
environment". This commit amends the behavior to be, "use the existing
environment of the token the process is being created for." The behavior
therefore stays the same when creating processes without specifying a
token. And it does the correct thing when creating processes when
specifying a token.

Fixes golang#32000

Change-Id: Ia57f6e89b97bdbaf7274d6a89c1d9948b6d40ef5
Reviewed-on: https://go-review.googlesource.com/c/go/+/176619
Run-TryBot: Jason Donenfeld <Jason@zx2c4.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>

@Kiku-Reise Kiku-Reise merged commit 3b699ee into Kiku-Reise:master May 16, 2019

@AlekSi

This comment has been minimized.

Copy link
Contributor

commented on 3e2c522 May 17, 2019

Is there any public discussion about reasons for that revert? https://go-review.googlesource.com/c/go/+/176997 doesn't have it IIUC.

Edit: #29934 (comment)

This comment has been minimized.

Copy link
Contributor

replied May 17, 2019

@AlekSi We don't use GitHub for code review. Very few people will see this comment. If you want to ask a question about a change, please comment on Gerrit, in this case at https://golang.org/cl/176997. Or ask on the golang-nuts mailing list. Thanks.

This comment has been minimized.

Copy link
Contributor Author

replied May 17, 2019

@AlekSi The decision on which portion of these changes to retain was announced here #29934 (comment) (buried in the issue comments, but there's a link to from the issue head). There's some follow-up discussion there.

@AlekSi

This comment has been minimized.

Copy link
Contributor

commented on src/errors/wrap.go in 3e2c522 May 17, 2019

@neild Here Wrapper type is still referenced. (Sorry, can't send CL myself)

This comment has been minimized.

Copy link
Contributor Author

replied May 17, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.