cmd/compile: avoid memclr for provable overwritten slices

[dsnet]

Consider the following pattern:

var n int
for _, src := range source {
	n += len(src)
}
b := make([]byte, 0, n)
for _, src := range source {
	b = append(b, src...)
}

which often occurs when merging segmented buffers.

Running this benchmark shows that the CPU time is dominated by runtime.memclrNoHeapPointers, which occupies 44% of the time, while runtime.memmove occupies 36% of the time (this is possibly faster than memclr due to caching effects from the former function).

The compiler should be able to prove that every byte of this slice is overwritten, and thus avoid the call to memclr entirely.

If detecting this pattern is too difficult, we could instead special-case bytes.Join such that the internal call to make avoids the memclr (perhaps by calling a runtime-internal variant of make that avoids zeroing). Thus, the pattern above could be replaced with bytes.Join(source, nil).

\cc @randall77 @martisch @lavalamp

[zigo101]

It is hard to implement this for all cases. But it might be a good idea to do some special optimizations for the later slices std package.

[martisch]

Generally I would not want to open the Pandora's box of giving std lib functions like bytes.Join special runtime access. Lets make the pattern simple enough to detect and then bytes.Join can have the exact pattern so programmers dont need to remember it.

I suggest

var b []byte
for _, src := range source {
	b = append(b, src...)
}

We can teach the compiler to detect that pattern and avoid unnecessary memclr (but only for non pointer types) as well as do only one allocation for the full b slice with enough room for all appends. This avoids the explicit len computation of b with the same effect of doing only one allocation.

To make it work for bytes.Join we can then expand this to allow an append of an unchanged variable or const in addition in the loop:

for _, src := range source {
	b = append(b, src...)
        b = append(b, var|const...)
}

Then join would be:

for _, src := range source {
	b = append(b, src...)
        b = append(b, var|const...)
}
b = b[:len(b)-len(var|const)]

Yes that over allocates by len(var|const) amount but if that matters we can always fine tune that later to detect that too and avoid it. If we start this lets have simple pattern detection first and expand to more versions when there is sufficient evidence its widely used and has high impact on performance.

[zigo101]

Another solution is to add a builtin concat (or merge) function to do the job (the need is common and the builtin slice manipulations are incomplete now).

Some related issues: #23905, #29186, #24926

[lavalamp]

Re:

var b []byte
for _, src := range source {
	b = append(b, src...)
}

Requiring that to be the canonical pattern has the weird property that someone pre-allocating with the "right" size will get worse performance.

When I encountered this my first wish was for append to solve the problem via a double definition:

func append[T any](base []T, args... T)
func append[T any](base []T, args... []T)

(I'm not claiming that's a great solution, just a data point about what one go-user would have found intuitive)

[martisch]

Requiring that to be the canonical pattern has the weird property that someone pre-allocating with the "right" size will get worse performance.

One can after the first iteration that only accepts new slices change to just match:

for _, src := range source {
	b = append(b, src...)
}

and have some checks around existing vs needed capacity that determines how the allocation is made. However it also complicates some things like when b has free capacity but not sufficient much. We then still need to update b with as many copies of slices as would have happened in the append loop, before allocating a new backing array, which requires more care and machinery.

I would still suggest we start with something more restricted and then iterate to be less restrictive than reaching for something more versatile but more complex right away.

func append[T any](base []T, args... []T)

I agree that a dedicated inbuild instead of a pattern makes detection easier and the intention clearer. A language change on the other side has a much higher bar to get accepted. There had been discussions in the past to allow a variadic list of slices to append but afaik they get negative feedback on the grounds of append shouldnt be an even more special language construct.

[lavalamp]

It seems like an alternative approach might be "lazy clearing" -- track an index i such that everything prior to that index doesn't need clearing and everything after it is uninitialized.

After an allocation just avoid clearing the remaining part of the array until one of these conditions is met:

• A read from the uncleared part of the array
• A write that doesn't start at or before i (and in this case i is incremented to become the end of the written data)
• The end of the function
• A reference to the array potentially escapes

[gopherbot]

Change https://go.dev/cl/456336 mentions this issue: bytes, strings: avoid unnecessary zero initialization

[dsnet]

Generally I would not want to open the Pandora's box of giving std lib functions like bytes.Join special runtime acces

Heh... that's exactly the approach I took in https://go.dev/cl/456336. I personally think it's fairly palatable since bytes and strings already depend on unsafe operations (especially through the internal/bytealg) package.

As part of that CL, I also optimized strings.Builder, which will have many downstream benefits. The way strings.Builder uses the []byte would be very difficult for the compiler to prove that the unused capacity never leaks to the end user.

In general, this provides 2x performance gains for sufficiently large buffers. This makes sense since we're removing an entire write pass through the memory.

[zigo101]

func append[T any](base []T, args... []T)

has ambiguity if the type argument for T is an interface type.

[dsnet]

good idea to do some special optimizations for the later slices std package.

I don't think this belongs in the slices package since this optimization only works for element types that do not contain pointers. There would need to be non-trivial runtime magic in the slices package to make this work (even more so than the internal/bytealg approach I took in https://go.dev/cl/456336).

[prattmic]

cc @golang/compiler

[martisch]

I personally think it's fairly palatable since bytes and strings already depend on unsafe operations (especially through the internal/bytealg) package.

Using unsafe and assembly is not the same as giving special access into the runtime by linknaming. Yes there are other packages that do that like reflect but I dont think bytes and strings are in the same category of being tied to internals. Additionally the bytes.MakeNoZero solves this for one specific type which is also not ideal. This will invite unsafe casts to other types.

Using internal/bytealg will also mean the functions are not copyable elsewhere as the dependency will not be met for anything but the std lib. This gives these functions special access that cant even be gained with unsafe and therefore std lib will win in performance without others just being able to copy the code and keep the same performance.

What I mean with pandorax box is that this invites adding linknames from std packages everywhere into the runtime to special operations for optimizations. "More efficient map copy" -> create a map package and link it to special function in runtime. Thats not to say we should not optimize the functions but I think we should do so by patterns detected in the compiler and replaced by runtime calls, or through builtins and not through linknaming which is not portable/maintainable within the go language as others can not just add their linknames into the runtime.