os: zero copy pipe splice falling back to read+write

Go version

go version go1.26.5 linux/amd64

Output of `go env` in your module/workspace:

AR='ar'
CC='gcc'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='g++'
GCCGO='gccgo'
GO111MODULE=''
GOAMD64='v1'
GOARCH='amd64'
GOAUTH='netrc'
GOBIN=''
GOCACHE='/home/arusekk/.cache/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/home/arusekk/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -m64 -pthread -fmessage-length=0 -ffile-prefix-map=/tmp/go-build992021822=/tmp/go-build -gno-record-gcc-switches'
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMOD='/dev/null'
GOMODCACHE='/home/arusekk/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/arusekk/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/lib/go'
GOSUMDB='sum.golang.org'
GOTELEMETRY='local'
GOTELEMETRYDIR='/home/arusekk/.config/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/lib/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.26.5'
GOWORK=''
PKG_CONFIG='pkg-config'

What did you do?

Running the following program:

package main

import (
        "fmt"
        "io"
        "os"
        "os/exec"
)

func main() {
        cmd := exec.Command("seq", "1", "20")

        out, err := cmd.StdoutPipe()
        if err != nil {
                fmt.Printf("cmd.StdoutPipe err: %v\n", err)
                return
        }
        if err := cmd.Start(); err != nil {
                fmt.Printf("cmd.Start err: %v\n", err)
                return
        }

        n, err := io.Copy(os.Stdout, out)
        if err != nil {
                fmt.Printf("io.Copy err: %v\n", err)
                return
        }
        fmt.Printf("io.Copy: %v\n", n)
}

go build pipesplice.go

then

strace -yy -eread,write,splice,sendfile,copy_file_range ./tryit | cat

My kernel version is 6.18.43_1; the |cat is optional, splice works equally fine for pipe->pipe and pipe->terminal (and pipe->devnull for that matter).
This problem affects workloads with heavy use of io.Copy.

What did you see happen?

copy_file_range(3<pipe:[675474]>, NULL, 1<pipe:[678331]>, NULL, 2147479552, 0) = -1 EINVAL (Invalid argument)
read(3<pipe:[675474]>, "1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\n13\n14"..., 32768) = 51
write(1<pipe:[678331]>, "1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11\n12\n13\n14"..., 51) = 51
read(3<pipe:[675474]>1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
, "", 32768)       = 0
write(1<pipe:[678331]>, "io.Copy: 51\n", 12io.Copy: 51
) = 12
+++ exited with 0 +++

What did you expect to see?

After failed copy_file_range I expected Go runtime to try zero-copy using splice, producing a trace like the following:

copy_file_range(3<pipe:[675474]>, NULL, 1<pipe:[678331]>, NULL, 2147479552, 0) = -1 EINVAL (Invalid argument)
splice(3<pipe:[675474]>, NULL, 1<pipe:[678331]>, NULL, 2147479552, 0) = 51
write(1<pipe:[678331]>, "io.Copy: 51\n", 12io.Copy: 51
) = 12
+++ exited with 0 +++

Related Issues

os: `File.ReadFrom` may busy-loop when source is a `net.Conn` #75304
os: io.Copy/ReadFrom uses copy_file_range when destination has O_APPEND set post-open, returns EBADF instead of falling back #79660
os: copy_file_range system call fails on some file systems #44272 (closed)
internal/poll: Splice() creates and destroys pipes in high frequency, wasting system resources #42740 (closed)
io: Copy to a pipe prevents process exit (Go 1.24rc2 on Linux regression) #71375 (closed)

Related Code Changes

os: don't use sendfile to a pipe on Linux

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

@panjf2000

see this comment from @panjf2000
https://go.googlesource.com/go/+/7237c87c8c679d9693c79e3d7d1efe390bc2249d/src/os/zero_copy_linux.go#69

@panjf2000

We can do better there. The code handles network streams but not pipes. It could handle pipes too, which should fix this case. CC @panjf2000

Why does it matter? Like I said in those comments, I haven't run any benchmarks on the pipe <-> pipe case, but I doubted that splice would help this case.

@Arusekk

@Arusekk
Why are you convinced that splice would help your case? Did you run benchmarks on splice against read+write? If so, please share your numbers, it would help us decide, thx.

Using the following (utterly ugly) code I am consistently getting a ~1.3x improvement copying 64 GiB of zeros to /dev/null:

package main

import (
	"errors"
	"fmt"
	"io"
	"os"
	"os/exec"
	"syscall"
)

func waitForReadable(fd int) error {
	var rfds syscall.FdSet
	rfds.Bits[fd / 64] |= 1 << fd%64

	nfd := fd + 1

	n, err := syscall.Select(nfd, &rfds, nil, nil, nil)
	if err != nil {
		return err
	}
	if n == 0 {
		return fmt.Errorf("timeout")
	}

	if rfds.Bits[fd/64] & (1 << fd%64) != 0 {
		return nil
	}
	return fmt.Errorf("select returned but fd not ready")
}

func main() {
	cmd := exec.Command("dd", "if=/dev/zero", "bs=64K", "count=1048576")

	out, err := cmd.StdoutPipe()
	if err != nil {
		fmt.Fprintf(os.Stderr, "cmd.StdoutPipe err: %v\n", err)
		return
	}
	if err := cmd.Start(); err != nil {
		fmt.Errorf("cmd.Start err: %v\n", err)
		return
	}

	if false {
		n, err := io.Copy(os.Stdout, out)
		if err != nil {
			fmt.Fprintf(os.Stderr, "io.Copy err: %v\n", err)
			return
		}
		fmt.Fprintf(os.Stderr, "io.Copy: %v\n", n)
	} else {
		var total int64
		for {
			n, err := syscall.Splice(3, nil, 1, nil, 2147479552, 0)
			if n == 0 {
				break
			}
			if err != nil {
				if errors.Is(err, syscall.EINTR) {
					continue
				}
				if errors.Is(err, syscall.EAGAIN) {
					for {
						err = waitForReadable(3)
						if err != nil {
							if errors.Is(err, syscall.EINTR) {
								continue
							}
							fmt.Fprintf(os.Stderr, "wFR: %v\n", err)
							return
						}
						break
					}
					continue
				}
				fmt.Fprintf(os.Stderr, "syscall.Splice err: %v\n", err)
				return
			}
			total += n
		}
		fmt.Fprintf(os.Stderr, "syscall.Splice: %v\n", total)
	}
}

[arusekk@fw13 gotest]$ ./rw |sha1sum
io.Copy: 68719476736
8a097071fc8aaa845f9dda2b99df61eaa6516109  -
[arusekk@fw13 gotest]$ ./sp |sha1sum
syscall.Splice: 68719476736
8a097071fc8aaa845f9dda2b99df61eaa6516109  -
[arusekk@fw13 gotest]$ time ./rw >/dev/null
io.Copy: 68719476736

real    0m13,171s
user    0m1,531s
sys     0m11,503s
[arusekk@fw13 gotest]$ time ./rw >/dev/null
io.Copy: 68719476736

real    0m12,452s
user    0m1,572s
sys     0m10,765s
[arusekk@fw13 gotest]$ time ./rw >/dev/null
io.Copy: 68719476736

real    0m12,808s
user    0m1,499s
sys     0m11,188s
[arusekk@fw13 gotest]$ time ./rw >/dev/null
io.Copy: 68719476736

real    0m12,497s
user    0m1,601s
sys     0m10,777s
[arusekk@fw13 gotest]$ time ./rw >/dev/null
io.Copy: 68719476736

real    0m12,895s
user    0m1,504s
sys     0m11,268s
[arusekk@fw13 gotest]$ time ./sp >/dev/null
syscall.Splice: 68719476736

real    0m9,320s
user    0m0,469s
sys     0m7,989s
[arusekk@fw13 gotest]$ time ./sp >/dev/null
syscall.Splice: 68719476736

real    0m9,931s
user    0m0,538s
sys     0m8,425s
[arusekk@fw13 gotest]$ time ./sp >/dev/null
syscall.Splice: 68719476736

real    0m10,371s
user    0m0,555s
sys     0m9,148s
[arusekk@fw13 gotest]$ time ./sp >/dev/null
syscall.Splice: 68719476736

real    0m10,559s
user    0m0,527s
sys     0m9,556s
[arusekk@fw13 gotest]$ time ./sp >/dev/null
syscall.Splice: 68719476736

real    0m10,462s
user    0m0,611s
sys     0m9,281s

So the science is roughly:

T	before	after	Δ
real	12.7646 (±0.2658)	10.1286 (±0.4578)	-21% (±4 pp) ≈ 1.3x better
user	1.5414 (±0.0395)	0.54 (±0.0458)	-65% (±3 pp) ≈ 2.9x better
sys	11.1002 (±0.2881)	8.8798 (±0.5815)	-20% (±6 pp) ≈ 1.3x better

Hope that helps.

Thanks for benchmarks, these numbers seem promising.
I'll try to do more benchmarks, if nothing goes wrong, I will send a CL for this.

It turns out that we didn't use splice or sendfile on pipes for a reason, check out #71375 for more context.

To sum up, there are too many quirks when using splice/sendfile on pipes, even some potential kernel bugs. We might break some existing code if we did zero-copy on pipes.

see also https://go-review.googlesource.com/c/go/+/646415

Navigation Menu

Repository navigation

os: zero copy pipe splice falling back to read+write #80977

Go version

Output of `go env` in your module/workspace:

What did you do?

What did you see happen?

What did you expect to see?

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

os: zero copy pipe splice falling back to read+write #80977

Description

Go version

Output of go env in your module/workspace:

What did you do?

What did you see happen?

What did you expect to see?

Activity

gabyhelp commented on Aug 20, 2026

seankhliao commented on Aug 22, 2026

ianlancetaylor commented on Aug 22, 2026

panjf2000 commented on Aug 23, 2026

panjf2000 commented on Aug 23, 2026

Arusekk commented on Aug 23, 2026

panjf2000 commented on Aug 23, 2026

panjf2000 commented on Aug 25, 2026

panjf2000 commented on Aug 25, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Output of `go env` in your module/workspace: