Skip to content

embed: document 2GB max file size ("file too large") #47627

@hchargois

Description

@hchargois

What version of Go are you using (go version)?

$ go version
go version go1.16.7 linux/amd64

Does this issue reproduce with the latest release?

Yes, this is the latest release

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/hugo/.cache/go-build"
GOENV="/home/hugo/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/hugo/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/hugo/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16.7"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/hugo/test/goembed/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build3344754598=/tmp/go-build -gno-record-gcc-switches"

What did you do?

  1. Create a 2001 MB (not MiB) file: dd if=/dev/zero of=2001M bs=1MB count=2001
  2. Try to embed it
package main

import (
	_ "embed"
	"fmt"
)

//go:embed 2001M
var embedded []byte

func main() {
	fmt.Println("hello", len(embedded))
}

What did you expect to see?

Successful build

What did you see instead?

Build error:

$ go build
# testgoembed
./main.go:9:5: embed 2001M: file too large

A 1999 MB (again, not MiB) file gets embedded without issue.

I think the cause is here: https://github.com/golang/go/blob/master/src/cmd/compile/internal/staticdata/data.go#L136-L142

Maybe it is unreasonable to embed 2 GB files, but if it is so, I think this size limit should be documented in the embed doc.

Metadata

Metadata

Assignees

No one assigned

    Labels

    DocumentationIssues describing a change to documentation.FrozenDueToAgeNeedsFixThe path to resolution is known, but the work has not been done.help wanted

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions