Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: embed: openFile should implement ReaderAt #57803

Open
ncruces opened this issue Jan 15, 2023 · 5 comments
Open

proposal: embed: openFile should implement ReaderAt #57803

ncruces opened this issue Jan 15, 2023 · 5 comments
Labels
Milestone

Comments

@ncruces
Copy link
Contributor

ncruces commented Jan 15, 2023

embed.openFile could trivially implement io.ReaderAt.

go/src/embed/embed.go

Lines 348 to 381 in 15605ca

var (
_ io.Seeker = (*openFile)(nil)
)
func (f *openFile) Close() error { return nil }
func (f *openFile) Stat() (fs.FileInfo, error) { return f.f, nil }
func (f *openFile) Read(b []byte) (int, error) {
if f.offset >= int64(len(f.f.data)) {
return 0, io.EOF
}
if f.offset < 0 {
return 0, &fs.PathError{Op: "read", Path: f.f.name, Err: fs.ErrInvalid}
}
n := copy(b, f.f.data[f.offset:])
f.offset += int64(n)
return n, nil
}
func (f *openFile) Seek(offset int64, whence int) (int64, error) {
switch whence {
case 0:
// offset += 0
case 1:
offset += f.offset
case 2:
offset += int64(len(f.f.data))
}
if offset < 0 || offset > int64(len(f.f.data)) {
return 0, &fs.PathError{Op: "seek", Path: f.f.name, Err: fs.ErrInvalid}
}
f.offset = offset
return offset, nil
}

That's the entire proposal.

Rationale

io.ReaderAt can be emulated with io.ReadSeeker, but this doesn't satisfy the goroutine safety of io.ReaderAt: “Clients of ReadAt can execute parallel ReadAt calls on the same input source.”

This came up twice now on implementing wazero syscalls, that follow pread semantics: tetratelabs/wazero#967, and now tetratelabs/wazero#1037.
It got its own mention in that project's design rationale.

It is also of note that io.ReaderAt is directly called out on the fs.File documentation: A file may implement io.ReaderAt or io.Seeker as optimizations.

@gopherbot gopherbot added this to the Proposal milestone Jan 15, 2023
@ncruces
Copy link
Contributor Author

ncruces commented Jan 15, 2023

If the proposal is accepted, I can happily do the legwork on implementation and testing. Or delegate it to the Go team, whichever is more efficient (the burden of reviewing untrusted code is often higher than the implementation, but I've contributed a couple fixes before, so I'm already familiar with the process).

@magical
Copy link
Contributor

magical commented Jan 19, 2023

My 2¢: We shouldn't promise that files from embed.FS implement ReaderAt - we should leave the door open for transparent compression of embedded files (for example) in future versions of Go and in alternative toolchains. But providing it as an optional interface upgrade, when available, seems reasonable to me.

@ncruces
Copy link
Contributor Author

ncruces commented Jan 19, 2023

I'm fine with that, but we already promise to implement io.Seeker.

For the embed case, I struggle to see a case where is Seek is “easy” but ReadAt “hard” (even considering e.g. compression, etc).

@magical
Copy link
Contributor

magical commented Jan 19, 2023

I'm fine with that, but we already promise to implement io.Seeker.

You're right, I missed that detail. Ah well. Probably not worth worrying about then.

@rsc
Copy link
Contributor

rsc commented Mar 15, 2023

This seems completely reasonable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Incoming
Development

No branches or pull requests

4 participants