Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: io/fs: track, list, interrupt pending ops #41249

Closed
networkimprov opened this issue Sep 7, 2020 · 3 comments
Closed

proposal: io/fs: track, list, interrupt pending ops #41249

networkimprov opened this issue Sep 7, 2020 · 3 comments
Labels
Projects
Milestone

Comments

@networkimprov
Copy link

@networkimprov networkimprov commented Sep 7, 2020

Operations by a filesystem (esp one using a network) must be interrupted if they stall. Herein is a way to do so without adding context.Context arguments to all functions of io/fs.

Mandating Context arguments would force them into third party package APIs that invoke any io/fs method. Code that depends on those packages would then be broken. If a package author did not amend its API, the callers would have to manage stalled ops some other way, and likely leak resources on every op retry.

However, we can define interfaces that support Context; see below.

The stdlib os filesystem would implement the PendingFS and PendingFile interfaces. Linux CIFS & FUSE and Windows I/O all support interruption of pending ops; Linux NFS does not. So interruption isn't assured for every deployment environment.

package fs

type PendingFS interface {
   FS
   TrackPending(on bool)               // turn on/off tracking; callable by multiple packages
   ListPending(kind uint) []PendingOp  // return recently pending ops
                                       // takes a set of |'d values, e.g. OpKindTop | OpKindFile
   Interrupt(op ...PendingOp) error    // interrupt one or more ops, if still pending
}

type PendingFile interface {
   File
   ListPending() []PendingOp           // return recently pending ops on the file
}

type PendingOp interface {
   Fn() string                         // "Open" etc
   Params() []interface{}              // all arguments
   Pathname() string                   // empty if not applicable
   String() string                     // summary suitable for logging
}

type InterruptError struct {...}       // returned by interrupted ops

const (
   OpKindFile uint = 1 << iota
   ...
)
package context // elsewhere?

type FS interface {
   fs.PendingFS
   WithContext(c Context, kind uint)   // cancellation interrupts all ops of 'kind'
}

type File interface {
   fs.PendingFile
   WithContext(c Context)              // cancellation interrupts all ops on the File
                                       // the context is preset if FS.WithContext got fs.OpKindFile | ...
}

Typical usage

afs := NewMyFs()
afs.TrackPending(true)
...
afl, err := afs.Open("myfile")
...
if problems {
   afs.Interrupt(afl.ListPending()...)
}

Related: #41054 - runtime: allow termination of blocked syscalls
Branched from #41190 (comment) - io/fs: add file system interfaces

cc @robpike @rsc

@rsc
Copy link
Contributor

@rsc rsc commented Sep 11, 2020

Redesigning the os.File interface is explicitly out of scope for the io/fs proposal.
We are only making a simple interface for the current world, not building a new world.

@networkimprov
Copy link
Author

@networkimprov networkimprov commented Sep 11, 2020

Since io/fs is not intended (per #41190 (comment)) to support possibly-unreliable filesystem implementations, I will redirect this to package os, where the file API is already used to access to network filesystems.

@networkimprov
Copy link
Author

@networkimprov networkimprov commented Sep 14, 2020

Folded into #41054 -- see #41054 (comment).

@rsc rsc moved this from Incoming to Declined in Proposals Sep 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Proposals
Declined
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.