io: clarify Writer bytes written semantics

[kevinburke]

The documentation for io.Writer states:

Writer is the interface that wraps the basic Write method.

Write writes len(p) bytes from p to the underlying data stream. It returns the number of bytes written from p (0 <= n <= len(p)) and any error encountered that caused the write to stop early. Write must return a non-nil error if it returns n < len(p). Write must not modify the slice data, even temporarily.

And for example, *os.File.Write() says this:

Write writes len(b) bytes from b to the File. It returns the number of bytes written and an error, if any. Write returns a non-nil error when n != len(b).

With some write API's, to sockets or files, you may get an error and not know whether the write actually succeeded or not (see https://danluu.com/file-consistency/ for examples).

If, say, 16 bytes are written and there is an error writing byte 17. The interface description isn't clear about what should happen:

• Write should always report (16, error)
• Write should try to determine whether the error indicates no data was written (and return 16), otherwise return 17
• Write should always report (17, error)
• Write can return 16 or 17 as it likes.

Under the hood, *os.File.Write for example eventually invokes the write syscall, which also does not give any clarity about what happens, at least on BSD.

Upon successful completion the number of bytes which were written is returned. Otherwise, a -1 is returned and the global variable errno is set to indicate the error.

My request is to clarify what should happen, and then document it for consistency.

[ianlancetaylor]

If the system call doesn't tell us what happened, then the Write method can't tell the caller what happened. So I'm not sure what we should document. "It would be nice if we reported this correctly but we can't promise?"

I think the intent is clear: Write should report the number of bytes that were actually written. But if we don't know, we don't know.

[kevinburke]

But if we don't know, we don't know.

Fair; if we don't know, then what do we report? Should the count returned by Write include all of the bytes that were attempted, or none? That's what I'm trying to document, I guess.

[ianlancetaylor]

I see. I think none. While we don't know whether that is accurate, it seems more likely than all or some, I think.

[extemporalgenome]

Much of Go's writer implementations or wrappers expect reliable byte stream: errors are often assumed to imply that the writer is unusable past that point.

The returned count from a Read call is particularly meaningful since Read can return early without error. In contrast, a Write call is expected to block until the full buffer has been written.

For writer-as-stream cases, it's not clear how valuable the returned n value would be. For seekable writers, it may be useful for tracking file offset, though WriterAt is probably more useful in those cases.

Since we should consistently return some value for n, due to the previously discussed ambiguity, perhaps we should just return the number of bytes affirmatively reported as being successfully written (which may be less than the number of bytes actually received, such as in the case of a broken network connection).

In the original scenario listed, this may mean that we report 0 bytes written, since the underlying transport may be unable to distinguish the number of bytes received on the other side and because we're sending all 17 bytes at once (we certainly wouldn't be writing 1 byte at a time just to have byte-level failure precision). If, internally, we sent a batch of 16 bytes (e.g. due to framing), which succeeded, and then sent 1 byte, which failed, we would certainly return 16, since that number was affirmatively successful.

[bcmills]

I think it would be more useful to document the semantics of specific errors than to try document the semantics of n independent of the error value.

If there are specific errors that always indicate an exact count of bytes written — or specific errors that always indicate an inexact count of bytes written — then it seems useful for callers who observe those specific errors to know those implications, but given the ambiguity in the io.Writer interface — and the fact that existing implementations may have interpreted it either way — in the absence of a specific error value that says otherwise (or documentation for a specific implementation of the Write method) I don't think the caller can safely assume anything about the number of bytes actually written.