Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net: errClosing not exported #4373

Open
gopherbot opened this issue Nov 11, 2012 · 104 comments
Open

net: errClosing not exported #4373

gopherbot opened this issue Nov 11, 2012 · 104 comments
Labels
Milestone

Comments

@gopherbot
Copy link

@gopherbot gopherbot commented Nov 11, 2012

by stephen@q5comm.com:

I have seen the issue in 1.0.3 and from checking tip.golang.org, it is still a problem.

net.errClosing is an errors.New() that is not exported but returned to the user. This
means that in order for a user to check for the "errClosing", they need to
check the error string.

I suggest renaming to ErrClosing so that a client can easily check if another goroutine
closed it's connection.
@gopherbot

This comment has been minimized.

Copy link
Author

@gopherbot gopherbot commented Nov 11, 2012

Comment 1 by stephen@q5comm.com:

Another option would be to return a syscall.EINVAL just like reading from a closed
connection. However, that would not be backwards compatible with anyone who is checking
the Error() string and would imply a syscall had taken place and failed.
If backwards compatibility was not an issue, I would probably want to make it so both
reading from a closed connection and a closing connection returned an ErrClosed.
@davecheney

This comment has been minimized.

Copy link
Contributor

@davecheney davecheney commented Nov 14, 2012

Comment 2:

Hello,
Can you give a bit of background why you need to know this particular error state ? 
The reason I ask is errClosing is an artefact of net.TCPConn implementations of the
net.Conn interface and is not guaranteed (or ever advertised, hence why it is not
exposed). 
For example, the net.Conn your program may be working with may wrapped by another
implementation, like the ssh package's ssh.ClientConn.
Cheers
Dave

Status changed to WaitingForReply.

@alberts

This comment has been minimized.

Copy link
Contributor

@alberts alberts commented Nov 14, 2012

Comment 3:

Based on my experience, there seems to be quite a few patterns with goroutines in
servers where you end up with 2 goroutines associated with the same Conn and where
developers (correctly, or not) deem it necessary that both goroutines close that Conn in
some error paths.
In this case, some people want to distinguish between errClosing, which is an expected
error and can be ignored, and any other error from Close (my favourite is EBADF), which
should probably be fatal to the program (i.e. the program should panic when it happens).
The general vibe seems to be that one should simply ignore the error from Close instead
of trying to distinguish between expected and unexpected errors, but this still doesn't
sit completely well with me...
@gopherbot

This comment has been minimized.

Copy link
Author

@gopherbot gopherbot commented Nov 18, 2012

Comment 4 by jimenezrick:

As the previous comment clearly states, sometimes a goroutine signals another goroutine
that it should stop reading from that socket. As with channels, closing the resource,
the socket in this case, seems like simple solution.
Cosmetic improvement: if this global value is ever exposed, should it be rename to
"ErrClosed"?
@gopherbot

This comment has been minimized.

Copy link
Author

@gopherbot gopherbot commented Nov 18, 2012

Comment 5 by stephen@q5comm.com:

That would imply it is also sent when the net.Conn is closed. Currently it returns the
error of the underlying syscall. If you are not trying to Read/Write at the time the
channel is closed, you would not receive this "net.ErrClosed".
@davecheney

This comment has been minimized.

Copy link
Contributor

@davecheney davecheney commented Nov 19, 2012

Comment 6:

A comment on a related issue https://golang.org/issue/4369?c=3
suggested that Accept() pre-empted by a Close should return io.EOF. Would that solution
be acceptable ?
@davecheney

This comment has been minimized.

Copy link
Contributor

@davecheney davecheney commented Nov 20, 2012

Comment 7:

For discussion http://golang.org/cl/6852070
@rsc

This comment has been minimized.

Copy link
Contributor

@rsc rsc commented Nov 26, 2012

Comment 8:

As much as I hate to do it, I think it is probably better to export
ErrClosing. These aren't EOFs.
@davecheney

This comment has been minimized.

Copy link
Contributor

@davecheney davecheney commented Nov 26, 2012

Comment 9:

My plan was to reduce the number of paths that could return errClosing with the hope of
eliminating it completely, or returning io.EOF if it was more appropriate.
@rsc

This comment has been minimized.

Copy link
Contributor

@rsc rsc commented Nov 26, 2012

Comment 10:

By itself that sounds fine, but I really think Closing and EOF are
different things and should be distinguished.
c.Read
c.Close
c.Read // EOF because you are confused and called Close
is different from
c.Read
c.Read // EOF because other side hung up
Russ
@davecheney

This comment has been minimized.

Copy link
Contributor

@davecheney davecheney commented Nov 27, 2012

Comment 11:

From the small play I had last week, I think there are problems where errClosing might
be wrapped by another error, from memory this is somewhere in the Accept path.
I agree that ErrClosing is not io.EOF, but I am worried about making this an additional
requirement of all implementations of net.Conn/net.Listener.
@rsc

This comment has been minimized.

Copy link
Contributor

@rsc rsc commented Nov 27, 2012

Comment 12:

I object to the 'rename errClosing to io.EOF' solution, but I don't
care much what else we do. I suggest either:
1) Define that people should not care, that code that wants to check
for errClosing is buggy to begin with. This is similar to getting rid
of closed(c).
2) Export ErrClosing.
Russ
@davecheney

This comment has been minimized.

Copy link
Contributor

@davecheney davecheney commented Nov 27, 2012

Comment 13:

+bradfitz, because of his comment, https://golang.org/issue/4369?c=3
I'd prefer to do 1, relying on errClosing is buggy, for the following:
* most times the errClosing is wrapped in an net.OpError, so a test like err ==
ErrClosing wouldn't work as expected. I'm sure there are situations where the error is
wrapped more than once. I suspect the OP didn't consider that.
* as an implementer of net.Conn implementations in things like the ssh package I
wouldn't like the additional burdon of having to return net.ErrClosing according to an
additional requirement of the net.Conn interface. I'm not even sure that I could detect
concurrent closes on a ssh channel muxed over an unknown net.Conn. Even if the original
ErrClosing was detected and retained, it would be very heavily wrapped, which speaks to
my previous point
* such a change to the net.Conn interface may make existing net.Conn implementations
broken according to the Go 1.x contract.
* Would this change bubble down to io.Reader/Writers ? It is easy to construct a struct
like
c := net.Dial(...)
v := struct { io.Reader; io.Writer; io.Closer }{ c, c, c }
Would there be an implied requirement to return a known ErrClosing when reading from an
io.Reader whose direct implementation was closed concurrently?
Fullung talked about not wanting to ignore errors from Close. I agree with this
sentiment, but wonder if the requirement could be restated as, "I would like to be able
to distinguish between expected and unexpected errors from Close". For the former they
could be safely ignored, the latter might mean a trip through os.Exit(). At the moment
errClosing, being unexported, makes it hard to tell if it is in the expected, or
unexpected class. I believe this is the core issue, and possibly what bradfitz was
suggesting when he suggested replacing errClosing with io.EOF.
At the risk of appearing obstructionist, and considering the number of lines spilt in
this CL vs the size of the original request, I'd like to hear from the OP about their
specific requirements.
@bradfitz

This comment has been minimized.

Copy link
Contributor

@bradfitz bradfitz commented Nov 28, 2012

Comment 14:

I wanted to be able to distinguish a listener closing due to my own closing it versus a
serious problem, but I don't care too much:  If I closed it, I know I closed it, even if
I have to pass that state around or make a net.Listener wrapper that does what I want.
So don't make things complicated for me, if this is hard.
@davecheney

This comment has been minimized.

Copy link
Contributor

@davecheney davecheney commented Nov 28, 2012

Comment 15:

@bradfitz, if we're talking about just specifying ErrClosing as a known response from
net.Listener.Accept() then I think that is reasonable. I think specifying it for all
methods on net.Conn is going to let a Genie out that we can't put back.
@mikioh

This comment has been minimized.

Copy link
Contributor

@mikioh mikioh commented Nov 28, 2012

Comment 16:

Just an idea.
The net I/O primitives on net.Conn which is created as stream
based bidirectional connection return;
- io.EOF, when the c is closed by the far end,
- io.ErrClosedPipe, when the c is closed by the near end.
@rsc

This comment has been minimized.

Copy link
Contributor

@rsc rsc commented Dec 9, 2012

Comment 17:

I think we should leave this as is.
Using a locally-Closed net.Conn is an error we won't require a behavior for.

Status changed to WontFix.

@davecheney

This comment has been minimized.

Copy link
Contributor

@davecheney davecheney commented Mar 16, 2013

Comment 18:

Issue #5062 has been merged into this issue.

@erikdubbelboer

This comment has been minimized.

Copy link
Contributor

@erikdubbelboer erikdubbelboer commented Mar 17, 2015

This issue was closed by @rsc because of Using a locally-Closed net.Conn is an error we won't require a behavior for.. As shown in #10176 this is actually not always an error and is behavior that is being used by the net/http.Client. I suggest reopening this issue.

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Mar 17, 2015

Reopening for further thought.

@mikioh

This comment has been minimized.

Copy link
Contributor

@mikioh mikioh commented Mar 17, 2015

I'm not sure the reason why both net/url and net/http packages don't conform with net.Error interface, I'd prefer to make net.Error interface (both Timeout and Temporary methods on error) work with both packages. Thoughts?

PS: #4856 is already filed for fixing the same issue of net package.

@smithwinston

This comment has been minimized.

Copy link

@smithwinston smithwinston commented Mar 19, 2015

I also have two use cases for detecting a closed connection when using goroutines:

  1. Remote end has closed the connection normally
  2. Closing a connection locally in order to stop a service

In both cases, my goroutine is blocked on a Read() and errors out with an *errors.errorString with the text "use of closed network connection".

In both use cases, I want to be able to detect these errors as io.EOF (use case 1) or io.ErrClosed (use case 2) and handle them accordingly. With the current errClosing, I can't cleanly detect the error and if the error message were to change, my code wouldn't detect it.

@davecheney

This comment has been minimized.

Copy link
Contributor

@davecheney davecheney commented Mar 19, 2015

In both use cases, I want to be able to detect these errors as io.EOF
(use case 1) or io.ErrClosed (use case 2) and handle them accordingly.

Can you please explain how you would handle these cases specifically if you
could identify them ?

On Fri, Mar 20, 2015 at 7:18 AM, smithwinston notifications@github.com
wrote:

I also have two use cases for detecting a closed connection when using
goroutines:

  1. Remote end has closed the connection normally
  2. Closing a connection locally in order to stop a service

In both cases, my goroutine is blocked on a Read() and errors out with an
*errors.errorString with the text "use of closed network connection".

In both use cases, I want to be able to detect these errors as io.EOF (use
case 1) or io.ErrClosed (use case 2) and handle them accordingly. With the
current errClosing, I can't cleanly detect the error and if the error
message were to change, my code wouldn't detect it.


Reply to this email directly or view it on GitHub
#4373 (comment).

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Nov 16, 2018

@networkimprov The Error Values proposal.

@ainar-g

This comment has been minimized.

Copy link
Contributor

@ainar-g ainar-g commented Dec 27, 2018

Any chance crypto/tls.errClosed will get the same treatment? Or should I open a separate issue for that?

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Dec 27, 2018

@ainar-g That seems like a separate issue, but one that should wait until this one is resolved.

@cpuguy83

This comment has been minimized.

Copy link

@cpuguy83 cpuguy83 commented Jan 22, 2019

Just to chime in here, and sorry if it's an annoyance but... basically what we are saying here is that handling these errors with strings.Contains is a better solution than anything presented here.

Of course I respect that adding interfaces is not a small task and must be supported forever (probably much like these error messages can never change w/o breaking things), but seems like an odd situation to be in.

@andybons andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019
@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
@samv

This comment has been minimized.

Copy link

@samv samv commented Nov 20, 2019

Just a quick note: I landed on this issue while trying to figure out why my reads were returning the "use of closed network connection" on them after I'd called syscall.Shutdown(fd, syscall.SHUT_WR) on the connection, instead of io.EOF as I was expecting from a read-only socket which has returned all of its data. I ended up adding this sort of workaround after the read:

if strings.Contains(err.Error(), "use of closed network connection") {
    err = io.EOF
}
@tv42

This comment has been minimized.

@riobard

This comment has been minimized.

Copy link

@riobard riobard commented Feb 3, 2020

Now that we have Go 1.13 with its fancy errors.Is helper, can we please export net.ErrClosing now so I can do errors.Is(err, net.ErrClosing) without worrying about many layers of (un)wrapping?

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Feb 4, 2020

Sure, let's take another pass at this for 1.15.

We still have to consider @davecheney 's comments above, e.g., #4373 (comment). Wrapping does make a difference; does it make enough of a difference?

@ianlancetaylor ianlancetaylor modified the milestones: Backlog, Go1.15 Feb 4, 2020
@riobard

This comment has been minimized.

Copy link

@riobard riobard commented Feb 4, 2020

The primary argument of #4373 (comment) is that net.Conn happens to be an io.Reader therefore exposing net.ErrClosing would create additional behavioral contract for net.Conn interface and break io.Reader contract.

The argument is evidently refuted by #4373 (comment), and in general wrong because unless net.Conn says

type Conn interface {
    io.Reader

no one should ever assume net.Conn and io.Reader follow exactly the same contract. In reality net.Conn just says

type Conn interface {
    // Read reads data from the connection.
    // Read can be made to time out and return an Error with Timeout() == true
    // after a fixed time limit; see SetDeadline and SetReadDeadline.
    Read(b []byte) (n int, err, error)

Nothing specific about io.Reader at all.

@riobard

This comment has been minimized.

Copy link

@riobard riobard commented Feb 5, 2020

It turns out that there are more critical but unexported error values in the standard library, and net.errClosing is just the tip of the iceberg.

I was writing an HTTP proxy using net.http for its support for HTTP/1 and HTTP/2. I need to detect errors and do correct things according to spec. But net.http has its own bundled copy of x/net/http2 with nothing exported. So I cannot do nice things suggested by Working with Errors in Go 1.13:

var h2err http2.StreamError
if errors.As(err, &h2err) { // dealing with errors

it will never match because the actual error is http.http2StreamError which is not exported. Similar unexported error values from http2 are here.

Basically it forces us to either do string match which is ugly as hell, or pretend it's some generic error we could not specifically address/ignore.

This issue stuck for over SEVEN YEARS is very frustrating and sad.

@cpuguy83

This comment has been minimized.

Copy link

@cpuguy83 cpuguy83 commented Feb 5, 2020

IMO (I'm not going to be humble about this one), we should be asking the package in question "Is it this kind of error?". A good(-ish) example of this is os.IsNotExist.

So, for example, net.IsClosing(err).

@nhooyr

This comment has been minimized.

Copy link
Contributor

@nhooyr nhooyr commented Feb 5, 2020

@riobard Why do you need to check for http2.StreamError? x/net/http2 is intentionally abstracted away into net/http as you shouldn't have to access anything http2 specific.

@riobard

This comment has been minimized.

Copy link

@riobard riobard commented Feb 5, 2020

@nhooyr Because to properly implement the spec I need to know what's going on with the underlying stream to decide if I should convey that error to the other end of the proxy.

It's actually a similar reason for exporting net.errClosing: we need, in many situations, to figure out the reason when a connection (as in net) or stream (as in http2) is broken, and handle differently.

@tv42

This comment has been minimized.

Copy link

@tv42 tv42 commented Feb 5, 2020

@cpuguy83 That used to be the recommendation, but things are switching to errors.Is/As. For example: os.ErrNotExist vs os.IsNotExist.

@cpuguy83

This comment has been minimized.

Copy link

@cpuguy83 cpuguy83 commented Feb 5, 2020

@tv42 Doesn't make it a good approach :)

@tv42

This comment has been minimized.

Copy link

@tv42 tv42 commented Feb 5, 2020

@cpuguy83 Just one mechanism instead of every package implementing its own IsFoo, IsBar etc, plus ability to work even after being wrapped by out-of-package code, both sound like wins. Also, you've already lost that fight.

@powerman

This comment has been minimized.

Copy link

@powerman powerman commented Feb 5, 2020

@cpuguy83 That used to be the recommendation, but things are switching to errors.Is/As. For example: os.ErrNotExist vs os.IsNotExist.

Yep. When it works. But when it doesn't - you'll have to fallback to os.IsNotExist (happens to me today).

@jefferai

This comment has been minimized.

Copy link

@jefferai jefferai commented Feb 20, 2020

And regardless, Is/As requires you to have an exported error to compare against.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.