Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net: errClosing not exported #4373

Open
gopherbot opened this issue Nov 11, 2012 · 108 comments
Open

net: errClosing not exported #4373

gopherbot opened this issue Nov 11, 2012 · 108 comments

Comments

@gopherbot
Copy link

@gopherbot gopherbot commented Nov 11, 2012

by stephen@q5comm.com:

I have seen the issue in 1.0.3 and from checking tip.golang.org, it is still a problem.

net.errClosing is an errors.New() that is not exported but returned to the user. This
means that in order for a user to check for the "errClosing", they need to
check the error string.

I suggest renaming to ErrClosing so that a client can easily check if another goroutine
closed it's connection.
@gopherbot
Copy link
Author

@gopherbot gopherbot commented Nov 11, 2012

Comment 1 by stephen@q5comm.com:

Another option would be to return a syscall.EINVAL just like reading from a closed
connection. However, that would not be backwards compatible with anyone who is checking
the Error() string and would imply a syscall had taken place and failed.
If backwards compatibility was not an issue, I would probably want to make it so both
reading from a closed connection and a closing connection returned an ErrClosed.
@davecheney
Copy link
Contributor

@davecheney davecheney commented Nov 14, 2012

Comment 2:

Hello,
Can you give a bit of background why you need to know this particular error state ? 
The reason I ask is errClosing is an artefact of net.TCPConn implementations of the
net.Conn interface and is not guaranteed (or ever advertised, hence why it is not
exposed). 
For example, the net.Conn your program may be working with may wrapped by another
implementation, like the ssh package's ssh.ClientConn.
Cheers
Dave

Status changed to WaitingForReply.

@alberts
Copy link
Contributor

@alberts alberts commented Nov 14, 2012

Comment 3:

Based on my experience, there seems to be quite a few patterns with goroutines in
servers where you end up with 2 goroutines associated with the same Conn and where
developers (correctly, or not) deem it necessary that both goroutines close that Conn in
some error paths.
In this case, some people want to distinguish between errClosing, which is an expected
error and can be ignored, and any other error from Close (my favourite is EBADF), which
should probably be fatal to the program (i.e. the program should panic when it happens).
The general vibe seems to be that one should simply ignore the error from Close instead
of trying to distinguish between expected and unexpected errors, but this still doesn't
sit completely well with me...
@gopherbot
Copy link
Author

@gopherbot gopherbot commented Nov 18, 2012

Comment 4 by jimenezrick:

As the previous comment clearly states, sometimes a goroutine signals another goroutine
that it should stop reading from that socket. As with channels, closing the resource,
the socket in this case, seems like simple solution.
Cosmetic improvement: if this global value is ever exposed, should it be rename to
"ErrClosed"?
@gopherbot
Copy link
Author

@gopherbot gopherbot commented Nov 18, 2012

Comment 5 by stephen@q5comm.com:

That would imply it is also sent when the net.Conn is closed. Currently it returns the
error of the underlying syscall. If you are not trying to Read/Write at the time the
channel is closed, you would not receive this "net.ErrClosed".
@davecheney
Copy link
Contributor

@davecheney davecheney commented Nov 19, 2012

Comment 6:

A comment on a related issue https://golang.org/issue/4369?c=3
suggested that Accept() pre-empted by a Close should return io.EOF. Would that solution
be acceptable ?
@davecheney
Copy link
Contributor

@davecheney davecheney commented Nov 20, 2012

Comment 7:

For discussion http://golang.org/cl/6852070
@rsc
Copy link
Contributor

@rsc rsc commented Nov 26, 2012

Comment 8:

As much as I hate to do it, I think it is probably better to export
ErrClosing. These aren't EOFs.
@davecheney
Copy link
Contributor

@davecheney davecheney commented Nov 26, 2012

Comment 9:

My plan was to reduce the number of paths that could return errClosing with the hope of
eliminating it completely, or returning io.EOF if it was more appropriate.
@rsc
Copy link
Contributor

@rsc rsc commented Nov 26, 2012

Comment 10:

By itself that sounds fine, but I really think Closing and EOF are
different things and should be distinguished.
c.Read
c.Close
c.Read // EOF because you are confused and called Close
is different from
c.Read
c.Read // EOF because other side hung up
Russ
@davecheney
Copy link
Contributor

@davecheney davecheney commented Nov 27, 2012

Comment 11:

From the small play I had last week, I think there are problems where errClosing might
be wrapped by another error, from memory this is somewhere in the Accept path.
I agree that ErrClosing is not io.EOF, but I am worried about making this an additional
requirement of all implementations of net.Conn/net.Listener.
@rsc
Copy link
Contributor

@rsc rsc commented Nov 27, 2012

Comment 12:

I object to the 'rename errClosing to io.EOF' solution, but I don't
care much what else we do. I suggest either:
1) Define that people should not care, that code that wants to check
for errClosing is buggy to begin with. This is similar to getting rid
of closed(c).
2) Export ErrClosing.
Russ
@davecheney
Copy link
Contributor

@davecheney davecheney commented Nov 27, 2012

Comment 13:

+bradfitz, because of his comment, https://golang.org/issue/4369?c=3
I'd prefer to do 1, relying on errClosing is buggy, for the following:
* most times the errClosing is wrapped in an net.OpError, so a test like err ==
ErrClosing wouldn't work as expected. I'm sure there are situations where the error is
wrapped more than once. I suspect the OP didn't consider that.
* as an implementer of net.Conn implementations in things like the ssh package I
wouldn't like the additional burdon of having to return net.ErrClosing according to an
additional requirement of the net.Conn interface. I'm not even sure that I could detect
concurrent closes on a ssh channel muxed over an unknown net.Conn. Even if the original
ErrClosing was detected and retained, it would be very heavily wrapped, which speaks to
my previous point
* such a change to the net.Conn interface may make existing net.Conn implementations
broken according to the Go 1.x contract.
* Would this change bubble down to io.Reader/Writers ? It is easy to construct a struct
like
c := net.Dial(...)
v := struct { io.Reader; io.Writer; io.Closer }{ c, c, c }
Would there be an implied requirement to return a known ErrClosing when reading from an
io.Reader whose direct implementation was closed concurrently?
Fullung talked about not wanting to ignore errors from Close. I agree with this
sentiment, but wonder if the requirement could be restated as, "I would like to be able
to distinguish between expected and unexpected errors from Close". For the former they
could be safely ignored, the latter might mean a trip through os.Exit(). At the moment
errClosing, being unexported, makes it hard to tell if it is in the expected, or
unexpected class. I believe this is the core issue, and possibly what bradfitz was
suggesting when he suggested replacing errClosing with io.EOF.
At the risk of appearing obstructionist, and considering the number of lines spilt in
this CL vs the size of the original request, I'd like to hear from the OP about their
specific requirements.
@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Nov 28, 2012

Comment 14:

I wanted to be able to distinguish a listener closing due to my own closing it versus a
serious problem, but I don't care too much:  If I closed it, I know I closed it, even if
I have to pass that state around or make a net.Listener wrapper that does what I want.
So don't make things complicated for me, if this is hard.
@davecheney
Copy link
Contributor

@davecheney davecheney commented Nov 28, 2012

Comment 15:

@bradfitz, if we're talking about just specifying ErrClosing as a known response from
net.Listener.Accept() then I think that is reasonable. I think specifying it for all
methods on net.Conn is going to let a Genie out that we can't put back.
@mikioh
Copy link
Contributor

@mikioh mikioh commented Nov 28, 2012

Comment 16:

Just an idea.
The net I/O primitives on net.Conn which is created as stream
based bidirectional connection return;
- io.EOF, when the c is closed by the far end,
- io.ErrClosedPipe, when the c is closed by the near end.
@rsc
Copy link
Contributor

@rsc rsc commented Dec 9, 2012

Comment 17:

I think we should leave this as is.
Using a locally-Closed net.Conn is an error we won't require a behavior for.

Status changed to WontFix.

@davecheney
Copy link
Contributor

@davecheney davecheney commented Mar 16, 2013

Comment 18:

Issue #5062 has been merged into this issue.

@erikdubbelboer
Copy link
Contributor

@erikdubbelboer erikdubbelboer commented Mar 17, 2015

This issue was closed by @rsc because of Using a locally-Closed net.Conn is an error we won't require a behavior for.. As shown in #10176 this is actually not always an error and is behavior that is being used by the net/http.Client. I suggest reopening this issue.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Mar 17, 2015

Reopening for further thought.

@mikioh
Copy link
Contributor

@mikioh mikioh commented Mar 17, 2015

I'm not sure the reason why both net/url and net/http packages don't conform with net.Error interface, I'd prefer to make net.Error interface (both Timeout and Temporary methods on error) work with both packages. Thoughts?

PS: #4856 is already filed for fixing the same issue of net package.

@smithwinston
Copy link

@smithwinston smithwinston commented Mar 19, 2015

I also have two use cases for detecting a closed connection when using goroutines:

  1. Remote end has closed the connection normally
  2. Closing a connection locally in order to stop a service

In both cases, my goroutine is blocked on a Read() and errors out with an *errors.errorString with the text "use of closed network connection".

In both use cases, I want to be able to detect these errors as io.EOF (use case 1) or io.ErrClosed (use case 2) and handle them accordingly. With the current errClosing, I can't cleanly detect the error and if the error message were to change, my code wouldn't detect it.

@davecheney
Copy link
Contributor

@davecheney davecheney commented Mar 19, 2015

In both use cases, I want to be able to detect these errors as io.EOF
(use case 1) or io.ErrClosed (use case 2) and handle them accordingly.

Can you please explain how you would handle these cases specifically if you
could identify them ?

On Fri, Mar 20, 2015 at 7:18 AM, smithwinston notifications@github.com
wrote:

I also have two use cases for detecting a closed connection when using
goroutines:

  1. Remote end has closed the connection normally
  2. Closing a connection locally in order to stop a service

In both cases, my goroutine is blocked on a Read() and errors out with an
*errors.errorString with the text "use of closed network connection".

In both use cases, I want to be able to detect these errors as io.EOF (use
case 1) or io.ErrClosed (use case 2) and handle them accordingly. With the
current errClosing, I can't cleanly detect the error and if the error
message were to change, my code wouldn't detect it.


Reply to this email directly or view it on GitHub
#4373 (comment).

@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
@samv
Copy link

@samv samv commented Nov 20, 2019

Just a quick note: I landed on this issue while trying to figure out why my reads were returning the "use of closed network connection" on them after I'd called syscall.Shutdown(fd, syscall.SHUT_WR) on the connection, instead of io.EOF as I was expecting from a read-only socket which has returned all of its data. I ended up adding this sort of workaround after the read:

if strings.Contains(err.Error(), "use of closed network connection") {
    err = io.EOF
}
@riobard
Copy link

@riobard riobard commented Feb 3, 2020

Now that we have Go 1.13 with its fancy errors.Is helper, can we please export net.ErrClosing now so I can do errors.Is(err, net.ErrClosing) without worrying about many layers of (un)wrapping?

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Feb 4, 2020

Sure, let's take another pass at this for 1.15.

We still have to consider @davecheney 's comments above, e.g., #4373 (comment). Wrapping does make a difference; does it make enough of a difference?

@ianlancetaylor ianlancetaylor modified the milestones: Backlog, Go1.15 Feb 4, 2020
@riobard
Copy link

@riobard riobard commented Feb 4, 2020

The primary argument of #4373 (comment) is that net.Conn happens to be an io.Reader therefore exposing net.ErrClosing would create additional behavioral contract for net.Conn interface and break io.Reader contract.

The argument is evidently refuted by #4373 (comment), and in general wrong because unless net.Conn says

type Conn interface {
    io.Reader

no one should ever assume net.Conn and io.Reader follow exactly the same contract. In reality net.Conn just says

type Conn interface {
    // Read reads data from the connection.
    // Read can be made to time out and return an Error with Timeout() == true
    // after a fixed time limit; see SetDeadline and SetReadDeadline.
    Read(b []byte) (n int, err, error)

Nothing specific about io.Reader at all.

@riobard
Copy link

@riobard riobard commented Feb 5, 2020

It turns out that there are more critical but unexported error values in the standard library, and net.errClosing is just the tip of the iceberg.

I was writing an HTTP proxy using net.http for its support for HTTP/1 and HTTP/2. I need to detect errors and do correct things according to spec. But net.http has its own bundled copy of x/net/http2 with nothing exported. So I cannot do nice things suggested by Working with Errors in Go 1.13:

var h2err http2.StreamError
if errors.As(err, &h2err) { // dealing with errors

it will never match because the actual error is http.http2StreamError which is not exported. Similar unexported error values from http2 are here.

Basically it forces us to either do string match which is ugly as hell, or pretend it's some generic error we could not specifically address/ignore.

This issue stuck for over SEVEN YEARS is very frustrating and sad.

@cpuguy83
Copy link

@cpuguy83 cpuguy83 commented Feb 5, 2020

IMO (I'm not going to be humble about this one), we should be asking the package in question "Is it this kind of error?". A good(-ish) example of this is os.IsNotExist.

So, for example, net.IsClosing(err).

@nhooyr
Copy link
Contributor

@nhooyr nhooyr commented Feb 5, 2020

@riobard Why do you need to check for http2.StreamError? x/net/http2 is intentionally abstracted away into net/http as you shouldn't have to access anything http2 specific.

@riobard
Copy link

@riobard riobard commented Feb 5, 2020

@nhooyr Because to properly implement the spec I need to know what's going on with the underlying stream to decide if I should convey that error to the other end of the proxy.

It's actually a similar reason for exporting net.errClosing: we need, in many situations, to figure out the reason when a connection (as in net) or stream (as in http2) is broken, and handle differently.

@tv42
Copy link

@tv42 tv42 commented Feb 5, 2020

@cpuguy83 That used to be the recommendation, but things are switching to errors.Is/As. For example: os.ErrNotExist vs os.IsNotExist.

@cpuguy83
Copy link

@cpuguy83 cpuguy83 commented Feb 5, 2020

@tv42 Doesn't make it a good approach :)

@tv42
Copy link

@tv42 tv42 commented Feb 5, 2020

@cpuguy83 Just one mechanism instead of every package implementing its own IsFoo, IsBar etc, plus ability to work even after being wrapped by out-of-package code, both sound like wins. Also, you've already lost that fight.

@powerman
Copy link

@powerman powerman commented Feb 5, 2020

@cpuguy83 That used to be the recommendation, but things are switching to errors.Is/As. For example: os.ErrNotExist vs os.IsNotExist.

Yep. When it works. But when it doesn't - you'll have to fallback to os.IsNotExist (happens to me today).

@jefferai
Copy link

@jefferai jefferai commented Feb 20, 2020

And regardless, Is/As requires you to have an exported error to compare against.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented May 18, 2020

I've come around to thinking that we should do this. But we didn't do it for 1.15, so marking for 1.16.

@Miosss
Copy link

@Miosss Miosss commented Jun 4, 2020

This should be exported (in some way).
I stumbled upon this today, when debugging net.TCPLIstener.accept() return values.

My code correctly closes the listener, and goroutine that accepts clients gets only ""use of closed network connection" as an error, when I would like to differentiate this 'not really an error but perfectly fine way to top this goroutine' from an actual error.

Some may (and have already) ask why it is important to be able to check different errors. I thought it was obvious. The most basic example is not to log the information about normal close, only about true errors. Second is trying to restart the function. If listener truly breaks, then I will try to start it again. If it was closed normally (bacause of context.cancel for example) then I won't.

I understand that go is fun language and doing hard things is easy with it. But the simple ones are usually the most challenging. Especially reliable error handling.

Reluctance to exporting specific errors is something I do not understand. Especially when there are no error codes, nothing more then plain string error in the error interface. Somebody in this issue wrote about http2 abstracting errors. I had the same situation and it mostly boils down to abstracting (or just hiding/obscuring) the true errors. I filled an issue back then at #28930

If this problem is not solved, and only strings will be THE error values, then parsing those string will become idiomatic.
And is is even already recognized in internal/poll/fd.go:

// ErrNetClosing is returned when a network descriptor is used after
// it has been closed. Keep this string consistent because of issue
// #4373: since historically programs have not been able to detect
// this error, they look for the string.
var ErrNetClosing = errors.New("use of closed network connection")
@noncombatant
Copy link

@noncombatant noncombatant commented Jun 17, 2020

I agree with @riobard in #4373 (comment): it's not necessarily the case that errClosing must be propagated to all other interfaces that wrap or work like this one.

In any case, if a public function returns an object, that object's type must also be public — otherwise the caller has no chance to fully know and understand the interface of the function they're calling. The inevitable result is this string matching silliness (which I also found myself having to do), when type switching is the idiomatic and much more clearly correct approach.

If we're willing to live with designs we don't love, and we have to be ;) , it seems straightforward that exporting the type is much less unlovable than the status quo (#4373 (comment)).

@awfm9
Copy link

@awfm9 awfm9 commented Jun 17, 2020

There are currently over 3,000 cases of matching this error string on GitHub, including projects such as Kubernetes:

https://github.com/search?l=Go&q=%22use+of+closed+network+connection%22&type=Code

Whatever people think is the "correct" way to work around the issue, it is clearly out of touch with the reality on the ground.

Let's please implement the pragmatic solution asap.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.