New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Socket.Poll timeout occurs unexpectedly #29772

Open
luigiberrettini opened this Issue May 17, 2018 · 8 comments

Comments

Projects
None yet
4 participants
@luigiberrettini
Collaborator

luigiberrettini commented May 17, 2018

I am trying to detect disconnection issues by using Socket.Poll.

Microsoft Docs say that Socket.Poll:

  • will block execution until the specified time period, measured in microseconds, elapses
  • cannot detect certain kinds of connection problems, such as a broken network cable,
    or that the remote host was shut down ungracefully (you must attempt to send or receive data to detect these kinds of errors)
  • when used with SelectMode.SelectRead returns:
    • true if Listen(Int32) has been called and a connection is pending
    • true if data is available for reading
    • true if the connection has been closed, reset, or terminated
    • false otherwise

Is it possible to detect disconnections with both SelectMode.SelectWrite and sending a 0 byte array or something similar that will be ignored by a standard Syslog server that is up and running with the client connected to it?

When the server is up and the connection ok, if I use Poll with SelectMode.SelectRead before sending data, I experience a delay due to the timeout expiring.
I do not understand if the expected behavior of Poll is:

  • returning true quickly, i.e. no timeout expiration
  • returning false only after the timeout elapses

If this is the case to check that everything is ok I should set a very low timeout.
What is a reasonable value i.e. 0 or 1 are ok?

@wfurt

This comment has been minimized.

Show comment
Hide comment
@wfurt

wfurt May 24, 2018

Member

I'm not sure I understand your use case @luigiberrettini Maybe simple sample code would help.
Did you try to us SelectMode.SelectError?
https://docs.microsoft.com/en-us/dotnet/api/system.net.sockets.selectmode?view=netcore-2.0
In general on Unix, you can also select/poll for read will give you notifications about errors. You can do that even if you only intent to write -> like sending data to syslog. Now depending on the network conditions, OS may or may not see socket errors. You can enable TCP keep-alive to do periodic probing if there is no traffic. That would of course not work for syslog over UDP.

Member

wfurt commented May 24, 2018

I'm not sure I understand your use case @luigiberrettini Maybe simple sample code would help.
Did you try to us SelectMode.SelectError?
https://docs.microsoft.com/en-us/dotnet/api/system.net.sockets.selectmode?view=netcore-2.0
In general on Unix, you can also select/poll for read will give you notifications about errors. You can do that even if you only intent to write -> like sending data to syslog. Now depending on the network conditions, OS may or may not see socket errors. You can enable TCP keep-alive to do periodic probing if there is no traffic. That would of course not work for syslog over UDP.

@luigiberrettini

This comment has been minimized.

Show comment
Hide comment
@luigiberrettini

luigiberrettini May 24, 2018

Collaborator

Hi @wfurt here you can find references to the behavior I need clarifications about:

Basically I am trying to detect UDP and TCP (besides using KeepAlive) disconnections before continuing sending data on a Socket.

  1. I cannot write packets with SelectMode.SelectWrite because if the Syslog server is not disconnected it will receive bad data and I do not know how it will handle it
  2. According to the documentation SelectMode.SelectRead is the one to be chosen (SelectMode.SelectError does not detect disconnections)
  3. I would like to have a confirmation that by design Poll with SelectMode.SelectRead returns false only after the timeout elapses whereas returns true immediately
  4. If the behavior at point 3 is confirmed I want to check disconnections (Poll with SelectMode.SelectRead returning true) without impacting on throughput when the Socket is connected (Poll with SelectMode.SelectRead returning false): I must set a very low timeout, but I would like to know if it is reasonable to set it to 0 or 1 or if a greater value is needed
Collaborator

luigiberrettini commented May 24, 2018

Hi @wfurt here you can find references to the behavior I need clarifications about:

Basically I am trying to detect UDP and TCP (besides using KeepAlive) disconnections before continuing sending data on a Socket.

  1. I cannot write packets with SelectMode.SelectWrite because if the Syslog server is not disconnected it will receive bad data and I do not know how it will handle it
  2. According to the documentation SelectMode.SelectRead is the one to be chosen (SelectMode.SelectError does not detect disconnections)
  3. I would like to have a confirmation that by design Poll with SelectMode.SelectRead returns false only after the timeout elapses whereas returns true immediately
  4. If the behavior at point 3 is confirmed I want to check disconnections (Poll with SelectMode.SelectRead returning true) without impacting on throughput when the Socket is connected (Poll with SelectMode.SelectRead returning false): I must set a very low timeout, but I would like to know if it is reasonable to set it to 0 or 1 or if a greater value is needed
@wfurt

This comment has been minimized.

Show comment
Hide comment
@wfurt

wfurt May 24, 2018

Member

AFAIK, there is no reliable way how to detect "disconnect" for UDP. (as there is no connection) Your data may or may not be delivered and there is no feedback or confirmation. The only option is getting ICMP (port)unreachable. And you will not get it unless you sent data to closed port.
The C# implementation will throw error only if underlying OS implementation errors out.

#1 from your previous comment is really confusing. Detection if socket is readable or writable is fully independent of data sent to the socket.

Member

wfurt commented May 24, 2018

AFAIK, there is no reliable way how to detect "disconnect" for UDP. (as there is no connection) Your data may or may not be delivered and there is no feedback or confirmation. The only option is getting ICMP (port)unreachable. And you will not get it unless you sent data to closed port.
The C# implementation will throw error only if underlying OS implementation errors out.

#1 from your previous comment is really confusing. Detection if socket is readable or writable is fully independent of data sent to the socket.

@luigiberrettini

This comment has been minimized.

Show comment
Hide comment
@luigiberrettini

luigiberrettini May 25, 2018

Collaborator

Using a heartbeat is not feasible because I am not the owner of the Syslog protocol

The only way to know if a Socket is really writable is to send data on it, but to avoid the server receiving bad data I would like to know if I can send some sort of data that is usually discarded (0 bytes array or something else).

I read that SelectMode.SelectWrite returns true based on the fact that the last write was successful but I want to check the connection just before I send data on it.

I know that UDP is a connectionless protocol, but in the UDP constructor A Connect is made which in turn retrieves the UDP server IP address and opens a socket.
Every time I call the SendAsync on the UdpClient it does not perform a Connect and does not simply send to the server ip address and port as I would expect.

As a result I need a way to understand if the Socket can be used to send data on it both for UDP and TCP

To me it is really strange that I am allowed to call Poll on a UDP socket and this does not detects disconnections as stated in the documentation: either the docs should be updated or there should be a TcpSocket class implementing Poll and a UdpSocket not implementing it.

Even if only for the TCP case should I call Poll 3 times with SelectMode.SelectRead, SelectMode.SelectWrite and SelectMode.SelectError to be sure to be able to write to a Socket?

Or should I use the approach suggested on StackOverflow?

I would like to have a confirmation that by design Poll with SelectMode.SelectRead returns false only after the timeout elapses whereas returns true immediately.
In this case I want to check disconnections (Poll with SelectMode.SelectRead returning true) without impacting on throughput when the Socket is connected (Poll with SelectMode.SelectRead returning false): I must set a very low timeout, but I would like to know if it is reasonable to set it to 0 or 1 or if a greater value is needed

Collaborator

luigiberrettini commented May 25, 2018

Using a heartbeat is not feasible because I am not the owner of the Syslog protocol

The only way to know if a Socket is really writable is to send data on it, but to avoid the server receiving bad data I would like to know if I can send some sort of data that is usually discarded (0 bytes array or something else).

I read that SelectMode.SelectWrite returns true based on the fact that the last write was successful but I want to check the connection just before I send data on it.

I know that UDP is a connectionless protocol, but in the UDP constructor A Connect is made which in turn retrieves the UDP server IP address and opens a socket.
Every time I call the SendAsync on the UdpClient it does not perform a Connect and does not simply send to the server ip address and port as I would expect.

As a result I need a way to understand if the Socket can be used to send data on it both for UDP and TCP

To me it is really strange that I am allowed to call Poll on a UDP socket and this does not detects disconnections as stated in the documentation: either the docs should be updated or there should be a TcpSocket class implementing Poll and a UdpSocket not implementing it.

Even if only for the TCP case should I call Poll 3 times with SelectMode.SelectRead, SelectMode.SelectWrite and SelectMode.SelectError to be sure to be able to write to a Socket?

Or should I use the approach suggested on StackOverflow?

I would like to have a confirmation that by design Poll with SelectMode.SelectRead returns false only after the timeout elapses whereas returns true immediately.
In this case I want to check disconnections (Poll with SelectMode.SelectRead returning true) without impacting on throughput when the Socket is connected (Poll with SelectMode.SelectRead returning false): I must set a very low timeout, but I would like to know if it is reasonable to set it to 0 or 1 or if a greater value is needed

@wfurt

This comment has been minimized.

Show comment
Hide comment
@wfurt

wfurt May 25, 2018

Member

no, I was not suggesting heat-beat and I'm somewhat familiar with syslog protocol.

Back to the Poll, discussion: The primary function is not to detect disconnects. It can detect errors on socket - and for TCP disconnect is one of them. do "man poll" on your linux system and read the page. Since UDP does not have connection, there is no disconnect and socket will not transition to error mode unless you get ICMP back as I mentioned. (all that is handled at OS level)

You can sue select() to get all the three attributes in single call. You can use 0 for timeout and the call would return immediately.

But why don't you just write your data and react to errors?
You can look at rfc5424 section 8.5. With UDP, you will not get reliability and that includes ability to detect broken servers.

Member

wfurt commented May 25, 2018

no, I was not suggesting heat-beat and I'm somewhat familiar with syslog protocol.

Back to the Poll, discussion: The primary function is not to detect disconnects. It can detect errors on socket - and for TCP disconnect is one of them. do "man poll" on your linux system and read the page. Since UDP does not have connection, there is no disconnect and socket will not transition to error mode unless you get ICMP back as I mentioned. (all that is handled at OS level)

You can sue select() to get all the three attributes in single call. You can use 0 for timeout and the call would return immediately.

But why don't you just write your data and react to errors?
You can look at rfc5424 section 8.5. With UDP, you will not get reliability and that includes ability to detect broken servers.

@luigiberrettini

This comment has been minimized.

Show comment
Hide comment
@luigiberrettini

luigiberrettini May 25, 2018

Collaborator

About heart-beats I only wanted to clarify that it was not feasible for me even if you did not suggest it.
With regard to the OS my library was targeting Windows and now also Linux and OS X.

The poll man page does not help so much in understanding how to detect a disconnection in C# and Specifying a timeout of zero causes poll() to return immediately, even if no file descriptors are ready which I honestly cannot link to a "connection is ok/not ok" statement.

As for Syslog RFCs I support both RFC 3164 and RFC 5424.

Unfortunately section 8.5 of RFC 5424 does not explain/suggest implementation details:
reliable delivery can be implemented in a way that intentionally discards messages when the syslog application would otherwise block. The advantage of reliable delivery in this case is that the syslog originator or relay knowingly discards the message and is able to notify the relay or collector about that fact. So the relay or collector receives the information that something is lost.

My code is already reacting to errors but I would like to minimize message lost: a potentially big log message can be split in mutiple Syslog messages and if only one fails it would cause the entire message to be lost.

Since I am quite confused, it would be really helpful if you could give me a crystal clear answer on what I should do.
For my understanding I should:

  • for UDP leave it fail
  • for TCP use select in a way that it is similar to checking that
    Poll(0, SelectMode.SelectRead) == false &&
    Poll(0, SelectMode.SelectWrite) == true &&
    Poll(0, SelectMode.SelectError) == false
Collaborator

luigiberrettini commented May 25, 2018

About heart-beats I only wanted to clarify that it was not feasible for me even if you did not suggest it.
With regard to the OS my library was targeting Windows and now also Linux and OS X.

The poll man page does not help so much in understanding how to detect a disconnection in C# and Specifying a timeout of zero causes poll() to return immediately, even if no file descriptors are ready which I honestly cannot link to a "connection is ok/not ok" statement.

As for Syslog RFCs I support both RFC 3164 and RFC 5424.

Unfortunately section 8.5 of RFC 5424 does not explain/suggest implementation details:
reliable delivery can be implemented in a way that intentionally discards messages when the syslog application would otherwise block. The advantage of reliable delivery in this case is that the syslog originator or relay knowingly discards the message and is able to notify the relay or collector about that fact. So the relay or collector receives the information that something is lost.

My code is already reacting to errors but I would like to minimize message lost: a potentially big log message can be split in mutiple Syslog messages and if only one fails it would cause the entire message to be lost.

Since I am quite confused, it would be really helpful if you could give me a crystal clear answer on what I should do.
For my understanding I should:

  • for UDP leave it fail
  • for TCP use select in a way that it is similar to checking that
    Poll(0, SelectMode.SelectRead) == false &&
    Poll(0, SelectMode.SelectWrite) == true &&
    Poll(0, SelectMode.SelectError) == false

@karelz karelz added this to the Future milestone Jun 13, 2018

@luigiberrettini

This comment has been minimized.

Show comment
Hide comment
@luigiberrettini

luigiberrettini Jul 5, 2018

Collaborator

One of the reasons for introducing this check was that during debug sessions I noticed that, after unplugging the cable, the first Write did not fail, therefore I was losing messages.
In a logging library this can be ok, but I would like to understand how not to lose messages only reacting on errors: should I keep a cache of the last sent message and how to avoid duplication?

Collaborator

luigiberrettini commented Jul 5, 2018

One of the reasons for introducing this check was that during debug sessions I noticed that, after unplugging the cable, the first Write did not fail, therefore I was losing messages.
In a logging library this can be ok, but I would like to understand how not to lose messages only reacting on errors: should I keep a cache of the last sent message and how to avoid duplication?

@wfurt

This comment has been minimized.

Show comment
Hide comment
@wfurt

wfurt Jul 5, 2018

Member

UDP is unreliable and you can always loose some messages. The write() will succeed when you get your data into socket buffer in kernel. You can try to write simple C program to study OS behavior.

Member

wfurt commented Jul 5, 2018

UDP is unreliable and you can always loose some messages. The write() will succeed when you get your data into socket buffer in kernel. You can try to write simple C program to study OS behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment