TcpListener.Stop hangs if Accept is in progress

[psantosl]

Hi,

We are experiencing an issue on Linux, building with publish -r linux-x64.

It looks like TcpListener.Stop() hangs when we try to orderly shutdown the server.

The socket is waiting for an Accept, then other thread does TcpListener.Stop().

The same code works fine on Windows (.NET Core) and also on Linux/Mono.

Here the sequence is:

• We start the server.
• We try to stop it.
• It takes forever to finish.
• Attaching the debugger (from Visual Studio on Windows, which is awesome :P) we see the process is stopped in TcpListener.Stop and never leaves.

Attached a repro case, but the code is super simple:

using System;
using System.Net;
using System.Net.Sockets;
using System.Threading;

namespace tcplistenertest
{
    class Program
    {
        static void Main(string[] args)
        {
            Thread listenerThread = StartBackgroundThread(new ThreadStart(ThreadProc));

            while(Console.ReadLine() != string.Empty)
            {
            }

            mListener.Stop();
            listenerThread.Join();
        }

        static void ThreadProc()
        {
            try
            {
                mListener = StartListening();

                AcceptLoop(mListener);
            }
            catch(Exception e)
            {
                Console.WriteLine($"ThreadProc error: {e.Message}");
                Console.WriteLine($"Stack trace:{Environment.NewLine}{e.StackTrace}");
            }
        }

        static Thread StartBackgroundThread(ThreadStart task)
        {
            Thread thread = new Thread(task);
            thread.IsBackground = true;
            thread.Start();
            return thread;
        }

        static TcpListener StartListening()
        {
            TcpListener result = new TcpListener(IPAddress.Any, PORT);
            result.Start();

            return result;
        }

        static void AcceptLoop(TcpListener listener)
        {
            // this is the main loop of the Socket based server
            while (true)
            {
                Socket socket = null;
                try
                {
                    Console.WriteLine("Accepting connection...");
                    socket = AcceptConnection(listener);
                    Console.WriteLine($"Connection accepted from {socket.RemoteEndPoint}");
                }
                catch (SocketException e)
                {
                    Console.WriteLine($"Socket error accepting connection: {e.Message}");
                    Console.WriteLine($"Stack trace:{Environment.NewLine}{e.StackTrace}");
                    break;
                }
                catch (Exception e)
                {
                    Console.WriteLine($"Generic error accepting connection: {e.Message}");
                    Console.WriteLine($"Stack trace:{Environment.NewLine}{e.StackTrace}");
                    break;
                }

                try
                {
                    socket.Close();
                }
                catch (Exception e)
                {
                    Console.WriteLine($"Error closing socket: {e.Message}");
                    Console.WriteLine($"Stack trace:{Environment.NewLine}{e.StackTrace}");
                    break;
                }
            }

            Console.WriteLine("Accept loop thread stopping");
        }

        static Socket AcceptConnection(TcpListener listener)
        {
            Socket socket = listener.AcceptSocket();
            try
            {
                socket.SetSocketOption(SocketOptionLevel.Tcp, SocketOptionName.Debug, 1);
                LingerOption optionValue = new LingerOption(true, 3);

                socket.SetSocketOption(SocketOptionLevel.Socket,
                    SocketOptionName.Linger, optionValue);

                return socket;
            }
            catch (Exception)
            {
                socket.Close();
                throw;
            }
        }

        static TcpListener mListener;
        const int PORT = 55000;
    }
}

tcplistenertest-1.zip

Thanks!
pablo

[EDIT] Add C# syntax highlighting by @karelz

[psantosl]

In the meantime, is there any workaround we can use? I guess we can use the async code instead? Or do we need to go for bare sockets? Any of the two would be fine, we want to get ready to release Plastic SCM in .NET Core :)

[wfurt]

I'll take a look @psantosl

[wfurt]

The problem with the code @psantosl is that the listener.AcceptSocket() is BLOCKING call.
Documentation is pretty clear about it. What it means that it won't return and unblock thread until either error happened or connection is accepted. You can see that easily by hitting the closing path and than connecting to the listening port. Once new connection is accepted, everything works as you expect e.g. code will stop listening for new connections and it will terminate.

I think your best option is using the async code as you suspected.

        static async void AcceptLoop(TcpListener listener)
        {
            // this is the main loop of the Socket based server
            while (true)
            {
                Socket socket = null;
                try
                {
                    Console.WriteLine("Accepting connection...");
                    //socket = AcceptConnection(listener);

                    socket = await listener.AcceptSocketAsync();

That does does not block the thread so when you ask for Stop() everything will terminate.

[psantosl]

Thanks @wfurt,
I know it is blocking. But if you close the listener (or the socket), the accept should exit. At least this is what you get in windows, and this has been the behavior on mono for years. I mean, yes, it blocks, no problem, but if the underlying socket is closed , it returns with an error or whatever

I will try the async code you suggest. I wonder if using a socket instead of tcplistener would work too (guess it won't)

Thanks

[wfurt]

The problem is that the close() does not happens because socket blocks for Accept().
You can also try following fragment:

       static void Main(string[] args)
        {
            Thread listenerThread = StartBackgroundThread(new ThreadStart(ThreadProc));

            while(Console.ReadLine() != string.Empty)
            {
            }
            mListener.Server.Shutdown(SocketShutdown.Both);
            mListener.Stop();
            listenerThread.Join();
        }

The shutdown will cause Accept() to fail and that breaks the blocking.

[flamencist]

You can use async version like this

public class TcpServer
	{
		#region Public.
		/// <summary>
		/// Create new instance of TcpServer.
		/// </summary>
		/// <param name="ip">Ip of server.</param>
		/// <param name="port">Port of server.</param>
		public TcpServer(string ip, int port)
		{
 			_listener = new TcpListener(IPAddress.Parse(ip), port);
		}

		/// <summary>
		/// Starts receiving incoming requests.
		/// </summary>
		public void Start()
		{
 			_listener.Start();
			_ct = _cts.Token;
			_listener.BeginAcceptTcpClient(ProcessRequest, _listener);
		}

		/// <summary>
		/// Stops receiving incoming requests.
		/// </summary>
		public void Stop()
		{ 
			//If listening has been cancelled, simply go out from method.
			if(_ct.IsCancellationRequested)
			{
				return;
			}

			//Cancels listening.
			_cts.Cancel();

			//Waits a little, to guarantee that all operation receive information about cancellation.
			Thread.Sleep(100);
			_listener.Stop();
		}
		#endregion

		#region Private.
		//Process single request.
		private void ProcessRequest(IAsyncResult ar)
		{ 
			//Stop if operation was cancelled.
			if(_ct.IsCancellationRequested)
			{
 				return;
			}
						
			var listener = ar.AsyncState as TcpListener;
			if(listener == null)
			{
 				return;
			}

			//Check cancellation again. Stop if operation was cancelled.
			if(_ct.IsCancellationRequested)
			{
 				return;
			}

			//Starts waiting for the next request.
			listener.BeginAcceptTcpClient(ProcessRequest, listener);
			
			//Gets client and starts processing received request.
			using(TcpClient client = listener.EndAcceptTcpClient(ar))
			{
 				var rp = new RequestProcessor();
				rp.Proccess(client);
			}
		}
		#endregion

		#region Fields.
		private CancellationToken _ct;
		private CancellationTokenSource _cts = new CancellationTokenSource();
		private TcpListener _listener;
		#endregion
	}

https://github.com/avgoncharov/how_to/blob/master/how_to/SimpleTcpServer/TcpServer.cs

[davidsh]

Reactivating since we reverted PR dotnet/corefx#37486.

[sguidos]

I have a DotNet Core 2.2 Console App. When targeting Windows, mListener.Stop() correctly cancels the in-progress tcpListener.AcceptTcpClient() blocking call. When targeting Linux Docker, mListener.Stop() hangs. (Note, I do not have the option of switching to the BeginAcceptTcpClient() async version at this time as flamencist suggests).

Under Linux Docker, I can add mListener.Server.Shutdown(SocketShutdown.Both) before the mListener.Stop() as wfurt suggests, and the tcpListener.AcceptTcpClient() will be correctly canceled.

However, under Windows, calling mListener.Server.Shutdown(SocketShutdown.Both) before mListener.Stop() throws an error. Catch-22 :-(

So what works for me, to correctly cancel the synchronous cpListener.AcceptTcpClient() blocking call under both Windows and Linux Docker, is:
if (Running_In_Docker) mListener.Server.Shutdown(SocketShutdown.Both);
mListener.Stop();

But of course I would prefer that the same code worked correctly and in the same way under both Windows and Linux Docker, thanks.

[davidfowl]

However, under Windows, calling mListener.Server.Shutdown(SocketShutdown.Both) before mListener.Stop() throws an error. Catch-22 :-(

Just catch the error for now?

[sguidos]

Just catch the error for now?

Yes, that's what I have done, but the inelegance of the code offends both my OCD and my sense of professionalism, LOL.

Seriously, I am amazed at how well the DotNet Core library works so well and effortlessly under both Windows and Linux Docker, and would love to see C# working perfectly and consistently in all supported environments.

[davidfowl]

https://github.com/aspnet/AspNetCore/blob/6dc65b16730abb9cba59e759012211e576f99188/src/Servers/Kestrel/Transport.Sockets/src/SocketConnectionListener.cs#L116-L125

[wfurt]

note that this will be fixed in 3.0 once dotnet/corefx#38804 is merged.

[psantosl]

Hi,

This is still open, correct??

The problem we have is that we are using websocket-sharp, and they use a listener.Close without running a Shutdown first and... well, it hangs forever on Linux 😥

[karelz]

@psantosl this has been fixed in 5.0 - see bug history and milestone.
Can you try it on 5.0?