Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exception when to disconnect. #92

Open
juliolitwin opened this issue Apr 23, 2020 · 14 comments
Open

Exception when to disconnect. #92

juliolitwin opened this issue Apr 23, 2020 · 14 comments

Comments

@juliolitwin
Copy link

Hi,

I do not know if it is a case that was already expected, but curious about Bedrock, I developed a simple client that can connect 500 simultaneous clients and send packets, so far everything is happening normally, but when I close the client (forcing the server to lose connection with all clients), it simply generates exceptions for each disconnection, but the problem is that it ends up making the server slow with so many exceptions, without being able to do new actions, such as accepting new connections or new packets.

I'm forgetting something?
Cheers.

@davidfowl
Copy link
Owner

How did you close the connection? You get an exception if there's a TCP reset but not a FIN.

@juliolitwin
Copy link
Author

juliolitwin commented Apr 28, 2020

I just force my client to close. It's gave me reset and "An existing connection was forcibly closed by the remote host."

Force closing is not the correct way, but in a normal socket these problems do not happen, is there any way to prevent me from the BedRock being stopped while disconnecting the connections and with the high use of cpu (because it is generating many exceptions)?

Because if there is no solution, anyone can attack by creating a simple client and try to connect several clients and then force the closing and cause this problem on the server.

With IntelliTrace using the 'snapshots (managed and native)' option, it is possible to see exceptions more clearly.

@davidfowl
Copy link
Owner

@juliolitwin Do you see high CPU repeatedly or do you see it once? Did you run a single performance test where clients connect and all disconnect or did you do it multiple times?

@juliolitwin
Copy link
Author

@davidfowl Whenever I force clients to close, exceptions occur and the CPU increases.

I recorded a GIF to show you, I'm using 1000 connections to be more visible and after I close the client, I wait a moment for you to see that the CPU process is "stopped" at a point in the diagnosis and it goes up to the high point and is stopped even when all clients are disconnected and gives all exceptions. Then I open the client again, just to show that the server is unviable, it cannot accept new connections because it is still processing.

Gif: https://gifyu.com/image/l5vv

@davidfowl
Copy link
Owner

I've seen similar behavior before, but mostly on the first set of disconnects. Try this:

  • Run in release mode
  • Run without logging at all (you have debug logging on in that sample).
  • Do multiple runs with the same server process running and check the CPU.

@juliolitwin
Copy link
Author

Oops, in the gif I enabled debugging just for you to see the exceptions, but with the log disabled and in the release mode it still keep giving samething.

About running the client several times, the cpu has no effect, as it does not accept any connections, it is as if the network thread / async was totally busy processing the exceptions and my client only return me saying can't to connect.

@davidfowl
Copy link
Owner

About running the client several times, the cpu has no effect, as it does not accept any connections, it is as if the network thread / async was totally busy processing the exceptions and my client only return me saying can't to connect.

Wait until the clients disconnect, then try again and do another run with the server still running and a new set of clients. What I'm asking is to run the test with clients multiple times with the same server instance.

@juliolitwin
Copy link
Author

I decrease the number of connections because it takes too long to disconnect all clients and also decreases the rate of packets sent. I'm using server in release and record two versions with log and without log.

GIF (no log): https://gifyu.com/image/l5nW

GIF (with log): https://gifyu.com/image/l5Wm

Strange that with 1000 connections, when I close and wait for the server to process and then try to open the client is unable to make a connection.

@davidfowl
Copy link
Owner

Can you also stop debugging? That slows things down significantly.

@juliolitwin
Copy link
Author

It really made a drastic difference.

@juliolitwin
Copy link
Author

@davidfowl Even though BedrockFramework using aspnetcore as a base, do you already consider the project ready for production use? Are there any recommendation to limit connections per socket?

@davidfowl
Copy link
Owner

@davidfowl Even though BedrockFramework using aspnetcore as a base, do you already consider the project ready for production use?

Yes it's pretty solid. There are still some things I'd like to bring over from experimental before shipping but haven't had time to spend on it as yet.

Are there any recommendation to limit connections per socket?

I assume you mean limit general connections? There's nothing in here but it's trivial to add.

@juliolitwin
Copy link
Author

In MyCustomProtocol in readasync, I can't get the received packet array, as in the example on the line:

_logger.LogInformation("Received a message of {Length} bytes", message.Payload.Length);

Is this expected? Because it only processes the packet received internally but I can't get the packet.

@davidfowl
Copy link
Owner

In MyCustomProtocol in readasync, I can't get the received packet array, as in the example on the line:

Not sure what you mean...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants