WebSocket configuration doesn't work properly with the buffer size limitation

[OleksiiKlochko]

Affects: 5.2.8.RELEASE

---

@Configuration
@EnableWebSocketMessageBroker
public class MyConfig implements WebSocketMessageBrokerConfigurer {

	@Override
	public void configureMessageBroker(MessageBrokerRegistry registry) {
		// ...
		registry.setPreservePublishOrder(true);
	}

	@Override
	public void configureWebSocketTransport(WebSocketTransportRegistration registry) {
		registry.setSendBufferSizeLimit(512 * 1024);
	}
}

The setSendBufferSizeLimit method has no more effect after setPreservePublishOrder(true).

The reason is that the instance of OrderedMessageSender keeps all messages and sends only one message at a time to the buffer.

Thus, the application can slow down significantly or even crash with OutOfMemory Error instead of just closing the WebSocket connection when the messages queue overflow.

Memory Analyzer shows how OrderedMessageSender can occupy the whole heap:

[rstoyanchev]

Indeed, sendBufferSizeLimit is at a much lower (transport) level while OrderedMessageSender buffers at the broker. Further the sendBufferSizeLimit is per session while OrderedMessageSender has one queue. That means one slow session/connection can force other sessions to wait and the queue to fill up.

Technically only messages within the same session need to be serialized, so we can explore an improvement where messages are held up only if there is another message from the same session already in progress. In addition we could also implement some sort of timeout at the level of OrderedMessageSender.

[rstoyanchev]

@OleksiiKlochko I've made a change so that messages are processed one at a time but only within a session and not across all sessions. This should remove a head-of-line blocking where slow sessions affect others.

Would you be able to give this a try using 5.3 snapshots? It is a significant improvement of throughput and might be sufficient to address your issue.

[OleksiiKlochko]

@rstoyanchev I dug deeper and found that we have one OrderedMessageSender per session. Please see

spring-framework/spring-messaging/src/main/java/org/springframework/messaging/simp/broker/AbstractBrokerMessageHandler.java

Lines 294 to 302 in 7288ae1

	/**
	* Get the MessageChannel to use for sending messages to clients, possibly
	* a per-session wrapper when {@code preservePublishOrder=true}.
	* @since 5.1
	*/
	protected MessageChannel getClientOutboundChannelForSession(String sessionId) {
	return this.preservePublishOrder ?
	new OrderedMessageSender(getClientOutboundChannel(), logger) : getClientOutboundChannel();
	}

I might be wrong but the current fix is not suitable.

I suppose that we should have the same sendBufferSizeLimit in OrderedMessageSender as for WebSocketTransportRegistration. This provides consistent behavior for ordered and unordered processing.

Thanks!

[rstoyanchev]

Thanks for taking a look. Indeed I overlooked that OrderedMessageSender is a per-session wrapper around the singleton MessageChannel. I reverted the commits.

Even ff the same limits are replicated, the challenge is what to do when the limit is reached. A MessageChannel can only pass messages, it can't close the session for example.

I think we have to avoid waiting until the current message is fully processed which includes the time to send. For the purpose of ordering as soon as the message is queued up in ConcurrentWebSocketSessionDecorator we can release the next. That will allow the existing send limits be enforced. I'll experiment with how this can be done.