Use Leshan library to simulate several clients

[sbernard31]

This issue aims to centralize information about using Leshan to simulate several clients.

Here is known issue, generally this is about using too many thread:

Problem	Github issue	Possible clean solution	Known workaround
UDPConnector of californium need at least 2 thread by client	eclipse-californium/californium#1203	Implement connector based on Netty ?
DTLSConnector of californium need at least 1 thread by client	eclipse-californium/californium#1203	Implement connector based on Netty ?	TBD
Not clear DefaultRegistrationEngine blocking issue	#1574 (comment)		TBD

Possible solution/workaround about using too many thread:

• Trying new Virtual Thread of Java 21. (not tested, feedback is welcome)
• Trying to increase your OS Thread limit.

Number of port limitation:
Using several clients on the same machine may reach to the limit of available ports. (see : #1406 (comment))

A solution could be to implement queue mode and so sleeping client could free IP port and so you will be able to simulate more client by machine but the maximum of client talking at the same time is still limited by the maximum number of port (~35k).

Here is old issue related to that:

• Use Leshan Client to launch thousands LWM2M clients in the same VM. #491
• Scaling Leshan v2.x to 10,000 Clients, General Performance Query.  #1406

Here is a tool which aims to to that : https://github.com/sbernard31/benchmark-clients

(Content of this issue will be updated when we get new information)

[jvermillard]

About thread limitation, using https://github.com/golioth/xk6-coap, which is Go-based (a similar solution to Java 21 virtual threads), will overcome this (with some extra benefits of the k6 performance framework)

Another solution for the number of ports is to run multiple interfaces on the client machine because the 16-bit port limitation is a per-interface limitation. I wonder if you can do this by running one docker container for the server and multiple docker containers for the client and keeping the traffic in the docker network.

Also, you can get more ports per interface by changing this kernel parameter: /proc/sys/net/ipv4/ip_local_port_range
Example: echo 10000 64000 > /proc/sys/net/ipv4/ip_local_port_range

[sbernard31]

About thread limitation, using https://github.com/golioth/xk6-coap, which is Go-based (a similar solution to Java 21 virtual threads), will overcome this (with some extra benefits of the k6 performance framework)

Tell, me If I misunderstood : this is an alternative of "Using Leshan library to simulate several clients", if you use it then you don't use Leshan client at all ?

[jvermillard]

Yes you are right. When I need to do these kinds of tests I often need to simulate the CoAP part (for example the registration) and also use some HTTP API to verify things on the server or trigger more commands like LWM2M reads. So that's why I'm exploring this way of load testing CoAP/LWM2M servers.