Request timeout at server side

[sbernard31]

@chuchiliusandc says :

hi @sbernard31

i noticed that, for bootstrapped simulator with security communication, 10 min after simulator sent the reg update, the request sent by the server cannot reach the simulator anymore. hence, i got coap timeout error. (note: lifetime is set to 15 min and request from server can still reach right after the next reg update.)

however, for non-bootstrapped simulator with no security communication, this won't happen.

i think this is related to DTLS.
i feel the data in NAT is removed after a while after reg up.
Do you know how this can be fixed ?

Thanks :)

[sbernard31]

for bootstrapped simulator with security communication

What you are calling bootstrapped simulator is leshan-client-demo ?

the request sent by the server cannot reach the simulator anymore. hence, i got coap timeout error

Are you using Queue mode ?

i feel the data in NAT is removed after a while after reg up.

Are you saying your device is behind a NAT ?

[chuchiliusandc]

Hi @sbernard31

Q: What you are calling bootstrapped simulator is leshan-client-demo ?
A: not completely the leshan-client-demo. I did change leshan client simulator to have its own certificates and some user defined object.

Q: Are you using Queue mode ?
A: I am not using Queue mode. I thought Q mode is not doing much, right ? currently. Just listen reg new and reg up channel ...

Q: Are you saying your device is behind a NAT ?
A: probably not the NAT. I put leshan client in one container in my docker and leshan client in another container in docker.

Thanks :)

[sbernard31]

This is maybe link to this issue : #542

Which version of Leshan are you using ?

[chuchiliusandc]

M5 ... does M5 has your fix ?

[sbernard31]

Looking at this #542 (comment).
M5 is not affected by this bug...

[sbernard31]

Should we close this issue ?

[chuchiliusandc]

hi @sbernard31 ,

recently, after upgrading to M8, when I am doing the firmware upgrade which requires sending lots of packages to the client side (i.e., device).

i got lots of coap timeout message.
it seems to be more serious than M5 version.
the package size is like 750MB and I have about 500 devices that I want to upgrade.

have you seen this kind of issue ?

Regards, -cliu

[sbernard31]

Nope, I didn't face this kind of behavior.

About timeout, you should have a look at this page, probably some timeout setting should be modified.

Another point, you are using CoAP and blockwise transfer to download very large package ? It's maybe not really adapted for so large package, you should maybe consider to use http instead. (using packageURI, instead of package resource)

This study talk a bit of CoAP blockwise transmission time over cellular networks :
" Especially in the case of LTE the transmission time depends not only on the total amount of data,
but also on the exact sequencing of data transfer. This has been
clearly observed in the evaluation of CoAP. Its implementation
of reliable data exchange is not suitable for the transmission
of large payloads over cellular networks. Protocols based on
TCP achieve a better performance due to TCP-features like
windowing."

[chuchiliusandc]

Got it. The thing is
the timeout issue for my case (500 devices with about 750 mb package) hardly happened
when I was using M5.

After moving to M8, things become more serious.

I think I could move to M9 since M9 is using M12 of californium.
Hopefully, things will be better then.

Otherwise, I think I will try to move back to M5.

FYI. I implemented a retry mechanism on top of leshan but in a higher level (write, execute), not the package level. Used to pass eventually ... now, after upgrading, many timeouts in package level. ha

Thanks :)

[sbernard31]

Just by curiosity, how many time to transfer 750Mb using block-wise transfer ?
You probably use a lot of RAM to do that ! (you should look at this : eclipse-californium/californium#306 (comment))

[chuchiliusandc]

hi @sbernard31 ,

in my californium.properties,
i have
PREFERRED_BLOCK_SIZE=512
and
BLOCKWISE_STATUS_LIFETIME=300000

i didn't cache the package I sent on the device simulator side.
i just did some basic validation for it.

[sbernard31]

1. Which value of timeout when on send ?

 /**
     * Sends a Lightweight M2M request synchronously. Will block until a response is received from the remote client.
     * 
     * @param destination the remote client
     * @param request the request to send to the client
     * @param timeout the request timeout in millisecond
     * @return the response or <code>null</code> if the timeout expires (given parameter or CoAP timeout).
     * 
     * @throws CodecException if request payload can not be encoded.
     * @throws InterruptedException if the thread was interrupted.
     * @throws RequestRejectedException if the request is rejected by foreign peer.
     * @throws RequestCanceledException if the request is cancelled.
     * @throws InvalidResponseException if the response received is malformed.
     * @throws ClientSleepingException if the client is sleeping and then the request cannot be sent. This exception
     *         will never be raised if Queue Mode support is deactivate.
     */
    <T extends LwM2mResponse> T send(Registration destination, DownlinkRequest<T> request, long timeout)
            throws InterruptedException, CodecException, InvalidResponseException, RequestCanceledException,
            RequestRejectedException, ClientSleepingException;

2 . I would like to know how many time you take to transfer your 750Mb with block-wise when it succeed?

[chuchiliusandc]

Hi @sbernard31 ,

Initially, it was 2 min by default.
I changed it to 40 min.
We modified RedisRequestResponseHandler.java according to our needs.
I think it's taking slightly more than 20+ min for the real device (that's why i changed it from 20 min to 40 min), less than 1 min for the simulator.

Regards :)

[sbernard31]

So problem solved ?
We can close this issue ?

[chuchiliusandc]

I decided to move back the earlier version, not doing the upgrade for this release.
Perhaps, doing it in the future. Please close. Thx :)