[Firmware Update][Blockwise]

[mathns28]

Question

Hello, I'm newer in lwm2m and I'm learning, so if I miss something forgive me.

I made a file transfer using the CaliforniumServerEndpointsProvider as base. I changed the RootResource for other implementation that controls the files.

The class I get from here: #70 (comment)

I tested locally and It works pretty well, but when I tested on the server I get every time a delay:

logPart.txt

This occurs randomly, sometimes after about 20 transmissions, sometimes after 500 transmissions.

The server configuration that I'm using is here. This is in default mode, I only changed the MAX_RESOURCE_BODY_SIZE, PREFERRED_BLOCK_SIZE and MAX_RETRANSMIT, because the files are more than 300Mb.

config.txt

I don't know where is the error, if I lost something, if I need to increase some value on the configs, or decrise, if I can get more logs to validate the message...

I would be grateful if someone could help me!!

[sbernard31]

I think this more a Californium question...

The class I get from here: #70 (comment)

I don't know what you refer ? cf-simplefile-server ?

I get every time a delay ?

What does it mean exactly ?

the files are more than 300Mb.

Mb (mega bits) or MB (mega byte) ?
Looking at log, I see block2 message with size2 option=144042036, we are more around 150MB than 300... 🤔
It seems that block size (szx) is 512.

{
 "Content-Format":"application/octet-stream",
 "Block2":"(szx=5/512, m=true, num=678)",
 "Size2":144042036
}

I don't know if I'm wrong somewhere but this means : 144042036 x 512 = 281332 message.
Knowing that MID (CoAP Message ID) is 16bits length.
Maybe you face a limitation because of that ...

(The Message ID is compact; its 16-bit size enables up to about 250 messages per
second from one endpoint to another with default protocol
parameters.)

See also 4.4 - Message Correlation:

(but If this is the reason, you should get same behavior when you test localy 🤔)

So, I don't know ... again ask to Californium Project, they should know better than me.

Also using CoAP for large payload could be a not so good idea : https://github.com/eclipse-leshan/leshan/wiki/Using-CoAP-Block-Wise-to-transfer-large-file-%3F

[mathns28]

About:

I don't know what you refer ? cf-simplefile-server ?

I got this SimpleFileServer and I put it as a RootResource.

What does it mean exactly ?

After a while the server stops to receive the client ACK. The server sends the exchange, but I dont see the response.
There is a way to see all the UDP trafic? I don't have the client logs.

Mb (mega bits) or MB (mega byte) ?
Looking at log, I see block2 message with size2 option=144042036, we are more around 150MB than 300... 🤔
It seems that block size (szx) is 512.

In this case, the file has 150MB almost, but I have one file that has 450MB.

I don't know if I'm wrong somewhere but this means : 144042036 x 512 = 281332 message.
Knowing that MID (CoAP Message ID) is 16bits length.
Maybe you face a limitation because of that ...

I don't know if The problem is there, because I got the file after a time locally.
Can I

Also using CoAP for large payload could be a not so good idea : https://github.com/eclipse-leshan/leshan/wiki/Using-CoAP-Block-Wise-to-transfer-large-file-%3F

In the firmware updates, can I write directly into 5/0/0, even if the client only supports COAP and COAPS requests?

[sbernard31]

There is a way to see all the UDP trafic?

Wireshark / tcpdump : https://github.com/eclipse-californium/californium/wiki/Logs-and-IP-Capturing-%E2%80%90-How-To-Provide-The-Right-Information#ip-capturing

In the firmware updates, can I write directly into 5/0/0, even if the client only supports COAP and COAPS requests?

Theoretically, you can. In practice, I'm not sure is a good idea for reason explain in wiki page I shared.

[jvermillard]

if you write to /5/0/0 large resource, be careful the default LeshanServer::send() timeout is 2 minutes; if you continue to send such a large payload, it will timeout for sure. To add my 2cents about why it's not a good idea: https://vermillard.com/post/coap-vs-tcp/

[sbernard31]

To add my 2cents about why it's not a good idea: https://vermillard.com/post/coap-vs-tcp/

Maybe RFC9177 - Constrained Application Protocol (CoAP) Block-Wise Transfer Options Supporting Robust Transmission could help but I'm not sure there is implementation available. (For sure neither Californium or java-coap support it)

This document specifies alternative Constrained Application Protocol (CoAP) block-wise transfer options: Q-Block1 and Q-Block2.These options are similar to, but distinct from, the CoAP Block1 and Block2 options defined in RFC 7959. The Q-Block1 and Q-Block2 options are not intended to replace the Block1 and Block2 options but rather have the goal of supporting Non-confirmable (NON) messages for large amounts of data with fewer packet interchanges. Also, the Q-Block1 and Q-Block2 options support faster recovery should any of the blocks get lost in transmission.

[mathns28]

I tryed to change the blockwise parammeters without success.

But, something interesting that I see in my tests was the Deduplicator.
I read this https://www.rfc-editor.org/rfc/rfc7252#section-4.5 but I did not get it.

The server's behavior is this, at first everything goes well, with CoapServer in main and the moment the problem occurs is when org.ecl.cal.cor.net.ded.SweepDeduplicator is called.
log.txt

What does It means?

[sbernard31]

I looked at the log and I only see that about Deduplicator:

iws10-wits-integration-firmware          | 2024-05-06 22:42:35,794 DEBUG [org.ecl.cal.cor.net.ded.SweepDeduplicator](CoapServer(secondary)#2) Sweep run took 0ms
iws10-wits-integration-firmware          | 2024-05-06 22:42:45,794 DEBUG [org.ecl.cal.cor.net.ded.SweepDeduplicator](CoapServer(secondary)#2) Sweep run took 0ms
iws10-wits-integration-firmware          | 2024-05-06 22:42:55,794 DEBUG [org.ecl.cal.cor.net.ded.SweepDeduplicator](CoapServer(secondary)#2) Sweep run took 0ms
iws10-wits-integration-firmware          | 2024-05-06 22:43:05,794 DEBUG [org.ecl.cal.cor.net.ded.SweepDeduplicator](CoapServer(secondary)#2) Sweep run took 0ms
iws10-wits-integration-firmware          | 2024-05-06 22:43:15,794 DEBUG [org.ecl.cal.cor.net.ded.SweepDeduplicator](CoapServer(secondary)#1) Sweep run took 0ms
iws10-wits-integration-firmware          | 2024-05-06 22:43:25,794 DEBUG [org.ecl.cal.cor.net.ded.SweepDeduplicator] (CoapServer(secondary)#1) Sweep run took 0ms
iws10-wits-integration-firmware          | 2024-05-06 22:43:35,794 DEBUG [org.ecl.cal.cor.net.ded.SweepDeduplicator] (CoapServer(secondary)#1) Sweep run took 0ms

I didn't check but I just guess that this a thread witch execute the "sweep" task every 10 seconds.
If you want more precise answer about Californium code, you should ask to californium project 😅

Again maybe you face a protocol limitation regarding the number of block message you need to send and the MID size but I guess this log isn't related to that.

Knowing that you want to download a firmware package ~150 / 450 MB, I guess your device is not a so constraint one.
So why not just using HTTP to download the package .... ?