BLE reassembly support

[carlescufi]

The BLE transport supports segmentation (i.e. splitting SMP reponses into multiple ATT_MTU-sized notifications) but not reassembly (i.e. reassembling SMP requests that have been split across multiple ATT_MTU-sized Write Commands).

CC @ccollins476ad @nvlsianpu @philips77

[carlescufi]

Note that this is required for interaction with Android phones version 5 and below, since they don't support big ATT_MTUs (I am told by @philips77)

[philips77]

This could also increase the speed of image/file upload. Right now each packet sent is ack with a notification. This takes 2 interval connections (one to send 252 bytes (from where 8 bytes is the SMP header, and number of bytes are taken by encoding offset, and "off" and "data" keys), one to receive a notification).
With reassembly implemented, an app could send a long SMP packet in 2+ BLE Write No Response packets, which can be send in a single interval connection. This saves a lot of bytes. The second+ packets would not include the header and any of CBOR fields, just the raw file content. With help of DLE the benefit is even greater.
Of course, the size of the buffer to reassembly should be calculated taking into account the interval connection. Perhaps lowering MTU would fit more packets in a single interval?
But even if only one packet could be sent in each int. conn., the benefit is worth implementing this feature.

[carlescufi]

paging @joerchan to see if he could take a look at how hard this would be to implement

[philips77]

Any movement here?
There is an idea to use Write Request and Long Write Subprocedure to send longer packets without changing MTU. Zephyr may be putting them together. It's still slow, but would allow to support Android 4.3-4.4.x.

[carlescufi]

@ccollins476ad, @sterlinghughes do you have bandwidth to take a look at this?

[ccollins476ad]

@bgiori

Re: Write Long Characteristic Values procedure- My "BLE" is a bit rusty, so please correct me if I am wrong. During a long write procedure, each fragment must be acknowledged by the server (Prepare Write Response). The protocol would still effectively be stop-and-wait, so I am not sure this would help much with performance.

I think we would want to do this at the application layer. Then we can still use unacknowledged writes. I can think of two ways we may want to achieve this (my apologies if I am rehashing things that have already been discussed):

1. Generic fragmentation mechanism. This is probably what you'd expect: the server buffers each fragment until the entire packet is received, then reassembles and processes.

Good: Works for all SMP command types.
Bad: Requires a lot of buffer space on the server.

2. DFU-specific mechanism. The image upload requests are not fragmented. Instead, the requests are selectively acknowledged. The server writes image chunks to flash as it receives them, but only acknowledges every X requests (or every X bytes). Both sides understand the acknowledgement policy, so the client can rapidly send X requests, then listen for a response.

Good: Does not require extra buffer space on the server.
Bad: a) Only helps with DFU; b) probably more complicated to implement.

Thoughts?

[philips77]

The protocol would still effectively be stop-and-wait, so I am not sure this would help much with performance.

That would not help for performance, but would allow to update from an Android 4.3-4.4.x phone where the MTU was fixed at 23.

Thoughts?

Well, I think 1. is more generic, so better. The SMP could return an error if payload length is set to a too high value in the first packet, possibly with buffer size, so that the client know what's the max value. But 2. could also be acceptable. Currently the issue is only with DFU, but I can imagine it may appear in FS manager as well, when a big file is to be sent.

For mobile apps 1. is easier to support, but 2. not much harder.

[pdunaj]

We will also need this as we fight to lower RAM usage.

[bgiori]

@ccollins476ad I think both (1) and (2) would be good additions and could work in tandem.

(1) Would be generally useful on both sides. (2) would help with firmware upload speeds, and could be implemented with an additional value sent in the initial request's payload to set the number of n of requests before expecting a response. A separate issue could be made for (2) to discuss details.