gcoap: add some client-side observe handling

[MichelRottleuthner]

Contribution description

Enables some client-side observe operation of gcoap (and improve server side a bit).
Currently, when registering an observe from a gcoap client, the first notification consumes the request memo state. Any following notifications are therefore dropped silently afterwards.
When I Initially saw this behavior I thought it's a bug, but turns out gcoap never had client-side observe handling 😮
This PR tries to improve the situation like this: requests initiated with an observe option are simply kept open beyond the first response.

Testing procedure

The first commit of this PR adds some client side observe functionality to examples/gcoap to reproduce this behavior.
With two native instances, check out the first commit and follow below steps.

Step 1: Reproduce missing feature.

sudo ./dist/tools/tapsetup/tapsetup
BOARD=native PORT=tap0 make -C examples/gcoap all term
> ifconfig
(...)
# *copy node_1_IP IP* and  continue on node_2 in second term
coap get -o [<node_2_IP>] /cli/stats
gcoap_cli: sending msg ID 9556, 17 bytes
> gcoap: response Success, code 2.05, 1 bytes
0

Meanwhile in a second terminal:

BOARD=native PORT=tap1 make -C examples/gcoap all term
> ifconfig
(...)
# *copy node_2_IP IP*
coap get [<node_1_IP>] /.well-known/core
# sad, no notification arrived on node_1, try again?
coap get [<node_1_IP>] /.well-known/core
# nope, nothing happening, but wireshark says the server is doing what it should do

Step 2: hopefully make it better.

The second commit of this PR should fix this. So git checkout somethingsomething, and repeat.
This time, doing many coap get [<node_1_IP>] /.well-known/core (term2) should something like this on node_1:

(...)
coap get -o [<node_2_IP>] /cli/stats
gcoap_cli: sending msg ID 1769, 17 bytes
> gcoap: response Success, code 2.05, 1 bytes
0
gcoap: response Success, code 2.05, 1 bytes
1
gcoap: response Success, code 2.05, 1 bytes
2
gcoap: response Success, code 2.05, 1 bytes
3
gcoap: response Success, code 2.05, 1 bytes
4
gcoap: response Success, code 2.05, 1 bytes
5
(...)

Noice. Done, right? ...and if we want to get rid of that perma-request again? ...no problem, just, ahh reboot? 🤨
Jump to commit No.6 of this PR for a solution (and more problems).

Repeat step 1 and 2, then make node1 forget about the observe via the new -d (deregister) option like this:
coap get -d [<node_2_IP>] /cli/stats
According to RFC7641, 3.6 Cancellation this implementation variant of simply "forgetting about the observe" should do.
For node1 (the observed observing client) it indeed looks fine now as the response handler isn't called anymore. But wireshark shows that node2 is still sending all the notifications.
This is actually a shortcoming of the server-side handling, as gcoap currently does not handle a received RST after it sent a notification.
This is fixed with commit No.7.

The last "REMOVEME" commit shows an attempt of how explicitly sending a de-registration request could reuse the existing request memo of the observe. It also shows why we probably shouldn't do that ;)
It is much more nasty than I expected. Some not really related changes to nanocoap are also part of that droppable commit.
Luckily, with the other changes of this PR this is not strictly necessary. Maybe it helps finding a more elegant solution.

TODO: interop with some aiocoap-ish thing?

Issues/PRs references

[Teufelchen1]

🦩

[MichelRottleuthner]

Note to myself and people interested in testing: a quick test showed that the server side of aiocoap also doesn't handle an RST returned after a notification. In my understanding this is valid behavior as per RFC a server can always safely pretend it didn't receive the RST. For the client side it indicates that it might be better to indeed also add a feature for explicit deregistration (i.e. sending a GET with observe option set to 1).

[MichelRottleuthner]

(Rebased to fix conflicts with upstream changes and also squashed the fixups.)

Another data point regarding the handling of silently forgetting observations on the client side: libcoap treats this well (on RST it removes stale registration).

[MrKevinWeiss]

Just after some IRL conversation...

I think maybe it would be better to make this an optional module (even though the server observe is not)... I also don't think there is a rush for this as @MichelRottleuthner will be busy for the next little while.

Maybe @bergzand might have some opinions on this as well.

I will start with a bit more automation on the release specs and see if I can get something that can test this... eventually.

[leandrolanzieri]

Nice! Here are some comments. I agree that 4256885 may not be the way. Given that de-registration of observations is kind of manual, and there are two options, I think we could really use some documentation on that.

[leandrolanzieri]

I think maybe it would be better to make this an optional module (even though the server observe is not)... I also don't think there is a rush for this as @MichelRottleuthner will be busy for the next little while.

Why? And also, how? The "forget" function would be optimized away if not used.

[leandrolanzieri]

I tested this with an aiocoap server, the observe notifications and the observation cancellation worked like a charm.

[riot-ci]

Murdock results

✔️ PASSED

12982a0 gcoap: update documentation on supported features

Success	Failures	Total	Runtime
10008	0	10010	15m:12s

Artifacts

• Documentation preview

[leandrolanzieri]

@MichelRottleuthner please squash and rebase

[MichelRottleuthner]

Whoopsie I only squashed. Rebased now.

[leandrolanzieri]

ACK!