sys/suit: initial support for SUIT firmware updates #11818

kaspar030 · 2019-07-08T13:24:34Z

Contribution description

This PR contains initial support for updating RIOT using an implementation of SUIT draft version 4.

This currently only supports a limited use case: one component, two slots (one active one passive). Basically, it allows SUIT compliant infrastructure and tooling to update a RIOT node.

Note: The implementation should not yet be considered production ready.

The implementation is based on a draft spec that will change in the near future. The implementation has also not yet gone through any security audit.
This should still be usable for less security sensitive deployments, and by getting this into master, we hope to get more exposure and feedback going towards production readiness.

The RIOT-fp project commits to maintaining and improving this implementation.

The PR is grouped in three commits:

one commit adding the sys/suit module, including manifest handling and a CoAP based update service
one commit adding necessary tooling (key- and manifest-generation) to dist/tools/suit
one commit adding examples/suit_update, show-casing SUIT usage in addition to providing an automated test case.

The commit history looks quite large due to the large list of dependencies. This PR should serve as tracking PR as long as there are open dependencies.

Testing procedure

The PR ships with an example application in examples/suit_update, which includes testing instructions using an automated test, which tests updating each slot over ethos.
The test is also run by the CI.

Apart from verifying that updating works for a given node, it would be very helpful to get feedback on the documentation and whether it describes the necessary steps in your setup.

Issues/PRs references

Depends on ~~#11801~~, ~~#11802~~, ~~#11803~~, ~~#11805~~, ~~#11816~~, ~~#11707~~, ~~#11690~~, ~~#11697~~, and ~~riotdocker #73~~

kaspar030 · 2019-07-08T13:45:36Z

removed dependency to nanocoap: make coap_get_block2() actually fill struct #11802, updated graph

kaspar030 · 2019-07-08T14:39:54Z

CI is missing some python libraries for manifest generation... RIOT-OS/riotdocker#73

kaspar030 · 2019-07-08T14:42:46Z

riotboot_flashwrite issues are fixed in riotboot: set FLASHFILE to RIOTBOOT_EXTENDED_BIN if riotboot feature is used #11690
libcose issues will be dealt with in pkg/libcose: bump version #11801

kaspar030 · 2019-07-10T10:23:01Z

rebased to remove dependency on nanocoap: make coap_get_block2() actually fill struct #11802 and to include updated riotboot: set FLASHFILE to RIOTBOOT_EXTENDED_BIN if riotboot feature is used #11690
fixed digest extraction

fjmolinas · 2019-07-10T10:37:05Z

sys/suit/coap.c

+        }
+
+#endif
+        if (res == 0) {


While reviewing #11805 I realized res is uninitialized if MODULE_SUIT_V4 is not used.

still needs addressing

kaspar030 · 2019-07-10T15:52:49Z

rebased with sys/riotboot: add initial image digest verification #11805 merged

kaspar030 · 2019-07-12T09:27:54Z

Found an issue with the key generation targets not being properly serialized before "clean".
Split out a new "CLEAN" variable into #11829.

kaspar030 · 2019-07-15T13:19:40Z

The dependency graph is getting smaller! 😄

kaspar030 · 2019-07-18T07:47:52Z

rebased after pkg/libcose: bump version #11801 merged. Only the CI PR left!

jcarrano

This PR has way too many lines with no tests. Please provide tests.

fjmolinas · 2019-10-09T14:12:24Z

There's an issue with nanocoap_server choosing the wrong source address for a coap reply. IMO this is unrelated, but causes the test to fail. We're investigating.

I had pointed out the initial issue IRL to @kaspar030

To be clear what was happening is that aiocoap was sending a trigger to the devices EUI64 link local address, so [fe80::20d:46ff:fe3b:adf0%riot0] but for some reason the ACK received by aiocoap did not come from the EUI64 link local address but from [fe80::2%riot0] and there was no more observed traffic after that, the test stalled.

aiocoap-client -m POST "coap://[fe80::20d:46ff:fe3b:adf0%riot0]/suit/trigger" \
	--payload "coap://[fd00:dead:beef::1]/fw/samr21-xpro/$(basename /home/francisco/workspace/RIOT/examples/suit_update/bin/samr21-xpro/suit_update-riot.suitv4_signed.latest.bin)" && \
	echo "Triggered [fe80::20d:46ff:fe3b:adf0%riot0] to update."
WARNING:coap:Received Type.ACK from <UDP6EndpointAddress [fe80::2%riot0]:5683 with local address>, but could not match it to a running exchange

All ACK from the device are with [fe80::2%riot0], so aiocoap keeps POSTS to the original address expecting a response from [fe80::20d:46ff:fe3b:adf0%riot0].

At first it answers correctly to the block requests from [fe80::20d:46ff:fe3b:adf0%riot0], but at some point it answers with destination unreachable (port unreachable)

This issue appears sometimes after flashing, setting up the network and launching the tests. When it appears setting-up the network again multiple times ends up fixing it.

I would say this would be fixed by #12404. EDIT: will test

fjmolinas · 2019-10-09T14:43:18Z

I'm having issues in finding a reliable way to reproduce this issue. I've tried testing a bunch on time rebased on #12404, and I haven't been able to reproduce the bug.

It shows up (edit: in this PR withouth #12404) from time to time when executing the setup for the automatic tests and launching. (I have to re-do the setup each time for it to work)

sudo dist/tools/ethos/setup_network.sh riot0 2001:db8::/64

BOARD=samr21-xpro make -C examples/suit_update test

BOARD=samr21-xpro make -C examples/suit_update test

kaspar030 · 2019-10-09T16:34:20Z

I've pushed a commit to disable testing on CI until the source address selection is resolved.

fjmolinas

All my request changes have been addressed and I have no issues with the code (I trust @aabadie review of the python code).

There is still work to be done, parts of the standard to be supported and the tests can be improved. But there are already PR's and current work to address this. The code is completely isolated so it should not impact any other feature, so I find it safe to merge this during soft-feature-freeze. This will also make it easy to improve and change the code base in the feature. This is a good initial support for suit and ota.

I have tested this code extensively: locally, over ethos and in wireless setups, with nodes continuously updating. There are some issues regarding network setup, explicitly when multiple addresses are present (link-local or global), but this is an issue with the source address selection, so unrelated (although it impacts this PR), I have stated the details of this issue and #12404, #12408 partially address this issue. I agree with disabling it in murdock since we are not sure it can be reliably ran in the CI. Running it locally still allows verifying the code and its functionality. IMO this is not a reason to block this PR.

The documentation is detailed in explaining the usage, and has been tested multiple times, and improved accordingly. The feature is tagged as @experimental, so should be treated as such by users.

What is your stance here @aabadie?

aabadie

What is your stance here @aabadie?

I think it's time to move forward with secure ota and suit. I know @fjmolinas tested this work extensively and I also did it a bit. This is working well.
The python code is good enough to be merged.
There's room for improvement, of course, but it can and will come in follow-up PRs, for sure.

ACK!

miri64 · 2019-10-09T18:58:08Z

https://ci.riot-os.org/RIOT-OS/RIOT/11818/3addde53777b489a901b1cecf1ff43ef4ddb4569/output/compile/tests/pkg_u8g2/esp32-wemos-lolin-d32-pro:gnu.txt Is the worker out of disk space, or what happened here?

kaspar030 · 2019-10-09T19:26:44Z

Is the worker out of disk space, or what happened here?

Seems like it ran out of tmpfs space.

fjmolinas · 2019-10-10T08:28:17Z

@kaspar030 go ahead and click the button :) !

kaspar030 · 2019-10-10T08:38:25Z

Congrats everyone! 😄

miri64 · 2019-10-10T08:40:05Z

🎉

bergzand · 2019-10-10T09:03:49Z

🎉

emmanuelsearch · 2019-10-10T09:22:10Z

Yay! Congrats to all involved! 🎉

benpicco · 2020-03-23T16:00:26Z

sys/suit/coap.c

+    if (size >= 0) {
+        LOG_INFO("suit_coap: got manifest with size %u\n", (unsigned)size);
+
+        riotboot_flashwrite_t writer;


This contains a uint8_t flashpage_buf[FLASHPAGE_SIZE]; with FLASHPAGE_SIZE being up to 8k on some platforms - is it a good idea to put that on the stack?

benpicco · 2020-04-09T10:40:33Z

sys/suit/coap.c

+#endif
+
+#ifndef SUIT_MANIFEST_BUFSIZE
+#define SUIT_MANIFEST_BUFSIZE   640


Where does that number come from?

benpicco · 2020-11-10T12:58:14Z

sys/suit/coap.c

+            riotboot_hdr_print(hdr);
+            xtimer_sleep(1);


What is this used for? 😉

benpicco · 2020-11-10T12:58:59Z

sys/suit/coap.c

+    }
+}
+
+ssize_t suit_coap_get_blockwise_url_buf(const char *url,


This is not in any header - should be static?

benpicco · 2020-11-10T12:59:43Z

sys/suit/coap.c

+    return (res < 0) ? (ssize_t)res : (ssize_t)_buf.offset;
+}
+
+static void _suit_handle_url(const char *url)


Should we export that so it can be used without the suit thread?

benpicco · 2022-01-05T22:28:06Z

sys/suit/coap.c

+                               coap_blockwise_cb_t callback, void *arg)
+{
+    /* mmmmh dynamically sized array */
+    uint8_t buf[64 + (0x1 << (blksize + 4))];


Why the added 64?

kaspar030 requested a review from bergzand July 8, 2019 13:24

fjmolinas mentioned this pull request Jul 8, 2019

Release 2019.07 - RC1 RIOT-OS/Release-Specs#128

Closed

77 tasks

kaspar030 mentioned this pull request Jul 8, 2019

add python dependencies for SUIT RIOT-OS/riotdocker#73

Merged

kaspar030 mentioned this pull request Jul 8, 2019

sys/riotboot: add initial image digest verification #11805

Merged

kaspar030 force-pushed the suit-pr branch from 760f978 to ce84ac4 Compare July 10, 2019 10:21

fjmolinas reviewed Jul 10, 2019

View reviewed changes

kaspar030 force-pushed the suit-pr branch from ce84ac4 to 638e6b1 Compare July 10, 2019 15:49

kaspar030 force-pushed the suit-pr branch 2 times, most recently from 80256bf to 57eeffa Compare July 12, 2019 09:21

kaspar030 removed the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Jul 12, 2019

This was referenced Jul 12, 2019

make: introduce $(CLEAN) #11829

Merged

dist/tools/ethos: add setup_network.sh script #11816

Merged

dist/tools/ethos: deduplicate setup_network.sh, start_network.sh #11840

Closed

kaspar030 force-pushed the suit-pr branch from 57eeffa to 9c06672 Compare July 15, 2019 13:17

fjmolinas mentioned this pull request Jul 17, 2019

pkg/libcose: bump version #11801

Merged

kaspar030 force-pushed the suit-pr branch from 9c06672 to 2171186 Compare July 18, 2019 07:45

jcarrano previously requested changes Jul 18, 2019

View reviewed changes

kaspar030 mentioned this pull request Oct 9, 2019

gnrc/netif: don't treat link local differently in prefix matching #12404

Closed

examples/suit_update: disable CI test for now

3addde5

fjmolinas approved these changes Oct 9, 2019

View reviewed changes

aabadie approved these changes Oct 9, 2019

View reviewed changes

kaspar030 added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR and removed CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Oct 9, 2019

kaspar030 merged commit 5ab8d92 into RIOT-OS:master Oct 10, 2019

kaspar030 deleted the suit-pr branch October 10, 2019 08:41

fjmolinas mentioned this pull request Oct 24, 2019

examples/suit_update:enable murdock #12563

Merged

kb2ma added this to the Release 2019.10 milestone Oct 28, 2019

smlng mentioned this pull request Dec 5, 2019

make: replace GNU cp -t with standard cp #12875

Merged

benpicco reviewed Mar 23, 2020

View reviewed changes

benpicco reviewed Apr 9, 2020

View reviewed changes

sys/suit/coap.c

#endif

#ifndef SUIT_MANIFEST_BUFSIZE

#define SUIT_MANIFEST_BUFSIZE 640

Copy link

Contributor

benpicco Apr 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where does that number come from?

benpicco reviewed Nov 10, 2020

View reviewed changes

sys/suit/coap.c

}

}

ssize_t suit_coap_get_blockwise_url_buf(const char *url,

Copy link

Contributor

benpicco Nov 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not in any header - should be static?

benpicco reviewed Nov 10, 2020

View reviewed changes

benpicco reviewed Jan 5, 2022

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

sys/suit: initial support for SUIT firmware updates #11818

sys/suit: initial support for SUIT firmware updates #11818

kaspar030 commented Jul 8, 2019 •

edited by emmanuelsearch

Loading

kaspar030 commented Jul 8, 2019

kaspar030 commented Jul 8, 2019

kaspar030 commented Jul 8, 2019

kaspar030 commented Jul 10, 2019

fjmolinas Jul 10, 2019

fjmolinas Sep 27, 2019

kaspar030 commented Jul 10, 2019

kaspar030 commented Jul 12, 2019

kaspar030 commented Jul 15, 2019

kaspar030 commented Jul 18, 2019

jcarrano left a comment

fjmolinas commented Oct 9, 2019 •

edited

Loading

fjmolinas commented Oct 9, 2019 •

edited

Loading

kaspar030 commented Oct 9, 2019

fjmolinas left a comment

aabadie left a comment

miri64 commented Oct 9, 2019

kaspar030 commented Oct 9, 2019

fjmolinas commented Oct 10, 2019 •

edited

Loading

kaspar030 commented Oct 10, 2019

miri64 commented Oct 10, 2019

bergzand commented Oct 10, 2019

emmanuelsearch commented Oct 10, 2019

benpicco Mar 23, 2020

benpicco Apr 9, 2020

benpicco Nov 10, 2020

benpicco Nov 10, 2020

benpicco Nov 10, 2020

benpicco Jan 5, 2022

sys/suit: initial support for SUIT firmware updates #11818

sys/suit: initial support for SUIT firmware updates #11818

Conversation

kaspar030 commented Jul 8, 2019 • edited by emmanuelsearch Loading

Contribution description

Testing procedure

Issues/PRs references

kaspar030 commented Jul 8, 2019

kaspar030 commented Jul 8, 2019

kaspar030 commented Jul 8, 2019

kaspar030 commented Jul 10, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kaspar030 commented Jul 10, 2019

kaspar030 commented Jul 12, 2019

kaspar030 commented Jul 15, 2019

kaspar030 commented Jul 18, 2019

jcarrano left a comment

Choose a reason for hiding this comment

fjmolinas commented Oct 9, 2019 • edited Loading

fjmolinas commented Oct 9, 2019 • edited Loading

kaspar030 commented Oct 9, 2019

fjmolinas left a comment

Choose a reason for hiding this comment

aabadie left a comment

Choose a reason for hiding this comment

miri64 commented Oct 9, 2019

kaspar030 commented Oct 9, 2019

fjmolinas commented Oct 10, 2019 • edited Loading

kaspar030 commented Oct 10, 2019

miri64 commented Oct 10, 2019

bergzand commented Oct 10, 2019

emmanuelsearch commented Oct 10, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kaspar030 commented Jul 8, 2019 •

edited by emmanuelsearch

Loading

fjmolinas commented Oct 9, 2019 •

edited

Loading

fjmolinas commented Oct 9, 2019 •

edited

Loading

fjmolinas commented Oct 10, 2019 •

edited

Loading