Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Release 2022.10 - RC2 #264

Closed
58 of 86 tasks
leandrolanzieri opened this issue Nov 3, 2022 · 9 comments
Closed
58 of 86 tasks

Release 2022.10 - RC2 #264

leandrolanzieri opened this issue Nov 3, 2022 · 9 comments
Assignees
@maribu @leandrolanzieri
Labels
help wanted

Comments

@leandrolanzieri
Copy link
Contributor

leandrolanzieri commented Nov 3, 2022 •
edited
Loading

This issue lists the status of all tests for the Release Candidate 2 of the 2022.10 release.

Specs tested:
@leandrolanzieri leandrolanzieri mentioned this issue Nov 3, 2022
Closed
88 tasks
@riot-ci
Copy link

riot-ci commented Nov 3, 2022 •
edited
Loading

Test Report

Task Outcome
✔ 01. Task 02 - Unittests on native PASSED
✔ 01. Task 03 - Unittests on native separated [tests-analog_util] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-base64] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-bcd] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-bhp_event] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-bhp_msg] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-bitfield] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-bloom] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-bluetil] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-checksum] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-chunked_ringbuffer] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-clif] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-color] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-core] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-credman] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-div] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-dns_cache] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-fib] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-fib_sr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-fmt] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-frac] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gcoap] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_ipv6] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_ipv6_hdr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_ipv6_nib] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_mac_internal] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_netif_pktq] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_sixlowpan_frag_vrb] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_udp] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-hashes] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ieee802154] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-inet_csum] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ipv4_addr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ipv6_addr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ipv6_hdr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-kernel_defines] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-libc] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-luid] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-matstat] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-mtd] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-nanocoap] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-nanocoap_cache] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-netopt] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-netreg] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-phydat] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-pkt] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-pktbuf] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-pktqueue] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-printf_float] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-priority_pktqueue] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-rtc] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-rtt_rtc] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-saul_reg] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-scanf_float] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-seq] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sht1x] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sixlowpan] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sixlowpan_ctx] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sixlowpan_sfr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sock_util] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-timex] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-tsrb] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-turo] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-uri_parser] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ut_process] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-uuid] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-vfs] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-zptr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ztimer64] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ztimer] PASSED
✔ 01. Task 04 - Unittests on iotlab-m3 PASSED
✔ 03. Task 01 - ICMPv6 multicast echo on native PASSED
✔ 03. Task 02 - ICMPv6 link-local echo on native PASSED
✔ 03. Task 03 - ICMPv6 link-local echo on native (1 hour) PASSED
✔ 03. Task 04 - ICMPv6 stress test on native (1 hour) PASSED
✔ 03. Task 05 - ICMPv6 stress test on native (neighbor cache stress) PASSED
✔ 03. Task 06 - ICMPv6 link-local echo on native (IPv6 fragmentation) PASSED
✔ 04. Task 01 - ICMPv6 link-local echo with iotlab-m3 PASSED
✔ 04. Task 02 - ICMPv6 multicast echo with iotlab-m3/samr21-xpro PASSED
✔ 04. Task 03 - ICMPv6 echo with large payload PASSED
✔ 04. Task 04 - ICMPv6 echo with iotlab-m3/samr21-xpro 15 minutes PASSED
🟡 04. Task 05 (Experimental) - ICMPv6 multicast echo with samr21-xpro/cc2538 [lille] SKIPPED
🟡 04. Task 06 (Experimental)- ICMPv6 link-local echo with samr21-xpro/cc2538 [lille] SKIPPED
🟡 04. Task 07 (Experimental) - ICMPv6 multicast echo with samr21-xpro/zero + xbee SKIPPED
🟡 04. Task 08 (Experimental) - ICMPv6 echo with samr21-xpro/zero + xbee SKIPPED
✔ 04. Task 09 - ICMPv6 stress test on iotlab-m3 PASSED
✔ 04. Task 10 (Experimental) - ICMPv6 echo with large payload (IPv6 fragmentation) PASSED
✔ 04. Task 11 (Experimental) - ICMPv6 stress test on nrf802154 PASSED
✔ 04. Task 12 (Experimental) - ICMPv6 multicast echo with iotlab-m3/nrf802154 PASSED
✔ 04. Task 13 (Experimental) - ICMPv6 link-local echo with iotlab-m3/nrf802154 PASSED
✔ 05. Task 01 - ICMPv6 echo unicast addresess on native (default route) PASSED
✔ 05. Task 02 - ICMPv6 echo unicast addresess on iotlab-m3 (default route) PASSED
✔ 05. Task 03 - ICMPv6 echo unicast addresess on native (specific route) PASSED
✔ 05. Task 04 - ICMPv6 echo unicast addresess on iotlab-m3 (static route) PASSED
✔ 06. Task 01 - UDP on iotlab-m3 PASSED
✔ 06. Task 02 - UDP on iotlab-m3 (UDP port compression) PASSED
✔ 06. Task 03 - UDP on native (non-existent neighbor) PASSED
✔ 06. Task 04 - UDP on iotlab-m3 (non-existent neighbor) PASSED
✔ 06. Task 05 - Empty UDP on native PASSED
✔ 06. Task 06 - Empty UDP on iotlab-m3 PASSED
✔ 07. Task 01 - ICMPv6 echo on iotlab-m3 with three hops (static route) PASSED
✔ 07. Task 02 - UDP on iotlab-m3 with three hops (static route) PASSED
✔ 07. Task 03 - ICMPv6 echo on iotlab-m3 with three hops (RPL route) PASSED
✔ 07. Task 04 - UDP on iotlab-m3 with three hops (RPL route) PASSED
✔ 08. Task 01 - ICMPv6 echo between native and Linux PASSED
✔ 08. Task 08 - UDP between GNRC and lwIP on iotlab-m3 PASSED
✔ 09. Task 01 - CORD Endpoint PASSED
✔ 09. Task 02 - Confirmable retries [False-0] PASSED
✔ 09. Task 02 - Confirmable retries [True-1] PASSED
✔ 09. Task 03 - Block1 PASSED
✔ 09. Task 04 - Block2 PASSED
✔ 09. Task 05 - Observe registration and notification PASSED
✔ 10. Task 01 - Destination unreachable - no route to destination PASSED
✔ 10. Task 02 - Destination unreachable - Beyond scope of source address PASSED
✔ 10. Task 03 - Destination unreachable - address unreachable (target node address) PASSED
✔ 10. Task 04 - Destination unreachable - address unreachable (neighbor cache miss) PASSED
✔ 10. Task 05 - Destination unreachable - port unreachable PASSED
✔ 10. Task 06 - Destination unreachable - port unreachable (large payload) PASSED
✔ 10. Task 07 - Packet too big PASSED
✔ 10. Task 08 - Time exceeded - hop limit exceeded in transit PASSED
✔ 10. Task 09 - Parameter problem - erroneous header field encountered PASSED
✔ 10. Task 10 - IPv6-in-IPv6 encapsulation PASSED
✔ 11. Task 01 - LoRaWAN example PASSED
✔ 11. Task 02 - OTAA join procedure PASSED
✔ 11. Task 03 - ABP join procedure PASSED
✔ 11. Task 04 - LoRaWAN device parameters persistence PASSED
✔ 11. Task 05 - GNRC LoRaWAN (Over The Air Activation) [otaa] PASSED
🟡 11. Task 06 - GNRC LoRaWAN (ABP) [abp] SKIPPED

@miri64
Copy link
Member

miri64 commented Nov 3, 2022 •
edited
Loading

✖️ 01. Task 04 - Unittests on iotlab-m3 FAILED
[…]
✖️ 04. Task 04 - ICMPv6 echo with iotlab-m3/samr21-xpro 15 minutes FAILED

There seems to be an erroring M3 node be used in the experiments: https://gist.github.com/riot-ci/3ca2aa5f766aac7810904dbc29d1319a/e0e2a185ec293dbf731329ade3358c4ac08d45e8.

@MrKevinWeiss
Copy link
Contributor

Task 03 - UDP exchange between iotlab-m3 and Contiki 
udp server start 3000
2022-11-03 14:27:38,053 # udp server start 3000
2022-11-03 14:27:38,055 # Success: started UDP server on port 3000
> udp nd fe80::0212:4b00:0422:9db8 3000 1
2022-11-03 14:27:41,078 # udp send fe80::0212:4b00:0422:9db8 3000 1
2022-11-03 14:27:41,081 # Success: send 1 byte to [fe80::0212:4b00:0422:9db8]:3000
2022-11-03 14:27:41,093 # Packets received: 1
udp send fe80::0212:4b00:0422:9db8 3000 1> 
2022-11-03 14:27:42,183 # udp send fe80::0212:4b00:0422:9db8 3000 1
2022-11-03 14:27:42,185 # Success: send 1 byte to [fe80::0212:4b00:0422:9db8]:3000
2022-11-03 14:27:42,197 # Packets received: 2
udp send fe80::0212:4b00:0422:9db8 3000 64
2022-11-03 14:27:50,822 # udp send fe80::0212:4b00:0422:9db8 3000 64
2022-11-03 14:27:50,826 # Success: send 64 byte to [fe80::0212:4b00:0422:9db8]:3000
2022-11-03 14:27:50,836 # Packets received: 3

@leandrolanzieri
Copy link
Contributor Author

After re-triggering some of the tests it looks like the automated tasks are passing. m3-11.saclay.iot-lab.info node seems to be failing, which caused some tests to fail.

@maribu
Copy link
Member

maribu commented Nov 8, 2022

I'm currently collecting output for compile_and_test_for_board.py here: https://github.com/maribu/riot-test-results/tree/main/2022.10-branch-bf7e4f6ca8096de94c7f2274d4406de7263e9493

There are a few failures, mostly compilation errors presumably due to better diagnostics of my more recent toolchains (e.g. use after free bugs in ccn-lite etc.). I intend to address them and backport them tomorrow.

The test failures are mostly due to missing miniterm.py or riotctrl. I will try to obtain them and re-run the tests.

@maribu
Copy link
Member

maribu commented Nov 8, 2022

OK, miniterm.py is actually provided by py3-pyserial, but the package didn't expose it in a way it was added to $PATH. That was easy to fix, though. I will also package riotctrl now. I bet the results will look much better then already :)

@miri64
Copy link
Member

miri64 commented Nov 8, 2022

From a quick skim over the results, you probably also will need scapy ;-).

@leandrolanzieri
Copy link
Contributor Author

08-interop - Task #11 - UDP exchange between iotlab-m3 and Zephyr: PASSED

Tested against Zephyr v3.2.0

RIOT node

> ifconfig 6 set pan_id 0xabcd
ifconfig 6 set pan_id 0xabcd
success: set network identifier on interface 6 to 0xabcd

> ifconfig 6 set chan 26 
ifconfig 6 set chan 26
success: set channel on interface 6 to 26

> ping fe80::9ccf:400:2e9a:9f3b
ping fe80::9ccf:400:2e9a:9f3b
12 bytes from fe80::9ccf:400:2e9a:9f3b%6: icmp_seq=0 ttl=64 rssi=-66 dBm time=16.642 ms
12 bytes from fe80::9ccf:400:2e9a:9f3b%6: icmp_seq=1 ttl=64 rssi=-66 dBm time=6.236 ms
12 bytes from fe80::9ccf:400:2e9a:9f3b%6: icmp_seq=2 ttl=64 rssi=-66 dBm time=8.469 ms

--- fe80::9ccf:400:2e9a:9f3b PING statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 6.236/10.449/16.642 ms

> udp server start 4242
udp server start 4242
Success: started UDP server on port 4242

> udp send fe80::9ccf:400:2e9a:9f3b 4242 "riot"
udp send fe80::9ccf:400:2e9a:9f3b 4242 "riot"
Success: sent 4 byte(s) to [fe80::9ccf:400:2e9a:9f3b]:4242
> PKTDUMP: data received:
~~ SNIP  0 - size:   4 byte, type: NETTYPE_UNDEF (0)
00000000  72  69  6F  74
~~ SNIP  1 - size:   8 byte, type: NETTYPE_UDP (4)
   src-port:  4242  dst-port:  4242
   length: 12  cksum: 0x0549
~~ SNIP  2 - size:  40 byte, type: NETTYPE_IPV6 (2)
traffic class: 0x00 (ECN: 0x0, DSCP: 0x00)
flow label: 0x00000
length: 12  next header: 17  hop limit: 64
source address: fe80::9ccf:400:2e9a:9f3b
destination address: fe80::7894:5ff:258d:e7c3
~~ SNIP  3 - size:  24 byte, type: NETTYPE_NETIF (-1)
if_pid: 6  rssi: -65  lqi: 255
flags: 0x0
src_l2addr: 9E:CF:04:00:2E:9A:9F:3B
dst_l2addr: 7A:94:05:FF:25:8D:E7:C3
~~ PKT    -  4 snips, total size:  76 byte

> udp send fe80::9ccf:400:2e9a:9f3b 4242 "riot2022.10"
udp send fe80::9ccf:400:2e9a:9f3b 4242 "riot2022.10"
Success: sent 11 byte(s) to [fe80::9ccf:400:2e9a:9f3b]:4242
> PKTDUMP: data received:
~~ SNIP  0 - size:  11 byte, type: NETTYPE_UNDEF (0)
00000000  72  69  6F  74  32  30  32  32  2E  31  30
~~ SNIP  1 - size:   8 byte, type: NETTYPE_UDP (4)
   src-port:  4242  dst-port:  4242
   length: 19  cksum: 0x42a7
~~ SNIP  2 - size:  40 byte, type: NETTYPE_IPV6 (2)
traffic class: 0x00 (ECN: 0x0, DSCP: 0x00)
flow label: 0x00000
length: 19  next header: 17  hop limit: 64
source address: fe80::9ccf:400:2e9a:9f3b
destination address: fe80::7894:5ff:258d:e7c3
~~ SNIP  3 - size:  24 byte, type: NETTYPE_NETIF (-1)
if_pid: 6  rssi: -66  lqi: 255
flags: 0x0
src_l2addr: 9E:CF:04:00:2E:9A:9F:3B
dst_l2addr: 7A:94:05:FF:25:8D:E7:C3
~~ PKT    -  4 snips, total size:  83 byte

Zephyr node

uart:~$ net ipv6
net ipv6
IPv6 support                              : enabled
IPv6 fragmentation support                : disabled
Multicast Listener Discovery support      : enabled
Neighbor cache support                    : enabled
Neighbor discovery support                : enabled
Duplicate address detection (DAD) support : enabled
Router advertisement RDNSS option support : enabled
6lo header compression support            : enabled
Max number of IPv6 network interfaces in the system          : 1
Max number of unicast IPv6 addresses per network interface   : 3
Max number of multicast IPv6 addresses per network interface : 4
Max number of IPv6 prefixes per network interface            : 2

IPv6 addresses for interface 1 (0x20001c7c) (IEEE 802.15.4)
=========================================================
Type            State           Lifetime (sec)  Address
autoconf        preferred       infinite        fe80::9ccf:400:2e9a:9f3b/128
manual          preferred       infinite        2001:db8::1/128
uart:~$ D: (udp6): UDP (IPv6): Received and replied with 4 bytes
D: (udp6): UDP (IPv6): Received and replied with 11 bytes
I: IPv6 UDP: Received 0 B/sec

@maribu
Copy link
Member

maribu commented Nov 29, 2022

Closing in favor of #266

@maribu maribu closed this as completed Nov 29, 2022
@maribu maribu mentioned this issue Nov 29, 2022
Closed
86 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@maribu maribu

@leandrolanzieri leandrolanzieri

Labels
help wanted
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@miri64 @maribu @leandrolanzieri @riot-ci @MrKevinWeiss