Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Release 2023.10 - RC3 #294

Closed
62 of 86 tasks
MrKevinWeiss opened this issue Nov 15, 2023 · 4 comments
Closed
62 of 86 tasks

Release 2023.10 - RC3 #294

MrKevinWeiss opened this issue Nov 15, 2023 · 4 comments
Labels
help wanted

Comments

@MrKevinWeiss
Copy link
Contributor

MrKevinWeiss commented Nov 15, 2023 •
edited
Loading

This issue lists the status of all tests for the Release Candidate 3 of the 2023.10 release.

Specs tested:
@MrKevinWeiss MrKevinWeiss mentioned this issue Nov 15, 2023
Closed
86 tasks
@riot-ci
Copy link

riot-ci commented Nov 15, 2023 •
edited
Loading

Test Report

Task Outcome
✔ 01. Task 02 - Unittests on native PASSED
✔ 01. Task 03 - Unittests on native separated [tests-analog_util] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-base64] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-bcd] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-bhp_event] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-bhp_msg] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-bitfield] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-bloom] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-bluetil] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-checksum] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-chunked_ringbuffer] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-clif] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-coding] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-color] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-core] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-credman] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-div] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-dns_cache] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ecc] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-fib] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-fib_sr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-fmt] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-frac] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gcoap] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_ipv6] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_ipv6_hdr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_ipv6_nib] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_mac_internal] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_netif_pktq] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_sixlowpan_frag_vrb] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-gnrc_udp] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-hashes] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ieee802154] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-inet_csum] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ipv4_addr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ipv6_addr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ipv6_hdr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-kernel_defines] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-libc] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-luid] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-matstat] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-mtd] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-nanocoap] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-nanocoap_cache] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-netopt] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-netreg] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-phydat] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-pkt] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-pktbuf] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-pktqueue] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-printf_float] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-priority_pktqueue] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-rtc] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-rtt_rtc] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-saul_reg] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-scanf_float] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-seq] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sht1x] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sixlowpan] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sixlowpan_ctx] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sixlowpan_sfr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-sock_util] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-timex] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-tsrb] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-turo] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-uri_parser] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ut_process] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-uuid] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-vfs] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-zptr] PASSED
✔ 01. Task 03 - Unittests on native separated [tests-ztimer64] PASSED
✔ 01. Task 04 - Unittests on iotlab-m3 PASSED
✔ 03. Task 01 - ICMPv6 multicast echo on native PASSED
✔ 03. Task 02 - ICMPv6 link-local echo on native PASSED
✔ 03. Task 03 - ICMPv6 link-local echo on native (1 hour) PASSED
✔ 03. Task 04 - ICMPv6 stress test on native (1 hour) PASSED
🟡 03. Task 05 - ICMPv6 stress test on native (neighbor cache stress) SKIPPED
✔ 03. Task 06 - ICMPv6 link-local echo on native (IPv6 fragmentation) PASSED
✔ 04. Task 01 - ICMPv6 link-local echo with iotlab-m3 PASSED
✔ 04. Task 02 - ICMPv6 multicast echo with iotlab-m3/samr21-xpro PASSED
✔ 04. Task 03 - ICMPv6 echo with large payload PASSED
✔ 04. Task 04 - ICMPv6 echo with iotlab-m3/samr21-xpro 15 minutes PASSED
✔ 04. Task 05 (Experimental) - ICMPv6 multicast echo with iotlab-m3/cc2538 [strasbourg] PASSED
✔ 04. Task 06 (Experimental)- ICMPv6 link-local echo with iotlab-m3/cc2538 [strasbourg] PASSED
🟡 04. Task 07 (Experimental) - ICMPv6 multicast echo with samr21-xpro/zero + xbee SKIPPED
🟡 04. Task 08 (Experimental) - ICMPv6 echo with samr21-xpro/zero + xbee SKIPPED
✔ 04. Task 09 - ICMPv6 stress test on iotlab-m3 PASSED
✔ 04. Task 10 (Experimental) - ICMPv6 echo with large payload (IPv6 fragmentation) PASSED
✔ 04. Task 11 (Experimental) - ICMPv6 stress test on nrf802154 PASSED
✔ 04. Task 12 (Experimental) - ICMPv6 multicast echo with iotlab-m3/nrf802154 PASSED
✔ 04. Task 13 (Experimental) - ICMPv6 link-local echo with iotlab-m3/nrf802154 PASSED
✔ 05. Task 01 - ICMPv6 echo unicast addresess on native (default route) PASSED
🟡 05. Task 02 - ICMPv6 echo unicast addresess on iotlab-m3 (default route) SKIPPED
✔ 05. Task 03 - ICMPv6 echo unicast addresess on native (specific route) PASSED
✔ 05. Task 04 - ICMPv6 echo unicast addresess on iotlab-m3 (static route) PASSED
✔ 06. Task 01 - UDP on iotlab-m3 PASSED
✔ 06. Task 02 - UDP on iotlab-m3 (UDP port compression) PASSED
✔ 06. Task 03 - UDP on native (non-existent neighbor) PASSED
✔ 06. Task 04 - UDP on iotlab-m3 (non-existent neighbor) PASSED
✔ 06. Task 05 - Empty UDP on native PASSED
✔ 06. Task 06 - Empty UDP on iotlab-m3 PASSED
✔ 07. Task 01 - ICMPv6 echo on iotlab-m3 with three hops (static route) PASSED
✔ 07. Task 02 - UDP on iotlab-m3 with three hops (static route) PASSED
✔ 07. Task 03 - ICMPv6 echo on iotlab-m3 with three hops (RPL route) PASSED
✔ 07. Task 04 - UDP on iotlab-m3 with three hops (RPL route) PASSED
🟡 07. Task 05 (Experimental) - UDP with large payload on iotlab-m3 with three hops (RPL route) SKIPPED
✔ 08. Task 01 - ICMPv6 echo between native and Linux PASSED
✔ 08. Task 03 - ICMPv6 echo between RIOT and Contiki-NG PASSED
✔ 08. Task 08 - UDP between GNRC and lwIP on iotlab-m3 PASSED
✔ 09. Task 01 - CORD Endpoint PASSED
✔ 09. Task 02 - Confirmable retries [False-0] PASSED
✔ 09. Task 02 - Confirmable retries [True-1] PASSED
✔ 09. Task 03 - Block1 PASSED
✔ 09. Task 04 - Block2 PASSED
✔ 09. Task 05 - Observe registration and notification PASSED
✔ 10. Task 01 - Destination unreachable - no route to destination PASSED
✔ 10. Task 02 - Destination unreachable - Beyond scope of source address PASSED
✔ 10. Task 03 - Destination unreachable - address unreachable (target node address) PASSED
✔ 10. Task 04 - Destination unreachable - address unreachable (neighbor cache miss) PASSED
✔ 10. Task 05 - Destination unreachable - port unreachable PASSED
✔ 10. Task 06 - Destination unreachable - port unreachable (large payload) PASSED
✔ 10. Task 07 - Packet too big PASSED
✔ 10. Task 08 - Time exceeded - hop limit exceeded in transit PASSED
✔ 10. Task 09 - Parameter problem - erroneous header field encountered PASSED
✔ 10. Task 10 - IPv6-in-IPv6 encapsulation PASSED
✔ 11. Task 01 - LoRaWAN example PASSED
✔ 11. Task 02 - OTAA join procedure [0] PASSED
✖ 11. Task 02 - OTAA join procedure [3] FAILED
✔ 11. Task 02 - OTAA join procedure [5] PASSED
✔ 11. Task 03 - ABP join procedure [0] PASSED
✔ 11. Task 03 - ABP join procedure [3] PASSED
✔ 11. Task 03 - ABP join procedure [5] PASSED
✔ 11. Task 04 - LoRaWAN device parameters persistence PASSED
✔ 11. Task 05 - GNRC LoRaWAN (Over The Air Activation) [otaa] PASSED
🟡 11. Task 06 - GNRC LoRaWAN (ABP) [abp] SKIPPED

@MrKevinWeiss
Copy link
Contributor Author

So I cannot get spec03 and task05 passing, it was in RC1 but now, rerunning on RC1 it doesn't. It is also marked as experimental in the pyfile and doesn't throw an error.

A few interop tests and the abp test was not done but the setup was a bit too cumbersome give the amount of work already went into this.

We should further try to automate (and maintain) the tests.

@MrKevinWeiss
Copy link
Contributor Author

MrKevinWeiss commented Dec 4, 2023 •
edited
Loading

Test Report

Task Outcome
✔ 08. Task 11 - UDP exchange between iotlab-m3 and Zephyr PASSED

@MrKevinWeiss
Copy link
Contributor Author

Since it is done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
help wanted
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@riot-ci @MrKevinWeiss