Add check to travis to make sure new boards are built, fix #1886 #2003

C47D · 2019-07-19T15:13:35Z

Still a draft and my python isn't perfect so let me know if I'm doing something silly.

TODO:

Get the boards from the json returned by build_board_info (the board names are in the json keys).
Get all the boards based on the lines that contain TRAVIS_BOARDS on the .travis.yml file.
Compare both results.
Profit!

EDIT:
Now i have two lists with the boards names got from board_info and TRAVIS_BOARDS:

============
Info boards
===========


arduino_mkr1300
arduino_mkrzero
arduino_zero
bast_pro_mini_m0
capablerobot_usbhub
catwan_usbstick
circuitplayground_express
circuitplayground_express_4h
circuitplayground_express_crickit
circuitplayground_express_digikey_pycon2019
cp32-m4
datalore_ip_m4
datum_distance
datum_imu
datum_light
datum_weather
electronut_labs_blip
electronut_labs_papyr
escornabot_makech
feather_m0_adalogger
feather_m0_basic
feather_m0_express
feather_m0_express_crickit
feather_m0_rfm69
feather_m0_rfm9x
feather_m0_supersized
feather_m4_express
feather_nrf52840_express
feather_radiofruit_zigbee
gemma_m0
gemma_m0_pycon2018
grandcentral_m4_express
hallowing_m0_express
itsybitsy_m0_express
itsybitsy_m4_express
kicksat-sprite
makerdiary_nrf52840_mdk
makerdiary_nrf52840_mdk_usb_dongle
meowmeow
metro_m0_express
metro_m4_airlift_lite
metro_m4_express
mini_sam_m4
particle_argon
particle_boron
particle_xenon
pca10056
pca10059
pewpew10
pewpew13
pirkey_m0
pybadge
pybadge_airlift
pygamer
pygamer_advance
pyportal
pyruler
robohatmm1
sam32
sparkfun_lumidrive
sparkfun_nrf52840_mini
sparkfun_redboard_turbo
sparkfun_samd21_dev
sparkfun_samd21_mini
trellis_m4_express
trinket_m0
trinket_m0_haxpress
uchip
ugame10


============
Travis boards
===========


arduino_mkr1300
arduino_mkrzero
arduino_zero
bast_pro_mini_m0
capablerobot_usbhub
catwan_usbstick
circuitplayground_express
circuitplayground_express_crickit
datum_distance
datum_imu
datum_light
datum_weather
electronut_labs_blip
electronut_labs_papyr
escornabot_makech
feather_m0_adalogger
feather_m0_basic
feather_m0_express
feather_m0_express_crickit
feather_m0_rfm69
feather_m0_rfm9x
feather_m4_express
feather_nrf52840_express
feather_radiofruit_zigbee
gemma_m0
grandcentral_m4_express
hallowing_m0_express
itsybitsy_m0_express
itsybitsy_m4_express
kicksat-sprite
makerdiary_nrf52840_mdk
makerdiary_nrf52840_mdk_usb_dongle
meowmeow
metro_m0_express
metro_m4_airlift_lite
metro_m4_express
mini_sam_m4
particle_argon
particle_boron
particle_xenon
pca10056
pca10059
pewpew10
pirkey_m0
pybadge
pybadge_airlift
pygamer
pygamer_advance
pyportal
pyruler
robohatmm1
sam32
sparkfun_lumidrive
sparkfun_nrf52840_mini
sparkfun_redboard_turbo
sparkfun_samd21_dev
sparkfun_samd21_mini
trellis_m4_express
trinket_m0
uchip
ugame10

…afruit#1886

…ds based on board_info and TRAVIS_BOARDS

…in travis

C47D · 2019-07-19T18:02:50Z

Output of the script:

Boards missing in TRAVIS_BOARDS:
datalore_ip_m4
pewpew13
trinket_m0_haxpress
cp32-m4
gemma_m0_pycon2018
circuitplayground_express_digikey_pycon2019
circuitplayground_express_4h
feather_m0_supersized

sommersoft

Looks good! In addition to the change marked, could you go ahead and add the command to run the script to .travis.yml so we can see how Travis treats it?

tools/travis_new_boards_check.py

…oards_check.py script

C47D · 2019-07-19T22:02:19Z

Looks good! In addition to the change marked, could you go ahead and add the command to run the script to .travis.yml so we can see how Travis treats it?

Done, let me know if i did something wrong :)

sommersoft

I waited until the build finished just to make sure. It works! Well done! 👏

.travis.yml

sommersoft

I'd say the placement is good in the Travis. That's my opinion though, and easily overruled. 😄

Found one last thing, and then I'll approve. I won't merge though, as I'd like @tannewt to weigh in on it, and I have an open question:

Are there any boards that actually should be ignored? Like say trinket_m0_haxpress?

tools/travis_new_boards_check.py

C47D · 2019-07-20T17:42:41Z

About the boards to ignore, i think we can make a list of boards to ignore and remove them from the missing_boards list after missing_boards = list(set(info_boards) - set(travis_boards)).

I will be away from the keyboard for a couple of days, can you do the lasts fixes?

sommersoft · 2019-07-20T17:49:27Z

I will be away from the keyboard for a couple of days, can you do the lasts fixes?

I should be able to handle it. Just a matter of git cooperating. 😄

tannewt

Thank you for doing this!

I think we should either add the boards to travis if we want to keep them on the downloads page or delete the board def if they aren't worth it. I'd choose to just add them for now. We'll need to speed up the build regardless of whether we build these four extra boards.

Also, did someone manually stop the last test job? I'd hope that the rest of the build continues if this test fails. It's important to find as many failures in a single build since they take so long.

tools/travis_new_boards_check.py

C47D · 2019-07-23T04:08:52Z

Hi, there's a sys.exit(1) in case of any missing boards, if it's more convenient we can only print the missing board and let the tests continue. I would like to finish this tomorrow morning if @sommersoft agree.

sommersoft · 2019-07-23T04:38:14Z

Also, did someone manually stop the last test job? I'd hope that the rest of the build continues if this test fails. It's important to find as many failures in a single build since they take so long.

That's on me. I recommended that this be placed in the before_script section. Ironically, for the same reason that builds take so long. Ostensibly, after this is initially merged, the only time a board will be missing from TRAVIS_BOARDS is when one is added. I felt its better to fail fast on that, instead of waiting for the build to finish only to find that the new board didn't build.

tannewt · 2019-07-25T19:11:53Z

@sommersoft failing fast is interesting actually! That makes sense since it'll have to be run anyway.

@C47D We definitely do want to fail. It's just a question of when. I'd suggest adding the sys.exit back and adding the missing boards to .travis.yml so the build passes.

Thank you both!

C47D · 2019-07-25T21:37:40Z

@tannewt I've fixed both requests, I added all the missing TRAVIS_BOARDS to travis.yml

C47D · 2019-07-25T22:40:26Z

I think the only board that isn't properly building is trinket_m0_haxpress:

Build trinket_m0_haxpress for zh_Latn_pinyin took 17.81s and failed
make: Entering directory '/home/travis/build/adafruit/circuitpython/ports/atmel-samd'
Use make V=1, make V=2 or set BUILD_VERBOSE similarly in your environment to increase build verbosity.
-184 bytes free in flash out of 253440 bytes ( 247.5 kb ).
24664 bytes free in ram for stack out of 32768 bytes ( 32.0 kb ).
Too little flash!!!
Makefile:313: recipe for target 'build-trinket_m0_haxpress/firmware.elf' failed
make: *** [build-trinket_m0_haxpress/firmware.elf] Error 255
make: *** Deleting file 'build-trinket_m0_haxpress/firmware.elf'
make: Leaving directory '/home/travis/build/adafruit/circuitpython/ports/atmel-samd'

Adding another means a build would be 6 concurrent jobs and make two not fit side by side in the 10 total we have.

tannewt

I tweaked a couple small things to get it to build and then this should be ready. Thank you so much!

Make it match feather_m0_express

C47D · 2019-07-27T15:44:55Z

@sommersoft @tannewt np, always happy to help and learn new things :)

sommersoft

This will definitely be a useful check! Thanks again @C47D.

C47D added 4 commits July 19, 2019 10:01

[Draft] Add check to travis to make sure new boards are built, fix ad…

a1db6c4

…afruit#1886

[travis new boards check] We now have two lists that contain the boar…

b279d6b

…ds based on board_info and TRAVIS_BOARDS

[travis new boards check] sort both lists of boards

ae41bb3

[travis new boards check] Exit with failure if a board in info isn't …

fc15941

…in travis

C47D requested review from sommersoft and tannewt July 19, 2019 18:03

sommersoft suggested changes Jul 19, 2019

View reviewed changes

tools/travis_new_boards_check.py Outdated Show resolved Hide resolved

C47D added 2 commits July 19, 2019 16:54

[travis check new boards] Reduce code logic to exit with failure

22c265b

[travis check new boards] Add command so Travis runs the travis_new_b…

00d91be

…oards_check.py script

C47D requested a review from sommersoft July 19, 2019 22:02

sommersoft suggested changes Jul 19, 2019

View reviewed changes

.travis.yml Outdated Show resolved Hide resolved

C47D added 2 commits July 19, 2019 18:19

[travis check new boards] Move script into the before_script section

06cb075

Fix missing module sh and remove unused module re

d1fecf5

sommersoft suggested changes Jul 20, 2019

View reviewed changes

tools/travis_new_boards_check.py Outdated Show resolved Hide resolved

exclude aliased boards from 'get_board_mapping()'

b630e56

sommersoft previously approved these changes Jul 21, 2019

View reviewed changes

tannewt requested changes Jul 23, 2019

View reviewed changes

tools/travis_new_boards_check.py Outdated Show resolved Hide resolved

tools/travis_new_boards_check.py Outdated Show resolved Hide resolved

C47D added 2 commits July 23, 2019 13:02

[travis check new boards] Do not end test if there are missing boards

c316231

[travis check new boards] missing_boards as set

8864cef

C47D dismissed sommersoft’s stale review via 8864cef July 23, 2019 18:03

C47D requested review from tannewt and sommersoft July 23, 2019 19:13

[travis check new boards] Fail in case of any missing boards

7e4d7a5

[travis check new boards] Add missing boards to TRAVIS_BOARDS

be1fd5e

tannewt added 2 commits July 25, 2019 22:54

Extend existing board lists

f8d2776

Adding another means a build would be 6 concurrent jobs and make two not fit side by side in the 10 total we have.

Turn off GC opt to save space on Trinket Haxpress

a87094e

tannewt previously approved these changes Jul 26, 2019

View reviewed changes

tannewt changed the title ~~[Draft] Add check to travis to make sure new boards are built, fix #1886~~ Add check to travis to make sure new boards are built, fix #1886 Jul 26, 2019

Reduce feather_m0_supersized build size

c61c874

Make it match feather_m0_express

tannewt dismissed their stale review via c61c874 July 26, 2019 07:31

tannewt approved these changes Jul 26, 2019

View reviewed changes

sommersoft approved these changes Jul 29, 2019

View reviewed changes

sommersoft merged commit 8eab946 into adafruit:master Jul 29, 2019

C47D deleted the travis_check_new_boards branch July 29, 2019 05:12

Add check to travis to make sure new boards are built, fix #1886 #2003

Add check to travis to make sure new boards are built, fix #1886 #2003

Uh oh!

Conversation

C47D commented Jul 19, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

C47D commented Jul 19, 2019

Uh oh!

sommersoft left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

C47D commented Jul 19, 2019

Uh oh!

sommersoft left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sommersoft left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

C47D commented Jul 20, 2019

Uh oh!

sommersoft commented Jul 20, 2019

Uh oh!

tannewt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

C47D commented Jul 23, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sommersoft commented Jul 23, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tannewt commented Jul 25, 2019

Uh oh!

C47D commented Jul 25, 2019

Uh oh!

C47D commented Jul 25, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tannewt left a comment

Choose a reason for hiding this comment

Uh oh!

C47D commented Jul 27, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sommersoft left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

C47D commented Jul 19, 2019 •

edited

Loading

C47D commented Jul 23, 2019 •

edited

Loading

sommersoft commented Jul 23, 2019 •

edited

Loading

C47D commented Jul 25, 2019 •

edited

Loading

C47D commented Jul 27, 2019 •

edited

Loading