Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Travis CI: Stages, OS update, apt-get relief! #9844

Merged
merged 10 commits into from Apr 12, 2019

Conversation

@cmonr
Copy link
Contributor

commented Feb 26, 2019

Description

A rather large refactor of Travis CI.

Changes:

  • Addition of build stages.
    • Adds some much needed job organization
    • Runs jobs in batches, allowing for better parallelism
    • Allows for better control over job setup (ie: no need to install python modules when only running a shell command)
  • Linux OS update.
    • All jobs are now running Ubuntu Xenial.
  • Creation of bash helper functions.
    • Helper functions are located in tools/test/travis-ci/functions.sh
    • Functions can even be sourced and run locally (with caveats)
    • Cleans up Travis CI config (a bit)
  • Misc changes
    • No more apt-get update! 馃帀
    • Enabled more caching
    • Some packages are now being sourced from S3
    • Easily updatable GCC archive when the time is right
    • Cleaned up some of the larger jobs

PR will be squashed and cleaned up tomorrow, along with the addition of Release Notes.

Pull request type

[ ] Fix
[x] Refactor
[ ] Target update
[ ] Functionality change
[ ] Docs update
[ ] Test update
[ ] Breaking change

Reviewers

Release Notes

Travis CI's testing config has been updated with some readability and performance improvements in mind.

Build Stages

Build stages have been added, which helps group similar jobs according to function.

Build stages also enocourage greater build parallelism by limiting the number of concurrent build jobs that a single PR can have, allowing multiple PRs to be tested in parallel instead of wauting in a build queue.

Finally, batching jobs into build stages allows for better job configuration. Some jobs that shouldn't have taken long were doing unneccesary actions such as downloading packages or installing python modules, which has now been corrected.

Python 3.7 by default

Jobs that require Python to run now use Py37 by default.

Ubutnu Xenial

The Linux VM that runs tests has been updated to use Xenail, Travis CI's latest supported Linux OS.

Minimal apt-get usage

Usage of apt-get has been greatly minimized by moving packages over to S3 and using dpkg to install cached packages when needed.

A major pain point would occur when a job would fail because the VM failed to run apt-get update successfully.

Bash functions

Common functions such as downloading packages or updating a job's status have now been collected into a single bash function. No strage shell invocations need to be done in order to use them which was not the case before.

@cmonr

This comment has been minimized.

Copy link
Contributor Author

commented Feb 26, 2019

@AnotherButler Feel free to take a look at the docs jobs and lmk what you think.

A PR to https://github.com/ARMmbed/mbed-os-5-docs will come tomorrow as well.

@cmonr cmonr changed the title Travis CI: Stages, OS update, apt-get relief Travis CI: Stages, OS update, apt-get relief! Feb 26, 2019

@cmonr

This comment has been minimized.

Copy link
Contributor Author

commented Feb 26, 2019

A note: My local fork didn't like running 'coveralls' with Py37. Time debugging that failure was next to nothing. Can take another look at it tomorrow.

@cmonr cmonr requested a review from SenRamakri Feb 26, 2019

@cmonr cmonr force-pushed the cmonr:travisCI-refactor branch 2 times, most recently Feb 27, 2019

@0xc0170

This comment has been minimized.

Copy link
Member

commented Feb 28, 2019

@cmonr I am moving this to 5.12.1 - this is test config that could land later than 5.12.0 ? As we have high number of PR for 5.12 and t his is not even reviewed.

Or at least landing on master, and each patch release will use old travis config - not ideal.

@cmonr

This comment has been minimized.

Copy link
Contributor Author

commented Feb 28, 2019

is this test config that could land later than 5.12.0?

It can, but was hoping to have it in sooner, since this fixes a fair amount of headaches with apt-get, and better supports testing multiple PRs at once.

@cmonr cmonr removed the risk: A label Feb 28, 2019

@cmonr

This comment has been minimized.

Copy link
Contributor Author

commented Mar 1, 2019

@0xc0170 I just remembered that a large reason I was trying to get this in for 5.12 was that it was possible that the test names were going to be updated.

@0xc0170

This comment has been minimized.

Copy link
Member

commented Mar 1, 2019

Usage of apt-get has been greatly minimized by moving packages over to S3 and using dpkg to install cached packages when needed.

What is the lifetime? Previously we had some issue with limited lifetime, solved now?

@0xc0170
Copy link
Member

left a comment

LGTM otherwise

@@ -0,0 +1,139 @@
#!/bin/bash -euf

This comment has been minimized.

Copy link
@0xc0170

0xc0170 Mar 1, 2019

Member

license header file here

This comment has been minimized.

Copy link
@cmonr

cmonr Mar 27, 2019

Author Contributor

Added

@cmonr cmonr added needs: work and removed needs: review labels Mar 5, 2019

@cmonr

This comment has been minimized.

Copy link
Contributor Author

commented Mar 7, 2019

What is the lifetime? Previously we had some issue with limited lifetime, solved now?

@0xc0170 What do you mean by lifetime?

@cmonr cmonr force-pushed the cmonr:travisCI-refactor branch to 231236c Mar 7, 2019

@0xc0170

This comment has been minimized.

Copy link
Member

commented Mar 8, 2019

@0xc0170 What do you mean by lifetime?

We had previously a problem aws stored only for 3 months or so. if this is not a problem anymore

cmonr added some commits Mar 7, 2019

Re-enabled PR status reporting.
Added external bash file to source functions from
Removed nearly all usage of 'apt-get'.
Primary cause of false Travis CI failures was running 'apt-get update'.
Refactored all instances where this was needed with manually fetching and installing dependencies in S3 instace.

This also includes GCC installation.
Cleaned up LittleFS and Event Queue tests.
Added {} around each variable.
Ported python invocations for simple math opts to coreutil commands
Fixed shellcheck issues found in functions.sh
Added comments to functions

@cmonr cmonr force-pushed the cmonr:travisCI-refactor branch from 1a65895 to 3f76f6e Apr 8, 2019

@0xc0170 0xc0170 added needs: CI and removed needs: work labels Apr 11, 2019

@0xc0170

This comment has been minimized.

Copy link
Member

commented Apr 11, 2019

CI started

@0xc0170 0xc0170 added needs: work and removed needs: CI labels Apr 11, 2019

@mbed-ci

This comment has been minimized.

Copy link

commented Apr 11, 2019

Test run: FAILED

Summary: 1 of 11 test jobs failed
Build number : 1
Build artifacts

Failed test jobs:

  • jenkins-ci/mbed-os-ci_exporter
@0xc0170

This comment has been minimized.

Copy link
Member

commented Apr 11, 2019

Exporters failed with some symlink errors, will restart once travis is fixed

@OPpuolitaival

This comment has been minimized.

Copy link
Contributor

commented Apr 11, 2019

symlink error comes from untar. Sounds like corrupted package. Probably need to run whole pipeline again

@cmonr cmonr force-pushed the cmonr:travisCI-refactor branch from f987632 to c79bb7e Apr 11, 2019

Fixed status reporting.
'tr' command wasn't doing any replacement

@cmonr cmonr force-pushed the cmonr:travisCI-refactor branch from c79bb7e to 2a2ec16 Apr 11, 2019

@cmonr cmonr added needs: CI and removed needs: work labels Apr 11, 2019

@cmonr

This comment has been minimized.

Copy link
Contributor Author

commented Apr 12, 2019

CI started

@mbed-ci

This comment has been minimized.

Copy link

commented Apr 12, 2019

Test run: FAILED

Summary: 1 of 11 test jobs failed
Build number : 2
Build artifacts

Failed test jobs:

  • jenkins-ci/mbed-os-ci_exporter
@0xc0170

This comment has been minimized.

Copy link
Member

commented Apr 12, 2019

Exporters restarted

@0xc0170

This comment has been minimized.

Copy link
Member

commented Apr 12, 2019

Internal CI symlink error again, aborting the current one and should be fixed in the next one

@0xc0170

This comment has been minimized.

Copy link
Member

commented Apr 12, 2019

Green 馃帀 @cmonr in?

@cmonr cmonr merged commit f1c4a02 into ARMmbed:master Apr 12, 2019

26 checks passed

continuous-integration/jenkins/pr-head This commit looks good
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
jenkins-ci/build-ARM Success
Details
jenkins-ci/build-GCC_ARM Success
Details
jenkins-ci/build-IAR Success
Details
jenkins-ci/cloud-client-test Success
Details
jenkins-ci/dynamic-memory-usage RTOS ROM(+88 bytes) RAM(+0 bytes)
Details
jenkins-ci/exporter Success
Details
jenkins-ci/greentea-test Success
Details
jenkins-ci/mbed2-build-ARM Success
Details
jenkins-ci/mbed2-build-GCC_ARM Success
Details
jenkins-ci/mbed2-build-IAR Success
Details
jenkins-ci/unittests Success
Details
travis-ci/astyle Success!
Details
travis-ci/docs Success!
Details
travis-ci/doxy-spellcheck Success!
Details
travis-ci/events Success! Runtime is 8572 cycles. (-2030+0
Details
travis-ci/gitattributestest Success!
Details
travis-ci/include_check Success!
Details
travis-ci/licence_check Success!
Details
travis-ci/littlefs Success! Code size is 8448B. (+0.48%)
Details
travis-ci/psa-autogen Success!
Details
travis-ci/tools-py2.7 Success!
Details
travis-ci/tools-py3.5 Success!
Details
travis-ci/tools-py3.6 Success!
Details
travis-ci/tools-py3.7 Success!
Details

@cmonr cmonr removed the needs: CI label Apr 12, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can鈥檛 perform that action at this time.