Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CI #123

Merged
merged 21 commits into from Jul 17, 2021
Merged

Add CI #123

merged 21 commits into from Jul 17, 2021

Conversation

rikhuijzer
Copy link
Contributor

@rikhuijzer rikhuijzer commented Jul 17, 2021

This PR sets up CI for building the tutorials. The suggested CI job consists of the following main steps:

  1. Build the tutorials* in parallel, see the section CI output below for an example of how the CI logs will look. This build takes about 60 to 90 minutes. Hopefully, we can speed this up a bit more in the future which will benefit users and developers. Only when a tutorial run fails, the logs are shown in the CI log. Note that, like with tests, the job fails after running the code and not during.

  2. Write the output to a branch called artifacts.

To get the output to show up on https://turing.ml, we only need to change

Pkg.develop(url="https://github.com/TuringLang/TuringTutorials")

to

Pkg.add(url="https://github.com/TuringLang/TuringTutorials", rev="artifacts")

in turing.ml/make-utils.jl. Note that I haven't removed the artifacts from TuringTutorials#master, so we can always switch this line back if the CI job doesn't work for some reason.

* @cpfiffer told me that tutorial 10 (differential equations) takes ≈10 hours to complete. Therefore, this tutorial has been excluded from CI for now. My hope is that we can speed the tutorial up and add it later.

Demo

A job has finished in my fork of TuringTutorials: https://github.com/rikhuijzer/TuringTutorials/actions/runs/1040205250.

The output of the run can be inspected at https://github.com/rikhuijzer/TuringTutorials/tree/artifacts-ci. To verify that the CI really updated something, compare https://turing.ml/dev/tutorials/00-introduction/ which shows Julia Version 1.5.3 with 00_introduction.md which shows Julia Version 1.6.2.

A note on tutorial 3

In the run, tutorial 3 (neural networks) was failing on:

advi = ADVI(10, 5_000)
q_hat = vi(m, advi, q);

with a BoundsError. This is a known issue: #59. I've removed the part for now.

CI output

00-introduction: Starting build
05-linear-regression: Starting build
00-introduction: Build took 7.3 minutes and allocated 18.98 GiB:
00-introduction: Writing log to /home/runner/work/TuringTutorials/TuringTutorials/tutorials/00-introduction/weave.log
01-gaussian-mixture-model: Starting build
05-linear-regression: Build took 10.11 minutes and allocated 30.9 GiB:
05-linear-regression: Writing log to /home/runner/work/TuringTutorials/TuringTutorials/tutorials/05-linear-regression/weave.log
06-infinite-mixture-model: Starting build
06-infinite-mixture-model: Build took 4.67 minutes and allocated 19.62 GiB:
06-infinite-mixture-model: Writing log to /home/runner/work/TuringTutorials/TuringTutorials/tutorials/06-infinite-mixture-model/weave.log
07-poisson-regression: Starting build
07-poisson-regression: Build took 5.51 minutes and allocated 25.78 GiB:
07-poisson-regression: Writing log to /home/runner/work/TuringTutorials/TuringTutorials/tutorials/07-poisson-regression/weave.log
08-multionomial-regression: Starting build
08-multionomial-regression: Build took 5.62 minutes and allocated 44.44 GiB:
08-multionomial-regression: Writing log to /home/runner/work/TuringTutorials/TuringTutorials/tutorials/08-multionomial-regression/weave.log
09-variational-inference: Starting build
01-gaussian-mixture-model: Build took 22.5 minutes and allocated 211.85 GiB:
01-gaussian-mixture-model: Writing log to /home/runner/work/TuringTutorials/TuringTutorials/tutorials/01-gaussian-mixture-model/weave.log
02-logistic-regression: Starting build
02-logistic-regression: Build took 7.31 minutes and allocated 25.54 GiB:
02-logistic-regression: Writing log to /home/runner/work/TuringTutorials/TuringTutorials/tutorials/02-logistic-regression/weave.log
03-bayesian-neural-network: Starting build
09-variational-inference: Build took 33.35 minutes and allocated 429.86 GiB:
[...]

@torfjelde
Copy link
Member

This is awesome! Thank you so much @rikhuijzer !

@torfjelde
Copy link
Member

Btw, should it be

Pkg.add(url="https://github.com/TuringLang/TuringTutorials", rev="artifacts")

i.e. withouth the hyphen, given that

branch: artifacts

Also, in that case, maybe we should just call it deployed or something?

@torfjelde
Copy link
Member

@yebai Can you allow the PR to run GH actions? I don't have the rights.

@yebai
Copy link
Member

yebai commented Jul 17, 2021

Not sure, I don't see an option to enable GH actions for this PR. Maybe we have to merge the PR first to enable GH actions?

@yebai yebai merged commit 4d45902 into TuringLang:master Jul 17, 2021
@yebai
Copy link
Member

yebai commented Jul 17, 2021

Many thanks, @rikhuijzer - I've merged this PR to enable CIs. Please feel free to create a new PR if there is any additional work.

@rikhuijzer
Copy link
Contributor Author

Btw, should it be

Pkg.add(url="https://github.com/TuringLang/TuringTutorials", rev="artifacts")

i.e. withouth the hyphen, given that

branch: artifacts

Also, in that case, maybe we should just call it deployed or something?

Well spotted. The hyphen shouldn't be there. I've updated the first post.

Many thanks, @rikhuijzer - I've merged this PR to enable CIs. Please feel free to create a new PR if there is any additional work.

Thank you! I'll keep an eye on it and see whether it's all good now

@rikhuijzer
Copy link
Contributor Author

Also, in that case, maybe we should just call it deployed or something?

I agree that artifacts is not perfect. I chose it because it's a commonly use term for these kinds of things in GitHub and GitLab, but I wouldn't mind changing it to deployed, generated or built.

@rikhuijzer
Copy link
Contributor Author

Interestingly, something which worked perfectly fine locally fails after 90 minutes. I'll soon open a PR with a patch.

leachim pushed a commit that referenced this pull request Aug 12, 2021
* Setup basic CI

* Add build_folder and notes on runtime

* Possibly fix error

* It should not pass

* Full run

* Fix 02

* Disable tutorial 10

* Use threads

* Small changes

* Update CI.yml

* Put original back

* Add rm .gitignore

* Put original artifacts back

* Add verify_logs

* Fix typo

* Update verify_logs

* Add CompatHelper

* Small tweaks

* Show stacktrace in logs

* Change overestimating to underestimating

* Fix the failing part of 03
leachim pushed a commit that referenced this pull request Aug 14, 2021
* Setup basic CI

* Add build_folder and notes on runtime

* Possibly fix error

* It should not pass

* Full run

* Fix 02

* Disable tutorial 10

* Use threads

* Small changes

* Update CI.yml

* Put original back

* Add rm .gitignore

* Put original artifacts back

* Add verify_logs

* Fix typo

* Update verify_logs

* Add CompatHelper

* Small tweaks

* Show stacktrace in logs

* Change overestimating to underestimating

* Fix the failing part of 03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants