Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reduce travis test time #1765

Closed
JJ opened this issue Feb 10, 2018 · 33 comments
Closed

Reduce travis test time #1765

JJ opened this issue Feb 10, 2018 · 33 comments
Assignees
@JJ
Labels
big Issue consisting of many subissues wishlist "nice to have" issues; might require a lot of work or a big change or be low priority

Comments

@JJ
Copy link
Contributor

JJ commented Feb 10, 2018

Since it's compiling rakudo, it takes on average half an hour. I would suggest changing it to a Docker container, or just somehow use a precompiled version of Rakudo to test.
@JJ JJ added the build label Feb 10, 2018
@AlexDaniel
Copy link
Member

AlexDaniel commented Feb 10, 2018 •
edited

Hey, what architecture is it running on? If it's linux x86_64/amd64, can't you just fetch http://whateverable.6lang.org/HEAD and use it?

To extract that archive you'd need zstd (it should be available in all stable releases of everything). It is done like this:

zstd -dqc -- "$PATH-TO-ARCHIVE" | tar x --absolute-names

If it's not possible to install zstd for some reason, I can make HEAD available as a .tar.gz or something more conventional.

Ah, as you may know, rakudo is not relocatable, that's why it has to be extracted with absolute names. The build will then be located in /tmp/whateverable/rakudo-moar/SOME-SHA-HERE/.

@JJ
Copy link
Contributor Author

JJ commented Feb 10, 2018

Might as well give it a try.

@JJ
Copy link
Contributor Author

JJ commented Feb 10, 2018

I can't seem to find zstd in the repos for the Travis build environment. https://travis-ci.org/JJ/doc
I think it's Ubuntu 14.04...

@AlexDaniel
Copy link
Member

Oh crap, ubuntu 14.04?? Is it possible to change that somehow?

@AlexDaniel
Copy link
Member

If not, I'll just make it available as .tar.gz, that's possible.

@JJ
Copy link
Contributor Author

JJ commented Feb 10, 2018 via email

@AlexDaniel
Copy link
Member

Hey @JJ, what about this? https://whateverable.6lang.org/HEAD.tar.gz
Can be extracted like so:

tar -x --absolute-names -f "$PATH-TO-ARCHIVE"

AlexDaniel referenced this issue in JJ/doc Feb 11, 2018
@JJ
Testing with whateverable
97daef5
@AlexDaniel
Copy link
Member

FWIW for anyone else reading, some discussion: JJ@97daef5#diff-354f30a63fb0907d4ad57269548329e3L43

AlexDaniel added a commit to Raku/whateverable that referenced this issue Feb 11, 2018
@AlexDaniel
Make the HEAD build available as .tar.gz
c527825 
In case somebody wants to use it on an ancient system where zstd is
not easily available. Maybe Raku/doc#1765 ?

We'll see how this goes and if it's useful I can improve the code
later.
@JJ
Copy link
Contributor Author

JJ commented Feb 11, 2018 via email

@JJ JJ closed this as completed in db93ff0 Feb 11, 2018
@JJ
Copy link
Contributor Author

JJ commented Feb 11, 2018

screenshot from 2018-02-11 11 41 33
Cut in half, the 15 minutes or so that it took initially to build perl6.
It could probably be cut even more by using a Docker container with all modules preinstalled, but that would mean around 1 minute or even less...

@JJ
Copy link
Contributor Author

JJ commented Feb 12, 2018

I'm reopening this because I've been experimenting with Docker, creating a perl6/doc specific container, and managed to reduce 4 minutes more from testing time. The downside is that it's a external Dockerfile, and it must be updated every month for the latest version of Perl6, as well as add other external dependencies if they change. I can maintain that or even move it to the perl6 set of repos. So I would like to hear your opinion about this before merging.

@JJ JJ reopened this Feb 12, 2018
@AlexDaniel
Copy link
Member

Personally I'd prefer more things to act as a canary if something goes wrong on HEAD, especially our own repos. But those who actually work on the doc repo should decide if 4 minutes is worth it :)

@coke
Copy link
Collaborator

coke commented Feb 12, 2018

IMO, I would rather the doc site was using a known good version rather than HEAD.

@AlexDaniel
Copy link
Member

By the way, why not have both?

@JJ
Copy link
Contributor Author

JJ commented Feb 12, 2018

@coke in that case, I would go for the Docker option. It's using a known version, and it's updated a few days after the monthly release comes out. It can be changed to Rakudo Star if needed. @AlexDaniel in that case, I would try and add another CI service, because they can't be done in parallel, far as I can tell. I could try shippable or Circle CI.
It would be interesting also to have our own CI based on Bazel, for instance, but that needs to be self-hosted, there's no free service.

@JJ JJ added the wishlist "nice to have" issues; might require a lot of work or a big change or be low priority label Feb 12, 2018
@JJ JJ self-assigned this Feb 12, 2018
@JJ
Copy link
Contributor Author

JJ commented Feb 13, 2018

I'm taking a look at Shippable, but it needs some configuration.
What do you think about setting up the Docker option for some time to see how it goes? We can always go back to this one if needed.

@timo
Copy link
Contributor

timo commented Feb 13, 2018

we're running multiple versions in parallel on many of our projects; moarvm is probably the most complicated, because we also run a coverage analysis alongside regular builds: https://github.com/MoarVM/MoarVM/blob/master/.travis.yml - not a good template for the docs, but it shows what you can do.

@JJ
Copy link
Contributor Author

JJ commented Feb 13, 2018 via email

@timo
Copy link
Contributor

timo commented Feb 13, 2018

no, I mean running one job with docker to get the latest release and one job with the HEAD build from whateverable. travis would usually do those two jobs at the same time.

@JJ
Copy link
Contributor Author

JJ commented Feb 13, 2018 via email

@JJ
Copy link
Contributor Author

JJ commented Feb 13, 2018 via email

@timo
Copy link
Contributor

timo commented Feb 13, 2018

oh, no, that's the "matrix" thing; for example in moarvm you give it a list of compilers (gcc and clang) and a list of operating systems (linux and osx) and whether to use the jit or not, and it'll run the cross-product of those tasks.

check out this page: https://travis-ci.org/MoarVM/MoarVM/builds/340577255

@JJ
Copy link
Contributor Author

JJ commented Feb 13, 2018 via email

@nxadm
Copy link
Contributor

nxadm commented Feb 13, 2018 •
edited

Is the idea to test against 1 rakudo version or against them all? I create Ubuntu 16.04 pkgs specially for building Rakudo for different Linux distributions (using Docker). You could use the latest released deb?

wget https://github.com/nxadm/rakudo-pkg/releases/download/v2018.01.1/rakudo-pkg-Ubuntu14.04_2018.01-01_amd64.deb && dpkg -i *.deb

"Buildtime"? A few seconds to download the deb and install it.

@AlexDaniel
Copy link
Member

"Buildtime"? A few seconds to download the deb and install it.

Yes but then a few minutes for zef to install all of the deps. This is the main issue here, otherwise whateverable builds would be enough.

@JJ
Copy link
Contributor Author

JJ commented Feb 13, 2018 via email

@JJ
Copy link
Contributor Author

JJ commented Feb 13, 2018 via email

@nxadm
Copy link
Contributor

nxadm commented Feb 13, 2018

I see. I don't mind creating a specialized travis deb with all the dependencies built in, if that helps. Whatever works for you, JJ & co.

@JJ
Copy link
Contributor Author

JJ commented Feb 13, 2018 via email

@AlexDaniel
Copy link
Member

AlexDaniel commented Feb 13, 2018 •
edited

FWIW whateverable allows you to download any build for any commit, for example:

(over 14000 builds available)

The only problem is that these are compressed with zstd, and I had to add some logic to recompress HEAD into .tar.gz. But if something else is needed, please let me know.

@JJ
Copy link
Contributor Author

JJ commented Feb 13, 2018 via email

@AlexDaniel
Copy link
Member

Are we currently parallelizing the build process? It does not feel like we do. There's also this ticket: #664

@JJ
Copy link
Contributor Author

JJ commented Mar 4, 2018 via email

@JJ JJ mentioned this issue Mar 15, 2018
Closed
JJ added a commit to JJ/doc that referenced this issue Apr 15, 2018
@JJ
Testing with docker
478e050 
Refs Raku#1765

Also remove allowed failures in OSX, as long as I was working on the
travis file. Refs Raku#1828
JJ added a commit that referenced this issue Apr 15, 2018
@JJ
Dumb typo refx #1765
23ac8a2
JJ added a commit that referenced this issue Apr 15, 2018
@JJ
Eliminating osx
e893614 
But leaving #1828 closed. The way it is now, it's going to be
impossible to have it built in OSX. Not only are we using Linux
binaries, but also packing them in a way that's not available in OSX.
We'll have to add it to the matrix with specific instructions for
installing stuff, but that's rather a matter for #1765
@JJ JJ added JJ TPF Grant big Issue consisting of many subissues labels Apr 15, 2018
@JJ JJ closed this as completed in 63b9d9c Apr 15, 2018
@nxadm nxadm mentioned this issue Apr 20, 2018
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@JJ JJ

Labels
big Issue consisting of many subissues wishlist "nice to have" issues; might require a lot of work or a big change or be low priority
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@JJ @timo @coke @nxadm @AlexDaniel