We need CI to prevent commits break builds easily

[marynate]

Notice that there're more and more build failure recently, I think it's time to integrate CI build into the main repository. ( I've setup some very basic travis-ci script on my fork: https://travis-ci.org/marynate/godot) So we can easily identify if a PR will break builds before merging into main repo.

[godotengine]

absolutely agreed. Right now i'm using a machine at work for builds, but
was considering moving it to a more open place that can also support CI.
Maybe Azure or travis.

The biggest problem is Mac builds. I have no idea how to get hosted mac
service for builds

On Mon, Mar 23, 2015 at 1:31 AM, marynate notifications@github.com wrote:

Notice that there're more and more build failure recently, I think it's
time to integrate CI build into the main repository. ( I've setup some very
basic travis-ci script on my fork: https://travis-ci.org/marynate/godot)
So we can easily identify if a PR will break builds before merging into
main repo.

—
Reply to this email directly or view it on GitHub
#1553.

OkamStudio

[hurikhan]

I have done some tests with http://buildbot.net/. It is master/slave orientated. The master is a centralized server in the internet. The slaves are compiling the new commits on the native OS triggered by the master.

The test was very promising. I only wanted to secure/harden my server a little bit more, before i show it up public as a demo.

This is the way blender is doing it. This is the web-interface of the blender master --> https://builder.blender.org/waterfall

[reduz]

I will try to see if I can work with punto on this

On Mon, Mar 23, 2015 at 1:59 AM, MSC notifications@github.com wrote:

I have done some tests with http://buildbot.net/. It is master/slave
orientated. The master is a centralized server in the internet. The slaves
are compiling the new commits on the native OS triggered by the master.

The test was very promising. I only wanted to secure/harden my server a
little bit more, before i show it up public as a demo.

This is the way blender is doing it. This is the web-interface of the
blender master --> https://builder.blender.org/waterfall

—
Reply to this email directly or view it on GitHub
#1553 (comment).

[marynate]

@reduz we have already base .travis.yml in master branch ( I only setup x11 build there so far ), you just have to setup a travis-ci account to link to github repository to get it started. We can start adding mac/ios/android/windows ci-build script base on that.

[silverkorn]

@weitjong is doing a wonderful work with Travis-CI for Urho3D.

There's a neat nightly build archive on Sourceforge (http://sourceforge.net/projects/urho3d/files/Urho3D/) and also direct download on their main page for their stable release on each platform (http://urho3d.github.io/).

Maybe you could inspire from that.

[reduz]

that sounds interesting

On Mon, Mar 23, 2015 at 9:28 AM, Danny Boisvert notifications@github.com
wrote:

@weitjong https://github.com/weitjong is doing a wonderful work with
Travis-CI for Urho3D https://github.com/urho3d/Urho3D.

There's a neat nightly build archive on Sourceforge (
http://sourceforge.net/projects/urho3d/files/Urho3D/) and also direct
download on their main page for their stable release on each platform (
http://urho3d.github.io/).

Maybe you could inspire from that.

—
Reply to this email directly or view it on GitHub
#1553 (comment).

[shackra]

what about this http://huonw.github.io/blog/2015/03/rust-infrastructure-can-be-your-infrastructure/ ?

[reduz]

I wonder if travis can be used to generate all the export templates, the build dependencies are quite large

[silverkorn]

Urho3D has made different branches for building each of them by generally changing only (I guess) the .travis.yml file so Travis can run all of them in parallel and avoid the Travis' timeout to build them all in one shot. This also add the opportunity to see which target has building problems.

If you plan following this kind of technique, I would suggest following a similar approach as the well know successful Git branching model and, for a cleaner readability, using the standard Git flow prefix model from Atlassian (feature/..., release/..., hotfix/...) so you could set them to ci/... and maybe even by version like ci/export/android/1.1 and ci/tools/windows/master, so everything could be done asynchronously.

I'm not 100% sure about all this since I never officially used hooks and CI for the moment but I guess you could get in touch with one of their team members such as @cadaver (Project Maintainer) or @weitjong (Majorly the CI Maintainer) for further information.

[reduz]

if anyone wants to help set up travis in a way it can generate builds of ALL the templates let me know

[silverkorn]

I can give it a try with Travis-CI by forking this repository and doing what I suggested earlier since I'll need to apply something similar with a self-hosted GitLab-CI server, so this might be a good experience for a good cause 😄 .
(Btw, I would then suggest to change the branch name 1.0 to release/1.0)

I'll follow all building steps from the Wiki and I might focus on different branches but maybe by using submodule instead of full repository updates for each platform, unless you need to build documentation, stubs, intelligent fixes or anything else that is not directly part of a build.

But at anytime feel free to stop me and suggest/test your own continuous integration implementation since I might not offer anything reliable in the next week(s) or month.

[silverkorn]

There's good progress on this but I cannot see why the android build never sees that prerequisites stuffs are installed and ready to be used...

According to your documentation, ANDROID_HOME & ANDROID_NDK_ROOT must be set which is done and I even added them in the PATH env. variable but the SCons script only detects x11, windows & server as possible builds.

What is actually searched to detect android as possible build?

You can look at the custom scripts on my feature/travis-ci branch, see dependencies.sh for installations with "sudo" and prepare.sh for preparation right before executing the SCons script, such as environment variable setups.

Thanks.

[silverkorn]

Anyway, if someone can check, here's what is working so far:
https://travis-ci.org/silverkorn/godot/builds/68024009:

Here are my current blocking points:

• Linux / x11 with rendering/visual server crash on 32-bit. (The 32-bit server works properly);
• "False-positive" for Android and JavaScript:
  • Can't figure out the problem with Android SDK/NDK to be detected;
  • Still looking with JavaScript's compilation.
• Blackberry is not suitable for the moment because the installation of Momentics IDE cannot be done automatically, it absolutely needs interaction to accept their licence agreement, unless someone find a way.

Finally, I may set them into zip and send them to GitHub's Release portal automatically when a new tag is created using Travis-CI deploy system. I will keep this at the end when all builds will work properly.

[vnen]

Well, there's an automatic build tool now, AFAIK, that's pushing the nightly builds in the website. Can we consider this as solved?

[shackra]

@vnen not until a nice badge telling the status of such builds is set in the "read me" file

[akien-mga]

This issue is not solved no, nightly builds are nice but do not constitute a true CI solution. We need something that checks each PR on all platforms and reports the status, like e.g. BuildBot (see for example the test in PRs of dolphin-emu: dolphin-emu/dolphin#3244).

[silverkorn]

The Travis-CI scripts were intended to be used for both build releases and test every platform for PRs.
The main reason using it was that it is free for open source projects and it also support iOS & MacOSX building (which seemed to be a main concern with @reduz).

Anyone is free to fork my feature/travis-ci branch and fix what might be missing (and remove debug verbose stuff such as ls and echo).

Otherwise, I would also suggest a self-running build server such as pointed out by @akien-mga.

[akien-mga]

The travis-ci script is finally enabled, so we should now have CI for x11, windows, android, osx and iphone (see #3120), so this issue should finally be fixed :)

There is of course still room for improvement, adding more platforms or testing both editor and templates build, etc., but it's start :)