Implement automated delivery pipeline (CI / Continuous Integration)

[Sing-Li]

Create a delivery pipeline that will, upon the creation of a pull request:

• run Rocket.Chat tests (completed - pending Velocity bug-fix/overhaul)
• build Rocket.Chat

And upon merge down of a pull request:

• run Rocket.Chat tests (completed - pending Velocity bug-fix/overhaul)
• add buildinfo data
• build Rocket.Chat
• configure and build specialized web executable for DEMO server

create the following artifacts

• create distribution-ready web executable in .tgz format
• create specialized web executable for DEMO server in .tgz format
• create deployable Docker image at rocketchat/rocket.chat - with Dockerfile similar to our official images - and no longer requires Meteor build (greatly reduces support cases)

-create unsigned Android executable (deemed unnecessary in meeting)

-optionally - (may not be possible) create iOS executable (deemed unnecessary in meeting)

perform the following tasks only upon successful build of the associated artifacts

• sign the distribution-ready .tgz and place it at a HTTPS backed URL for download (Github raw, CDN, or our own server)

- [ ] place the Android and iOS artifacts in dev archive

trigger download of specialized DEMO server executable and re-deployment of DEMO server

Optionally, the following secondary artifacts are built
- [ ] publish an npm installable module for Rocket.Chat - we should study Ghost as they are the first large-footprint 3-tier node app to use npm

• create a Sandstorm SPK --- thanks @jparyani !!

Also, create webhooks to notify bots of:

- [ ] successful/failed build (and eventually test) (move to Chatops)
- [ ] successful/failed delivery of each artifact (move to Chatops)

Other work items:

• enhance the bot(s) to report CI events
• create viewers IN EACH ARTIFACT (android, iOS, web, Electron, etc) for the buildinfo resource

- [ ] create a "quickie" curl | sh deployment script for systems with node already running

• rewrite documentation to reflect the new, streamlined, supported deployment scenarios

About BUILDINFO:

• buildinfo is a resource, created by the build pipeline, that will be bundled with all artifacts
• buildinfo contains version <major>.<minor> as well as last git commit information
• buildinfo should contain the build-time content of .meteor/version
• buildinfo should be parsed and easily viewable ON ALL artifacts (we need a way to seed it into client-only artifacts)
• buildinfo is vital for support as we move forward

ONLY DURING BUILD

• we WILL support ALL servers/clients with full <major>.<minor> plus last git-commit info
• we WILL NOT support any custom developer modified builds
• we WILL revamp ALL forms of deployments to NOT require 'Meteor build' (which is both slow and CPU + memory intensive), but use only direct download of our build artifacts; 'Meteor build' should be reserved for developers only

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[Sing-Li]

All comments and suggestions welcomed before beginning this major undertaking that will have impact on our daily workflow.

[geekgonecrazy]

Mobile Builds

I think Android / iOS builds need to be a separate process. Maybe this process kick starts those. But this process should not be held up by those. As compiling for meteor already takes long enough, but compiling for Android takes longer. Not to mention the time to install the SDK every time.

I have a process that does this. It takes too long.

After Successful build

I think the deployment process should be manual. It can build specialized for the Demo / rocket.chat community. But shouldn't auto deploy.

Build Info

So this piece is defaulted in the repo as <major>.<minor>.DEV then on build its set to build number?

[Sing-Li]

So this piece is defaulted in the repo as <major>.<minor>.DEV then on build its set to build number?

Yes.

[geekgonecrazy]

Cool! I'm 1,000% for it. I think this will help us a lot!

[Sing-Li]

Awesome!

To address the Android build time issue. I created and tested a heavily customized Android SDK docker image and it now can compile an Android APK from Meteor source in about 1 minute. It can be hosted on a server/VPS and be triggered by a webhook from CI - so the Cordova build will not impede the 'normal build' workflow. Thanks for bringing up the issue.

[Sing-Li]

We want to include the building of a Sandstorm SPK as one of the secondary build artifacts.

@paulproteus, @kentonv , @jparyani , @dwrensha

Any idea of how to either:

• build it using the limited travis-ci VM resources available for open source pipelines

OR

• create a docker image for building the Sandstorm SPK

Thanks.

[kentonv]

Hmm, unfortunately I think it's going to be tough to run vagrant-spk inside of another VM. If you switched to plain spk, then building on Travis should be straightforward, but I don't think I'd recommend that route as plain spk requires more manual work and our tools and documentation mostly focus on vagrant-spk.

[Sing-Li]

Core features of the CI/CD pipeline is now in place and working.

Will be creating new tickets for all remaining add-on features independently -- enabling help from extended team and community. And this case will be closed in a few days.

[paulproteus]

Cool! I know @jacksingleton was also interested in SPK creation via CI; further discussion here: https://groups.google.com/forum/#!topic/sandstorm-dev/dE42AeWcsLs

[Sing-Li]

Thanks goes to @kentonv 's Sandstorm team and @jparyani for making Sandstorm SPK a first-class citizen with every Rocket.Chat build!

[engelgabriel]

Rodrigo just removed the dependency on the custom http header.. this should make it easier of the mobile and desktop apps to work with sandstorm.

[engelgabriel]

Now that we have the webbhooks, "enhance the bot(s) to report CI events" should be trivial. We jus need to create a integration point on our DEMO server and add the CURL command on our Travis config.