Introduce continous integration for CKEditor 5 packages

[Reinmar]

Implements: ckeditor/ckeditor5-design#45.

CKEditor 5 needs CI, it's pretty obvious. Especially, taken CKEditor's multi-repo architecture this is unavoidable.

We haven't tracked our initial research, but we decided to go with Travis. Initially, we'll only run tests in Chrome on CI. In the future, we can think about adding more browsers, using e.g. BrowserStack.

[Reinmar]

We've got first CIs up and running: https://travis-ci.org/ckeditor :). We should be able to add CIs for all remaining CKEditor 5 packages today.

[Reinmar]

Possible improvements:

• run tests with source maps – it's currently very hard to tell where a test fails based on the stack trace that you get on CI.
• turn on the silent mode – https://github.com/ckeditor/ckeditor5-dev-compiler/issues/57 and https://github.com/ckeditor/ckeditor5-dev-utils/issues/25.
• run project integration tests from https://github.com/ckeditor/ckeditor5 – each commit to this repo (which in 90% cases represents a preset change) and (optionally, if possible) every merge to master on dependent packages should trigger integration tests in this repo.

[Reinmar]

In the future, we can think about adding more browsers, using e.g. BrowserStack.

There's a risk that once we implement more browser related quirks we'll also need to work on combining code coverage results. Currently, we're happy with our 100% and we wouldn't like this to be gone.

[Reinmar]

One more thing is missing now – linter check.

[Reinmar]

And another thing – the build badge should in my opinion show the master branch status. Otherwise, every failed PR will affect the build status visible in the README.

So e.g. in the list package we should be using: https://travis-ci.org/ckeditor/ckeditor5-list.svg?branch=master

[Reinmar]

And to keep in mind – pushing code coverage reports to CodeClimate for non ckeditor5-dev-* packages doesn't work for some reason.

[Reinmar]

Another thing – in each PR two things are changed:

Sometimes, I've got one of these checks failed and I can see that a pretty odd commit is then checked.

First of all – do I understand correctly that Travis checks if the code in the actual commit is ok (first check) and that merging this PR won't break the master (second check)?

Second – why did it fail in here: ckeditor/ckeditor5-utils#98? The failed tests are weird cause it seems that the tests were taken from the master and the source from the branch... or something. Needs to be verified.

[pomek]

And another thing – the build badge should in my opinion show the master branch status. Otherwise, every failed PR will affect the build status visible in the README.

Fixed. In each package we will show build status from master.

[pomek]

And to keep in mind – pushing code coverage reports to CodeClimate for non ckeditor5-dev-* packages doesn't work for some reason.

Reported - https://github.com/ckeditor/ckeditor5-dev-tests/issues/16

[pomek]

run tests with source maps – it's currently very hard to tell where a test fails based on the stack trace that you get on CI.

Worth checking - litixsoft/karma-mocha-reporter#81

[Reinmar]

Fixed. In each package we will show build status from master.

Not yet. You forgot about the dev-* packages.

[pomek]

Not yet. You forgot about the dev-* packages.

Done.

[Reinmar]

Another thing – update the gulp create-package task.

[pomek]

https://github.com/ckeditor/ckeditor5-dev-env/issues/28 :)

[Reinmar]

Let's sum up what remained to be done:

• CI for ckeditor5 package which will allow testing setup of branches.
• Move manual tests out of Bender.
• Switching to ESLint (Switch to ESLint #366).
• Configuring CodeClimate or removing its badge from the READMEs.
• Linter check on CI (Introduce linter check on CI #365).
• gulp create-package (ckeditor/ckeditor5-dev-env#28).
• Check tests in Safari and Firefox 51+ (https://developer.mozilla.org/en-US/Firefox/Releases/51 – cause it brings finally the full support for for-of loops).

[pomek]

CI for ckeditor5 package which will allow testing setup of branches.

#356

[pomek]

Move manual tests out of Bender.

https://github.com/ckeditor/ckeditor5-dev-tests/issues/27

[Reinmar]

Check tests in Safari and Firefox 51+ (https://developer.mozilla.org/en-US/Firefox/Releases/51 – cause it brings finally the full support for for-of loops).

I've run tests in Safari and Firefox 51 and 2-4 fail on each of these browsers, so we're doing great :).

[Reinmar]

I removed the code climate badge in all repos.

[Reinmar]

I consider this finished. I reported missing followups.