Help wanted: Continuous integration

[nathansobo]

It would be great to get a PR helping us setup a CI build on Travis. This would also force us to iron out our build on Linux. It's not as simple as just slapping a travis.yml file into the root folder however...

Running only the tests that matter

We're planning on xray being a mono-repository to avoid the transaction costs and community fragmentation of hosting hundreds of separate, interconnected repos. However, it seems like this could make our builds horrible if we're not careful.

For this reason, even though we're versioning all these components together, we want their inter-dependencies to be clearly specified, and we want the global build script to be smart about knowing when to run which component's test suite.

So for example, if we make a change in xray_core, then we want to run tests on xray_node and xray_electron. A change to xray_electron should run tests only on xray_electron. A change to the documentation should run no tests.

How should we achieve this? We could cache some sort of fingerprint file on a per-module basis with a digest of the module's contents, plus make each module's build smart about checking the fingerprints of dependencies. Or maybe there's a better, standard way of dealing with this in mono-repos?

[rleungx]

Hey, @nathansobo. I am trying to do something about it but I am not quite understand how to test each crate. Improving the README for each crate is great.

[max-sixty]

Testing only the changes would be cool, but I'm don't believe there's a pre-packaged solution.

Is it worthwhile running CI on all the tests for now, and then adding 'only the tests that matter' later? Unless I'm not correctly running the tests, it's not a big test suite yet.

[lucat1]

This could be achieved by analyzing the git diff in PRs and commits, similar to what a common code coverage tool does. If I understand the behavior you're trying to achieve correctly, if you do a change in a package this may trigger test for the package itself but also for other packages that are dependent on it and so could be broken. For this we could either do:

Build a dependency tree by reading the package.jsons and find which packages are dependent on others, or statically type it into a field of the package.json(or somewhere else, doesn't matter). I could work on this as a simple nodejs script if it's okay for you. Just tell which way you think is best.

[nathansobo]

Build a dependency tree by reading the package.jsons and find which packages are dependent on others

This sounds good. We should only have a few interdependent high level components, so the dependencies should be pretty simple to analyze. We're actually planning a pretty big change to the arrangement of the high level components that I'll be writing about today or Monday, but the strategy would probably be the same... some kind of script in the root folder that decides which subcomponent builds to run based on the diff since the last green build.

[nathansobo]

@maxim-lian As for setting up CI on the whole thing in a simple way, it does seem like a reasonable place to start. When the dust settles on these architectural changes we'd be interested in help with an interim solution that just runs all the builds, but that wouldn't close this issue.

[max-sixty]

The very simplest travis branch is here. It's failing because of napi - I'll wait for the architecture changes before fixing (or anyone else who wants to fix now - please go ahead)

[lucat1]

@nathansobo I'm working on a prototype. I'll post a PR as soon as I got something simple working.

[nathansobo]

Okay thanks to both of you! I’m away from my keyboard this weekend but will take a look Monday and be in a position to merge once we make these larger changes.

[viddo]

Another idea/food for thought: have you looked at how other monorepo-projects solves this? E.g. https://github.com/facebook/jest have a monorepo setup, and is consuming Jest to test itself. One of their core maintainers (@rogeliog) also gave a talk on Jest as a Platform where he mentions integrating Python tests. I figure Rust integration could be achieved in a similar way?

[tsenart]

Build systems like Bazel solve this problem of building only what changes. https://github.com/bazelbuild/rules_rust

[Avi-D-coder]

What is the present state of continuous integration? Are there any semi defined tasks that someone could help with?