Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: decrease CI build times - low hanging fruits #151

Merged
merged 32 commits into from
Nov 19, 2021
Merged

Conversation

o0Ignition0o
Copy link
Contributor

@o0Ignition0o o0Ignition0o commented Nov 16, 2021

feat: decrease CI build times.

This commit lowers the CI build times by up to 50%, by doing the following:

  • cargo xtask: prebuild the first test setup while waiting for the subservices to be up
  • .circleci/config: refactor the job steps into commands so they can be reused
  • run build / test / lint jobs in parallel if applicable ( the lint job is currently the fastest, and it will have cargo deny steps as well in the future)
  • cache: pull and push specific cache layers, depending on the jobs.

There's still quite a couple more things we can do, to decrease the CI build times further, which are being discussed in #157

@o0Ignition0o o0Ignition0o changed the base branch from main to merge-grpc November 16, 2021 18:18
@o0Ignition0o
Copy link
Contributor Author

Ok so it turns out i circleci doesn't allow us to curl docker-compose containers. it means we cannot remove nodejs from the ci env. the good news though is that it's for CI only

@o0Ignition0o o0Ignition0o changed the title WIP feat: remove dependencies to node WIP feat: decrease CI build times - low hanging fruits Nov 17, 2021
@Geal Geal mentioned this pull request Nov 17, 2021
This commit does a couple of things that lower CI build times:

  - Separate and run lints / builds / tests in parallel
  - Cache node_modules (in tests), and cargo build output
  - Run cargo test --no-run while we're waiting for the nodejs subservices to be up and running

The overall goal is to let the CI fail as early as possible (lint -> build -> tests).
@o0Ignition0o o0Ignition0o changed the base branch from merge-grpc to main November 17, 2021 18:15
@o0Ignition0o o0Ignition0o marked this pull request as ready for review November 18, 2021 09:43
@o0Ignition0o o0Ignition0o requested review from cecton, BrynCooke, abernix and Geal and removed request for cecton November 18, 2021 09:43
docker-compose.yml Outdated Show resolved Hide resolved
xtask/src/commands/test.rs Show resolved Hide resolved
xtask/src/commands/test.rs Outdated Show resolved Hide resolved
xtask/src/commands/test.rs Outdated Show resolved Hide resolved
xtask/src/commands/test.rs Outdated Show resolved Hide resolved
xtask/src/commands/test.rs Outdated Show resolved Hide resolved
xtask/src/commands/test.rs Outdated Show resolved Hide resolved
xtask/src/commands/test.rs Outdated Show resolved Hide resolved
@o0Ignition0o o0Ignition0o changed the title WIP feat: decrease CI build times - low hanging fruits feat: decrease CI build times - low hanging fruits Nov 18, 2021
xtask/src/commands/test.rs Show resolved Hide resolved
xtask/src/commands/test.rs Show resolved Hide resolved
@@ -3,9 +3,9 @@ use structopt::StructOpt;
use xtask::*;

#[derive(Debug, StructOpt)]
pub struct Check {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love that one

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean "Compliance" sounds good

xtask/src/commands/test.rs Outdated Show resolved Hide resolved
xtask/src/commands/test.rs Outdated Show resolved Hide resolved
xtask/src/commands/test.rs Show resolved Hide resolved
xtask/src/commands/test.rs Outdated Show resolved Hide resolved
xtask/src/commands/test.rs Outdated Show resolved Hide resolved
xtask/src/commands/test.rs Outdated Show resolved Hide resolved
o0Ignition0o and others added 3 commits November 19, 2021 12:20
Co-authored-by: Cecile Tonglet <cecile.tonglet@cecton.com>
Co-authored-by: Cecile Tonglet <cecile.tonglet@cecton.com>
@o0Ignition0o o0Ignition0o merged commit 7645857 into main Nov 19, 2021
@o0Ignition0o o0Ignition0o deleted the igni/ci branch November 19, 2021 11:42
@o0Ignition0o o0Ignition0o self-assigned this Mar 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants