Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Enforce Complement test Docker build times in CI #14316

Open
MadLittleMods opened this issue Oct 28, 2022 · 1 comment
Open

Enforce Complement test Docker build times in CI #14316

MadLittleMods opened this issue Oct 28, 2022 · 1 comment
Labels
A-Testing Issues related to testing in complement, synapse, etc T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks. Z-Dev-Wishlist Makes developers' lives better, but doesn't have direct user impact

Comments

@MadLittleMods
Copy link
Contributor

MadLittleMods commented Oct 28, 2022

As mentioned in the weekly backend team call and in the internal backend room


Testing with Complement has gotten so slow, #13204

Before people put effort into making it faster again, it would be nice to add some CI to enforce that we don't regress the timing. We've fixed and regressed the timing many times so I think it's worth actually enforcing. And as @clokep describes it, "slow tooling really impacts velocity." :feelsgood:

What we want to test is that a rebuild after a Synapse source change is fast. Not concerned with initial build times here since that's not the primary dev experience when iterating on a feature/bug.

Potential solutions

Add a Complement Docker image rebuild job and fail the CI if it goes over a threshold duration.

It would be nice to do this as part of the existing Complement job since it has to do the initial Docker build there anyway.

@MadLittleMods MadLittleMods added Z-Dev-Wishlist Makes developers' lives better, but doesn't have direct user impact A-Testing Issues related to testing in complement, synapse, etc labels Oct 28, 2022
@DMRobertson DMRobertson added the T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks. label Oct 28, 2022
@kegsay
Copy link
Member

kegsay commented Oct 18, 2023

The easiest thing to do would probably be to patch a print statement somewhere at startup, then run complement.sh and assert we see that print statement. Then tweak the statement and re-run, timing how long that takes. This would ignore initial build times which are amortised costs.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A-Testing Issues related to testing in complement, synapse, etc T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks. Z-Dev-Wishlist Makes developers' lives better, but doesn't have direct user impact
Projects
None yet
Development

No branches or pull requests

3 participants