-
-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[DECIDED] Always run all checks for pack CI #7
Comments
Is there any doc that describes the desired outcomes in general for packci? I have quite a bit of experience running my own exchange in the past year, and one think I always wondered, once I figured everything out, is what was the desired outcome of the ci process. I think it’s to prevent bad code from getting into production, especially in community ( plus listing and stuff) but I think an important part of refactoring it is to understand, let’s say, the goals as of know vs. how they might be changed. |
I'm in support of this. Seen a lot of PRs around problems with the selective checks. Always checking everything leaves a lot less room for error. |
I don't remember the reasons behind why we did selective checks in StackStorm-Exchange CI. Maybe dig into Github history? Was that changed at some point or is it designed that way since the beginning? If no important reasons, +1 as well to move to "check all" approach to reduce complexity & edge cases. |
I’m curious with the newer circle credit based payment if this changes the economics significantly. |
BTW we can share the current |
We could alternatively email them and they will CHANGE that org to the FOSS Plan
…Sent from my iPhone
On Apr 20, 2020, at 10:09 AM, Eugen C. ***@***.***> wrote:
BTW we can share the current StackStorm plan with the child StackStorm-Exchange in CircleCI so they'll use paid + free Linux containers from the parent.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
If we can get extra (free) Linux containers on CircleCI, that would be great. |
Okay, tracking this down...
Digging through st2contrib, the check for
However, that PR actually pulled in code from the original PR 332, when the packs tests were still all running on Travis. Reading through that PR, it did change the build times a bit:
[1] This looks like it's comparing apples and oranges though, since the old build may have failed due to timeout and the new build completed successfully. I think it did save the original coders an appreciable amount of time when they were on Travis. But I think CircleCI is just slower overall, since it launches each step separately, and that seems to eat up the most time nowadays. The only pack that's build times will be greatly extended by this proposal is the AWS pack. And to be fair, that pack does seem to see more churn than other packs, but it also has more actions by 1-3 orders of magnitude compared to other packs. I think for all non-AWS packs this isn't going to matter much, and I think the reduced complexity is worth the slightly longer build times. |
Agreed. Make the change. Have seen too many oddities in the past |
Good stuff here with the Discussion, Proposal, root-cause finding and also new solution like applying for FOSS plan for StackStorm-Exchange github org 👍 @blag I assume you're driving the https://circleci.com/docs/2.0/oss/ for https://github.com/stackstorm-exchange/ ? |
Sorry for not replying here. The StackStorm-Exchange organization on CircleCI was already on their OSS plan - yay! |
I guess the current plan for StackStorm-Exchange is just a plan that applies to any org with non-private repositories. No action is needed for that. Our current plan for StackStorm-Exchange is While https://circleci.com/docs/2.0/oss/#overview lists
|
For pack CI builds, we currently try to only run checks on files that have changed in the PR. This is arguably faster, but it comes at the cost of increased complexity, and a higher incidence of related bugs.
It would be much simpler to just always run all tests on all files for packs, regardless of what or how many have changed. This will be slower, but I don't think it would be particularly slower for almost all packs. The only pack that might have issues is the AWS pack since that's got 3000+ actions that all would get checked on every push, but even then that only takes a few minutes to complete.
For merges into
master
and the scheduled weekend tests, we useFORCE_CHECK_ALL_FILES
to check all files in every pack.Does anybody know why we originally tried to restrict what we tested in PRs? @LindsayHill @Kami
The text was updated successfully, but these errors were encountered: