Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.github/workflows/ci-linux.yml: Fine-tune max-parallel #36660

Merged
merged 1 commit into from
Nov 14, 2023

Conversation

mkoeppe
Copy link
Member

@mkoeppe mkoeppe commented Nov 5, 2023

... for constructive clogging, #36616 (comment)

📝 Checklist

  • The title is concise, informative, and self-explanatory.
  • The description explains in detail what this PR is about.
  • I have linked a relevant issue or discussion.
  • I have created tests covering the changes.
  • I have updated the documentation accordingly.

⌛ Dependencies

@kwankyu
Copy link
Collaborator

kwankyu commented Nov 5, 2023

In addition to that, I still think "minimal" and others should be put to the queue by smaller chunks, max-parallel 10 or something (instead of current default 30), to allow room for PRs.

But this also seems to depend on how many PR jobs are waiting to get into the queue. If that number is small, perhaps we don't need that adjustment. Let's see.

Copy link

github-actions bot commented Nov 6, 2023

Documentation preview for this PR (built with commit 8e26e8e; changes) is ready! 🎉

@kwankyu
Copy link
Collaborator

kwankyu commented Nov 6, 2023

No idea how the specific numbers 8 and 24 are obtained.

Anyway, overall decreasing of max-parallel numbers looks good to me.

@mkoeppe
Copy link
Member Author

mkoeppe commented Nov 6, 2023

Thanks!

@kwankyu
Copy link
Collaborator

kwankyu commented Nov 12, 2023

Blocker?

@mkoeppe
Copy link
Member Author

mkoeppe commented Nov 12, 2023

Sure, to get it merged in the next rc.

@kwankyu
Copy link
Collaborator

kwankyu commented Nov 13, 2023

"default" workflow finished after ~3 hours after release. Now 4 hours have passed since release. But the queue is still clogged with jobs of "standard-pre" and "minimal-pre". Many B&T jobs for PRs are waiting in the queue. We still have this problem.

This PR seems insufficient to solve the problem. How about removing "minimal-pre" jobs from the first batch and putting it after "standard-pre"? Decreasing max-parallel for all jobs except "standard-pre" seems to help. No?

@mkoeppe
Copy link
Member Author

mkoeppe commented Nov 13, 2023

I think a big chunk of extra clogging comes from https://github.com/sagemath/sage/actions/runs/6842482905, a "CI Linux incremental" workflow run with >= 24 parallel jobs.

@mkoeppe
Copy link
Member Author

mkoeppe commented Nov 13, 2023

Hence #36697

@vbraun vbraun merged commit 9953321 into sagemath:develop Nov 14, 2023
16 of 23 checks passed
@mkoeppe mkoeppe deleted the ci_clogging_refine branch November 15, 2023 05:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants