Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New CI Runners #2087

Merged
merged 29 commits into from
Nov 20, 2023
Merged

New CI Runners #2087

merged 29 commits into from
Nov 20, 2023

Conversation

muellerzr
Copy link
Collaborator

@muellerzr muellerzr commented Oct 26, 2023

Migrates CI runners to use AWS

Some notes:

  • Use [self-hosted, intel-cpu, 8-cpu, ci] to utilize a CPU runner. These aren't stored in our local runners and are spun up by demand, hence why they do not show up in settings
  • Use [self-hosted, docker-gpu] for GPU runners. (We can also fine-tune these using push-ci and nightly-ci for whether it's a nightly run or pushing to main)
  • In general now these CI runners assume no persistent storage to make things easier

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Oct 26, 2023

The documentation is not available anymore as the PR was closed or merged.

@muellerzr muellerzr changed the title [DO NOT MERGE] New ci setup/modifications for new runners Make merge tests more reliable Nov 1, 2023
@muellerzr muellerzr marked this pull request as ready for review November 17, 2023 14:11
@muellerzr muellerzr changed the title Make merge tests more reliable New CI Runenrs Nov 17, 2023
@muellerzr muellerzr changed the title New CI Runenrs New CI Runners Nov 17, 2023
Copy link
Contributor

@ydshieh ydshieh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall lgtm, but having 2 questions that confuse me ..?

.github/workflows/build-docker-images-release.yml Outdated Show resolved Hide resolved
.github/workflows/self_hosted_integration_tests.yml Outdated Show resolved Hide resolved
@muellerzr muellerzr merged commit ca300c0 into main Nov 20, 2023
24 checks passed
@muellerzr muellerzr deleted the new-CI branch November 20, 2023 16:41
@muellerzr muellerzr restored the new-CI branch November 20, 2023 17:06
graemenail pushed a commit to graemenail/accelerate that referenced this pull request Nov 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants