Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

External runners for Windows #2169

Open
jaimergp opened this issue May 1, 2024 · 3 comments
Open

External runners for Windows #2169

jaimergp opened this issue May 1, 2024 · 3 comments

Comments

@jaimergp
Copy link
Member

jaimergp commented May 1, 2024

Comes from Quansight/open-gpu-server#31


@wolfv and @baszalmstra asked in today's core call about the possibility of using the recently added Cirun + self-hosted GHA runner infra to add beefy Windows runners for the most demanding feedstocks. I think the idea is to sponsor this for PyTorch packages, in principle.

From the top of my head, this is what needs to happen, roughly:

  1. Choose a Cirun-compatible cloud provider for the Windows runners.
  2. Create VM images for Windows containing the necessary bits (compilers and other tooling not present by default). Make sure the licensing aspects are ok.
  3. Configure the cloud provider accordingly, and register the credentials in the Cirun app for conda-forge.
  4. Add support for registration and access control in the admin-requests + .cirun infra.
  5. Optional. Figure out the legal bits for liabilities and service abuse (e.g. restrict to approved users or maintainers).
@carterbox
Copy link
Member

  1. Create VM images for Windows containing the necessary bits (compilers and other tooling not present by default). Make sure the licensing aspects are ok.

I worked on creating Docker containers a few years back (2022). AFAIR, the biggest two obstacles were that (1) Windows containers were large: around 11GB after you got all the stuff in there and (2) the "nano" Windows base image was 64-bit only and conda still required some 32-bit dependencies (that's probably changed?).

Maybe the Docker part isn't relevant to this issue, but there's some good links in that PR about how to install the MSVC tool chain without a GUI.

@aktech
Copy link

aktech commented May 15, 2024

Thanks for creating the issue @jaimergp. I have just received an Azure account from @wolfv (thanks for that 🎉 ). I'll soon start the work and update here.

@aktech
Copy link

aktech commented Jun 20, 2024

I have created a PR for the same in the access control repo here: conda-forge/.cirun#14

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

3 participants