Should we move GPUs from OpenStack to OpenShift?

[joachimweyl]

Motivation

We have 16 V100s at 4%-20% usage and 16 A100SXM4s at 0% usage in OpenStack. Is OpenShift a better location for them? Also worth noting is that we have a request for 4 SXM4 BM nodes pending from RH we might not even need to move them into OpenShift we can move them to ESI and then directly to BM for a while until RH is done with them and then move them into OpenShift Prod.
We already have one request to move a V100 GPU out of OpenStack.

Completion Criteria

Decide if we should move some of the GPU nodes from OpenStack to OpenShift.

Description

• Decide if we should move GPUs from OpenStack to OpenShift
• Decide how many of each GPU node we should move
• Create an issue to move them.

Original Move Suggestion

• 6 V100 nodes with 12 V100 GPUs
• 6 A100SXM4 nodes with 12 A100SXM4 GPUs

Completion dates

Desired - 2024-08-09
Required - 2024-08-21

[joachimweyl]

@syockel your thoughts on this would be helpful

[joachimweyl]

If we can't come to a decision by next Wed I will bring it up in the Operations meeting.

[joachimweyl]

Decision made with Scott & Michael to move 8 V100s and 8 A100SXM4s out of OpenStack. 1 V100 to go to OpenShift testing cluster, 7 V100s to go to OpenShift production. all 8 A100SXM4s to go to OpenShift Production.

[joachimweyl]

#680 created.