feat: AMD GPU support

[cyberpython]

Feature request

It would be nice to have the option to use AMD GPUs that support ROCm .

PyTorch seems to support ROCm AMD GPUs on Linux - the following was tested on Ubuntu 22.04.2 LTS with an AMD Ryzen 5825U (Radeon Vega Barcelo 8-core, shared memory) and ROCm 5.5.0 and PyTorch for ROCm 5.4.2 installed:

>>> import torch
>>> torch.cuda.is_available()
True

A cursory look seems to indicate that currently there are only CPU and NVidia Resource implementations in BentoML:

BentoML/src/bentoml/_internal/resource.py

Line 217 in c346890

class NvidiaGpuResource(Resource[t.List[int]], resource_id="nvidia.com/gpu"):

Motivation

This feature would enable running models with OpenLLM on AMD GPUs with ROCm support.

Other

No response

[aarnphm]

Let's open an issue over there? I think the default resource and strategies for BentoML will need some more discussion internally from the team and how we could support AMD GPUs on K8s

[cyberpython]

Issue opened under bentoml/OpenLLM, closing this one