Docker run using https://replicate.com/dribnet/pixray-api gives sm_86 error using Nvidia 3090

[valentinvieriu]

I'm trying to run the Docker version of https://replicate.com/dribnet/pixray-api using an Nvidia 3090.

docker run -d -p 5000:5000 --gpus=all r8.im/dribnet/pixray-api@sha256:a249606da3a0c7f32eed4741f1e6f1792470a39a5825fc8814272cceea30ad32
curl http://localhost:5000/predict -X POST \
  -F settings=...

I'm getting this error

/root/.pyenv/versions/3.8.12/lib/python3.8/site-packages/torch/cuda/__init__.py:106: UserWarning:
NVIDIA GeForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.
If you want to use the NVIDIA GeForce RTX 3090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

I suspect the problem is CUDA version (CUDA11.1 ) ( see https://discuss.pytorch.org/t/geforce-rtx-3090-with-cuda-capability-sm-86-is-not-compatible-with-the-current-pytorch-installation/123499/8 )

Is it possible to upgrade CUDA version ing cogs.yaml?

[dribnet]

Yes, try changing the line

  cuda: "10.2"

in cogs.yaml and then match up the lines

torch==1.9.0+cu102
torchvision==0.10.0+cu102

in requirements.txt.

Also note that cog is optional and you can instead just setup a python environment and use requirements.txt