ERROR: Detected NVIDIA Tesla K80 GPU, which is not supported by this container, STT Docker Image

[jmukiibi]

While running the STT docker image on Tesla K80 GPU,
docker run -it --gpus all --mount type=bind,source="$(pwd)"/stt-data,target=/code/stt-data 5adb1e5d8af5

The container starts and once the image is loaded, I'm welcomed by the following message:

ERROR: Detected NVIDIA Tesla K80 GPU, which is not supported by this container
ERROR: No supported GPU(s) detected to run this container

NOTE: Legacy NVIDIA Driver detected. Compatibility mode ENABLED.

That's the message I get while running on a p2.xlarge 1 GPU, 4 vCPUs and 61 GB RAM --- Still a Tesla K80 . I get the following message

---

On a p2.8xlarge, 8 GPUs, 32vCPUs and 488 GB RAM

== TensorFlow ==

NVIDIA Release 21.05-tf1 (build 22596046)
TensorFlow Version 1.15.5

Container image Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved.
Copyright 2017-2021 The TensorFlow Authors. All rights reserved.

NVIDIA Deep Learning Profiler (dlprof) Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved.

Various files include modifications (c) NVIDIA CORPORATION. All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license
ERROR: Detected NVIDIA Tesla K80 GPU, which is not supported by this container
ERROR: Detected NVIDIA Tesla K80 GPU, which is not supported by this container
ERROR: Detected NVIDIA Tesla K80 GPU, which is not supported by this container
ERROR: Detected NVIDIA Tesla K80 GPU, which is not supported by this container
ERROR: Detected NVIDIA Tesla K80 GPU, which is not supported by this container
ERROR: Detected NVIDIA Tesla K80 GPU, which is not supported by this container
ERROR: Detected NVIDIA Tesla K80 GPU, which is not supported by this container
ERROR: Detected NVIDIA Tesla K80 GPU, which is not supported by this container
ERROR: No supported GPU(s) detected to run this container

NOTE: Legacy NVIDIA Driver detected. Compatibility mode ENABLED.

I'm I right to say that the this image works for a section of the NVIDIA GPUs?

[reuben]

The base image we use is an official NVIDIA image, so this is a bit surprising. You might want to try reverting the base image to tensorflow/tensorflow:1.15.4-gpu-py3 and seeing if that helps, but otherwise this is at most a documentation bug on our side.

[jmukiibi]

Alright, Thanks @reuben

[reuben]

@jmukiibi have you tested my suggestion? Did it work? Would be nice to know so we can include it in docs.

[jmukiibi]

@reuben I didn't try this out though. I ended up switching to a Tesla V100 GPU before I saw the response. Its worth a try. I will try to create the same instance and reproduce the error then try to see how it goes. Thanks

[etlweather]

I have a different GPU (RTX 3070) but had the same issue. So I built a container by changing the Dockerfile.train base image (FROM) with nvcr.io/nvidia/tensorflow:21.12-tf1-py3. I had to make a few tweaks but it overall worked and I was able to create a new audio model using this container and my 4 GPUs.

But then to export it to TFLITE, I used the Docker image from the documentation as the latest TF1 does not support TFLITE.