Clarify pip install instructions: GPU vs. CPU

[felker]

Is your feature request related to a problem? Please describe.

I have seen quite a few issues posted where the poster has installed both pip install onnxruntime onnxruntime-gpu.

At least since September 2019 and #1826, as far as I understand it, if you have installed onnxruntime-gpu via pip there is no reason to install onnxruntime and you can switch to the CPU provider programmatically in Python, e.g. discussion in #486 and #7313.

Could it cause issues having both packages installed at once?

Describe the solution you'd like

https://onnxruntime.ai/docs/get-started/with-python.html would ideally be updated with an explicit comment stating that only onnxruntime-gpu should be installed to get CPU and GPU capabilities on machines with GPUs.

I believe at one point TensorFlow's documentation https://www.tensorflow.org/install/pip indicated something similar for tensorflow vs tensorflow-gpu meta-package, beforetensorflow-gpu became the same as tensorflow after 2.1.

For those who hadnt used tensorflow-gpu back then, the parallel distinction between onnxruntime vs. onnxruntime-gpu might be lost on them and hence it would be valuable to have explicit instructions in the ORT documentation.

[snnn]

Nat Kershaw (MSFT) (@natke) please let me know what you think.

Technically, these two packages have some other slight differences:

1. The CPU one does not have C++ RTTI. GPU does.
2. The CPU one enables Link-Time Code Gen. The GPU one could do the same, but I haven't tried.
3. The CPU one use protobuf-lite, the GPU one use the full version of protobuf because TensorRT need it.
4. The CPU one can support ARM , the GPU one haven't.
5. The CPU one can support macOS , the GPU one doesn't.

Things like this. For example, as long we still have a CPU package, I would not add macOS support to the GPU package. We will not be able to make CUDA working on macOS, I'm saying CPU-only. It makes no sense to make a GPU package without having GPU functionality.

From technical perspective, the CPU package always has its unique value. But as you said, in most cases these things are not important.

Could it cause issues having both packages installed at once?

I believe so.

[natke]

snnn (@snnn), thanks for pointing out the nuances between the two packages. It sounds like for most cases, you can install onnxruntime-gpu and have it work on CPU.

I can add a note to this effect, pointing out the cases where this is not true.

Let me know whether this approach makes sense.

[snnn]

I think we don't need to emphasize the tiny differences. From product point of view you may make the decision how we recommend the two packages to our users.

[felker]

Thanks for the reply and details SnN (@snn). As a user, I think adding a line or two to https://onnxruntime.ai/docs/get-started/with-python.html should mention:

1. only install one of the two pip packages
2. the GPU package includes most CPU capabilities
3. stick to onnxruntime for ARM CPUs and/or macOS operating system

[natke]

Kyle Gerard Felker (@felker) and snnn (@snnn) Check out https://natke.github.io/onnxruntime/docs/get-started/with-python.html and let me know what you think

[natke]

Fixed by #10718