PyTorch GPU memory allocation

[riyadshairi979]

How to prevent shared libraries from allocating memory in GPU? I see that even before any shared library function is used, GPU memory uses increases significantly with PyTorch as soon as the process gets started. Any workaround?

    torch::Device cpuDevice(torch::kCPU);

    torch::Tensor t1 = torch::ones({5, 5}, gpuDevice);
    torch::Tensor t2 = torch::ones({5, 5}, gpuDevice);
    torch::Tensor t = t1 + t2;

    torch::Tensor tcpu = t.to(cpuDevice);

With this simple example code nvidia-smi shows usage of 781MB of memory!

| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0       467      C   ./example-gpu                                633MiB |
|    0      2283      G   /usr/lib/xorg/Xorg                           104MiB |
|    0      2320      G   /usr/bin/gnome-shell                          55MiB |
|    0      3184      G   /usr/lib/xorg/Xorg                           390MiB |
|    0      3321      G   /usr/bin/gnome-shell                         297MiB |
|    0      4239      G   ...quest-channel-token=8978750699062060003    41MiB |
|    0     30984      G   ...uest-channel-token=11524677140754815412    44MiB |
+-----------------------------------------------------------------------------+

cc @ngimel

[ptrblck]

The first CUDA operation will create the CUDA context on your device, which will use some memory.
I'm not sure, if I understand the question correctly regarding the shared libraries.

[riyadshairi979]

I see increased gpu memory even before the first CUDA operation. Trying to see where it gets allocated.