how to release gpu memory after session.run

[ZTurboX]

Describe the issue

how to release gpu memory after session.run

To reproduce

how to release gpu memory after session.run

Urgency

No response

Platform

Linux

OS Version

ubuntu

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.17.1

ONNX Runtime API

Python

Architecture

X86

Execution Provider

CUDA

Execution Provider Library Version

cuda 11.7

[pranavsharma]

Just let the session destruct if you don't intend to use the session anymore. All memory is released to the GPU when the session's destructor is called. If you intend to use the session again, you can configure this run option:

onnxruntime/include/onnxruntime/core/session/onnxruntime_run_options_config_keys.h

Line 27 in 72ce4de

static const char* const kOrtRunOptionsConfigEnableMemoryArenaShrinkage = "memory.enable_memory_arena_shrinkage";

[ZTurboX]

Just let the session destruct if you don't intend to use the session anymore. All memory is released to the GPU when the session's destructor is called. If you intend to use the session again, you can configure this run option:

how to use python method