Why does performing inference consumes VRAM? #12090
malfonsoarquimea
started this conversation in
General
Replies: 1 comment 1 reply
-
|
it requires memory to compute activations and intermediate states when you do a forward pass. |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
Thanks, this is what i wanted to know! |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Once the model is already loaded into memory and so are the data points that will be used to train the model, why performing the forward pass consumes memory? How to calculate how much memory will be consumed?
Thanks very much, I am starting to dig into memory consumption optimization and I have a lot of doubts. If you consider that I should read something to better understand my problem, just let me know. Thanks
Beta Was this translation helpful? Give feedback.
All reactions