Replies: 1 comment
-
|
I tried on build ~9300 with --cache-ram -1 and it did not work either but I was not playing with checkpoints |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
My launch arguments has:
I initially assumed that --cache-ram is the total memory cap for kv and such but I guess I am wrong.
When --ctx-checkpoints is set to 1, it does limit the consumption to be around ~11GB but when increase to 2, it can shoot up to 3x GB. The math doesn't make sense to me. The md doc only mention
Set the maximum cache size in MiB. 💢Beta Was this translation helpful? Give feedback.
All reactions