Skip to content

b12006

  • b12006
  • ab3768c
  • Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
  • Choose a tag to compare

  • b12006
  • ab3768c
  • Choose a tag to compare

  • Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
@ServeurpersoCom ServeurpersoCom tagged this 30 Oct 08:11
* memory: refuse rollbacks past the snapshots of the last ubatch

The recurrent memory and the DSV4 cache accepted any rollback up to
n_rs_seq, whether or not the last ubatch wrote that snapshot. Each now
records the deepest rollback its last ubatch guarantees and seq_rm
refuses anything past it:

- recurrent: n - 1 for a ubatch of n <= n_rs_seq + 1 tokens, 0 for a
  longer one, which only guarantees the final state
- DSV4: min(n, n_rs_seq), the snapshots are rebuilt from the state
  before the ubatch

The dead rollback block of the rollback test becomes the assertion it
asked for, and rollback_bound checks the exact bound of each memory
after ubatches of 1, 2, n_rs_seq + 1 and n_rs_seq + 2 tokens.

* tests: address review from ggerganov

Detect architectures with llama_arch_from_string in comp_padding and
arch_reserves_single_seq, as in rollback_bound.
Assets 2
Loading