b3181

tokenizer : BPE fixes (#7530)

* Random test: add_bos_token, add_eos_token
* Random test: add BPE models for testing
* Custom regex split fails with codepoint 0
* Fix falcon punctuation regex
* Refactor llm_tokenizer_bpe: move code to constructor
* Move 'add_special_bos/eos' logic to llm_tokenizer_bpe
* Move tokenizer flags to vocab structure.
* Default values for special_add_bos/eos
* Build vocab.special_tokens_cache using vocab token types
* Generalize 'jina-v2' per token attributes
* Fix unicode whitespaces (deepseek-coder, deepseek-llm)
* Skip missing byte tokens (falcon)
* Better unicode data generation
* Replace char32_t with uint32_t

b3177

chore: clean useless beam search param (#7985)

Signed-off-by: thxCode <thxcode0824@gmail.com>

b2955

CUDA: deduplicate mmq code (#7397)

b2921

Update and fix Vulkan soft_max and argsort implementations (#7237)

* Update and fix Vulkan softmax implementation

* Update and fix Vulkan argsort implementation