Skip to content

veloGB10 v0.5.2

Choose a tag to compare

View all tags
@sf-stav sf-stav released this 02 Sep 17:34
v0.5.2
7c7a51f

veloGB10 v0.5.2 — vLLM-compatible tokenize / detokenize endpoints

POST /v1/tokenize — vLLM-compatible tokenization: {tokens, count, max_model_len}, a pure
tokenizer call (no forward / KV / GPU). Accepts a prompt string or a chat messages array; the
messages mode renders exactly as the chat path so its count equals usage.prompt_tokens. Empty
prompt returns {tokens: [], count: 0} (vLLM behavior); over-length returns 400
code=context_length_exceeded (truncation keeps the last n tokens).

POST /v1/detokenize — vLLM-compatible decode half of the pair ({model, prompt}) for exact-N
prompt building.

Added so our engine can be benchmarked more correctly. max_model_len mirrors the configured context
size. Minor bug fixes and optimizations.

Source: dev repo @ 5f6b77c. Built from the public tree (self-contained), binary sha256
dbaf23dc88c61363778d25871af961a2a96eca87ff89a3e0da085601e6e68e1e.

Downloads

  • velogb10-v0.5.2-gb10-sm121.tar.gz - engine binary + all 15 PTX kernel artifacts (incl. vision) + TP launch scripts
  • SHA256SUMS.txt - checksums
  • PROVENANCE.txt - build provenance
Assets 5
Loading