Lamina v0.3.1

v0.3.1

Requests that do not set sampling now use each model's recommended settings
instead of greedy decoding, and MTP speculation also speeds up those sampled
requests. The server implements presence_penalty, and setup offers an
abliterated build of Qwen3.6 as a third model.

• Omitted sampling fields come from the model card. Qwen3.6 uses temperature
  1.0, top_p 0.95, top_k 20 and presence_penalty 1.5 with thinking, and 0.7 /
  0.8 / 20 / 1.5 without it. Ornith uses 0.6 / 0.95 / 20 / 0 in both modes.
  Fields a request sets always win, and "temperature": 0 stays plain greedy.
  Set LAMINA_SAMPLING_DEFAULTS=greedy to restore the old behavior.
• presence_penalty (-2 to 2) penalizes every token that already appeared in
  the current answer, in both ordinary and speculative sampling.
• Sampled MTP: at nonzero temperature the engine verifies the MTP draft against
  its filtered distribution, so output follows the requested sampling. On an
  RTX 3050 8 GB / Ryzen 7 5700X, Qwen3.6 with the non-thinking defaults ran at
  43.9 tokens/s with MTP against 37.2 without it, and llama.cpp b11474 ran at
  28.2 with the same settings. Greedy decoding stays faster at 46.9 tokens/s.
  Set LAMINA_SAMPLED_MTP=0 to turn sampled speculation off.
• GPU image encoding no longer discards the conversation: the engine saves its
  device-KV checkpoint to a temporary file before the encoder takes the GPU and
  restores it afterwards, so follow-up turns prefill only new content.
• START-HERE.bat --model abliterated selects an abliterated (safety
  guardrails removed) Qwen3.6-35B-A3B v4 Q4_K_M. Setup downloads it and
  re-encodes its SSM tensors to F32 when needed. This build has not been
  benchmarked.

No model layer math changed. In the RTX 3050 checks, greedy output stayed
token-identical to the engine before the sampling changes. RTX 30/40/50 kernels are compiled. Runtime checks ran
on RTX 3050 8 GB and RTX 4060 8 GB from source builds; see the README benchmark
records for commands and conditions. SHA-256 checksum files accompany both
packages.

The v0.3.0 tag was never released: its release build stopped because the
build script no longer copied the cuBLAS runtime. v0.3.1 has the same changes
plus that fix. A locally built v0.3.1-equivalent package passed START-HERE.bat,
streaming text, tool calls, thinking, sampled and greedy MTP, and CPU image
input on the RTX 3050 with only Windows system directories in PATH.

v0.2.0

Run the local API server with GPU image input and sampled video from the portable
Windows package. Both CPU and GPU image encoders, Python, CUDA runtimes and the
video decoder are bundled. Qwen3.6 and Ornith use their own verified projectors;
model data stays in the sibling Lamina-data folder.

• START-HERE.bat --reconfigure saves image/video input and CPU/GPU selection.
  Use --vision-device gpu for GPU image encoding. The default launch starts
  the OpenAI-compatible server; chat supports image/video files and prompts.
• Pi integration includes an image configuration guide and a lamina-video
  skill that samples local clips and submits frames to the running server.
  MP4/H.264 is recommended; MOV, MKV, AVI and WebM were also checked. Video
  analysis uses sampled images, without audio or native video input.
• GPU vision and text inference take turns using VRAM, fixing the resident
  encoder crash during long conversations on 8 GB cards.
• A bounded image-embedding cache and image-aware conversation checkpoints
  prevent repeated images from forcing another full prefill. New images and
  server restarts can still require a slow initial pass.
• Long text around images uses the optimized prefill schedule. On RTX 4060 /
  Ryzen 7 1700X, a paired 32K-token two-image check improved first content from
  90.3 to 63.2 seconds with identical answers. See the linked README benchmarks
  for commands, memory use and limits; short OCR replies are not decode-speed
  benchmarks.

The portable and engine-only ZIPs include the server engine and both encoders.
Windows x64, a compatible NVIDIA driver and sufficient RAM remain required.
RTX 30/40/50 kernels are compiled; runtime validation is on RTX 4060. No model
layer math was changed. SHA-256 checksum files accompany both packages.

v0.1.4

Choose Qwen3.6-35B-A3B or Ornith-1.5-35B-A3B in the portable Windows ZIP.
Both support text, tools, reasoning and image input. Python, the engine and
the CPU image encoder are included; no compiler or CUDA toolkit is needed.
Setup downloads the selected model and its own verified image projector.

START-HERE.bat --reconfigure asks for model, context and MTP again; Enter
keeps each saved choice. The README explains setup, saved settings and coding
clients, including advertising image support in pi's model configuration.

Retain the full 128K context option and include the measured expert-admission
scheduling improvements. No model math, quantization or history truncation is
changed. Image preprocessing uses sufficient resolution for text reading;
CPU encoding can take a minute or more on older CPUs. Compiled GPU coverage is
RTX 30/40/50; this release is tested on RTX 4060, with other generations pending.

Earlier releases

Lamina's first portable Windows x64 release runs Qwen3.6-35B-A3B with a local
OpenAI-compatible API, streaming tool calls, thinking, and optional greedy MTP
speculative decoding.

Download lamina-windows-x64-portable.zip, extract into a writable folder,
and run START-HERE.bat. Python, the engine and runtime libraries are included;
no developer toolchain is needed. The model downloads separately on first use
into the sibling Lamina-data folder (22 GB plus optional MTP/assets).

Requirements: Windows x64, AVX2/FMA/F16C/BMI2 CPU, NVIDIA RTX 30/40/50 GPU with
at least 8 GB VRAM, CUDA 13.3-compatible driver, 64 GB RAM recommended, and about
40 GB free disk space. This package includes text inference; the optional image
encoder is excluded. SHA-256 files accompany both release ZIPs.

The engine-only ZIP is for the source launcher's automatic download after
publication. Ordinary users should choose the portable ZIP.

Draft validation on an RTX 4060 / Ryzen 7 1700X / 64 GB Windows machine:

• Hardware-free default CMake configure and lamina-gguf build passed.
• All 46 Python tests passed, including verified release download/install,
  corrupt checksums, archive traversal and compiler-free portable setup.
• Portable native sampling, CUDA elementwise and attention checks passed.
• Engine DLL imports contain cuBLAS and Windows system DLLs; no MSVC runtime
  dependency. Embedded Python imports and START-HERE.bat setup passed in a
  fresh extracted directory with PATH limited to Windows system directories.
• After the owner stopped the existing server, the exact uploaded portable ZIP
  passed START-HERE.bat model preload, greedy generation with MTP enabled,
  streaming text and usage, streaming function name/arguments, and streamed
  reasoning on the RTX 4060, with only Windows system directories in PATH.
• RTX 30/50 kernels compiled; their runtime is unverified. This is a functional
  packaging check, not a performance benchmark or a clean-Windows-install test.
  No new performance claim is made.

Published as v0.1.0 (prerelease). RTX 30/50 runtime validation remains unverified.

v0.1.1

Reuse unchanged system/tool prefixes for CUDA device-KV server requests.
On RTX 4060 / Ryzen 7 1700X / 64 GiB, repeated 1,602-token Pi requests reached
the first token in 0.33 s versus 5.03 s with reuse disabled. The first request
still builds the prefix and took 4.53 s. Host-KV and image requests retain the
existing path. Old engine binaries are detected and use RESET. Disable reuse
with LAMINA_PREFIX_CACHE=0. See the benchmark record for exact conditions.

Fix runtime dependency installation and a Windows cancellation-test race in CI.

v0.1.2

Reuse the whole conversation prefix across agent turns. v0.1.1 could only reuse an
unchanged system/tools block, so coding agents that resend the conversation each
turn re-prefilled the entire prompt every request: 41-47 s to the first token at
38K prompt tokens. The server now caches the message history up to the generation
prompt and advances that checkpoint when the conversation grows by an append,
prefilling only the new messages. On the RTX 4060 Ti 16 GB the second turn of a
grown conversation reached the first token in 0.5 s versus 5.2 s cold, with
identical output; a checkpoint that is not an exact BPE prefix still falls back to
RESET. The server log now reports new versus reused prompt tokens.

Fast mode also uses the overlapping expert pipeline while the GPU expert cache is
still filling and switches to the serial path once it is warm, roughly doubling
the first ~32 decode tokens of a cold request without changing steady decode;
LAMINA_HYBRID=0 restores the previous behavior. Under cache eviction the
selected trajectory can differ from the previous release, as it already does
between the stream and cpu-miss policies. LAMINA_PREFIX_CACHE=0 disables prefix
reuse. See the conversation-prefix and warm-up benchmark records for conditions.

v0.1.3

Run Ornith-1.5-35B-A3B, and choose the model at setup. Ornith shares Lamina's
qwen35moe graph, so the engine now tolerates its in-file MTP layer
(block_count == 40 + nextn_predict_layers) and splits the grouped qkv/gate and
q/k/v projections (single-token and pair) when their encodings differ, instead of
requiring one encoding per group. This also fixes a null MoE output buffer on the
scalar fallback path.

START-HERE.bat --model ornith (or the first-run question) selects Ornith; the
default stays Qwen3.6. python -m tools.lamina_ornith setup fetches Ornith and its
tokenizer, re-encodes the two SSM tensors to F32, and packs the in-file MTP head
(`too...

Lamina v0.2.0 — GPU images, video and server

v0.2.0

Run the local API server with GPU image input and sampled video from the portable
Windows package. Both CPU and GPU image encoders, Python, CUDA runtimes and the
video decoder are bundled. Qwen3.6 and Ornith use their own verified projectors;
model data stays in the sibling Lamina-data folder.

• START-HERE.bat --reconfigure saves image/video input and CPU/GPU selection.
  Use --vision-device gpu for GPU image encoding. The default launch starts
  the OpenAI-compatible server; chat supports image/video files and prompts.
• Pi integration includes an image configuration guide and a lamina-video
  skill that samples local clips and submits frames to the running server.
  MP4/H.264 is recommended; MOV, MKV, AVI and WebM were also checked. Video
  analysis uses sampled images, without audio or native video input.
• GPU vision and text inference take turns using VRAM, fixing the resident
  encoder crash during long conversations on 8 GB cards.
• A bounded image-embedding cache and image-aware conversation checkpoints
  prevent repeated images from forcing another full prefill. New images and
  server restarts can still require a slow initial pass.
• Long text around images uses the optimized prefill schedule. On RTX 4060 /
  Ryzen 7 1700X, a paired 32K-token two-image check improved first content from
  90.3 to 63.2 seconds with identical answers. See the linked README benchmarks
  for commands, memory use and limits; short OCR replies are not decode-speed
  benchmarks.

The portable and engine-only ZIPs include the server engine and both encoders.
Windows x64, a compatible NVIDIA driver and sufficient RAM remain required.
RTX 30/40/50 kernels are compiled; runtime validation is on RTX 4060. No model
layer math was changed. SHA-256 checksum files accompany both packages.

Extracted v0.2.0 server validated on RTX 4060 8 GB / Ryzen 7 1700X / 64 GB RAM with PATH restricted to Windows system directories. Native checks, embedded Python imports, API streaming/tools/reasoning, GPU image encoding, image cache reuse and sampled video passed. RTX 30/50 runtime remains unverified.

Lamina v0.1.4 — portable Ornith and image support

v0.1.4

Choose Qwen3.6-35B-A3B or Ornith-1.5-35B-A3B in the portable Windows ZIP.
Both support text, tools, reasoning and image input. Python, the engine and
the CPU image encoder are included; no compiler or CUDA toolkit is needed.
Setup downloads the selected model and its own verified image projector.

START-HERE.bat --reconfigure asks for model, context and MTP again; Enter
keeps each saved choice. The README explains setup, saved settings and coding
clients, including advertising image support in pi's model configuration.

Retain the full 128K context option and include the measured expert-admission
scheduling improvements. No model math, quantization or history truncation is
changed. Image preprocessing uses sufficient resolution for text reading;
CPU encoding can take a minute or more on older CPUs. Compiled GPU coverage is
RTX 30/40/50; this release is tested on RTX 4060, with other generations pending.

Validated on RTX 4060 8 GB / Ryzen 7 1700X / 64 GB RAM: 51 Python tests, default CMake/lamina-gguf, native sampling/CUDA elementwise/attention, and extracted portable startup at 128K capacity with MTP, SSE text/usage, streamed tools, reasoning and prefix reuse for both models. Both models read the image fixture; the extracted portable Ornith image request also passed with the application in a folder containing spaces and PATH limited to Windows system directories. First image responses took about 84–87 seconds on this older CPU. This is a functional check, not a full-context speed or general OCR accuracy claim. RTX 30/50 runtime remains unverified.