Skip to content

Releases: arterialist/kube32

Release list

kube32 0.1.0

Choose a tag to compare

@github-actions github-actions released this 04 Nov 07:02

Added

  • Pod workers on both cores (docs/architecture/wasm-runtime.md). A board with PSRAM ticks pods and delivers their events on two workers, one per core, so two busy pods run at the same time. kube32.io/core puts a pod on a core; otherwise a pod goes to the less busy worker, a pod that calls another in-process brings the callee onto its worker, and every 2 s the busier worker gives a group of pods to the other. K32_CMD:PODCORES:1 keeps the single tick and event threads of a board without PSRAM.

  • JSON Patch (application/json-patch+json, RFC 6902) on every resource that accepts a patch, so kubectl patch --type json, Helm, Argo CD and k9s work (docs/api-reference.md). The operations are add, remove, replace, move, copy and test at JSON Pointer paths, with ~0 and ~1 escapes and - to append to an array. A patch applies whole or not at all, and a failing operation answers 422 Invalid. The Deployment scale subresource takes JSON patches against the Scale object.

  • The wordsketch example (apps/wordsketch, examples/wordsketch.yaml) turns a word spoken to the Cardputer into a doodle and a short story. A pod on the Cardputer listens with a keyword spotter (cat, dog, bird, house, tree, bed; integer front end, weights read in place from a blob) in 79 KiB of its pod pool, an entry-point pod on a board with PSRAM draws the word with a class-conditional diffusion model and publishes every step's image, stories260K writes the story, and the Cardputer's screen shows both as they form. The models, their training code and their C inference, which computes the reference's bits, are in models/.

  • k32_attention_full_f32_i8 (docs/app-development.md): the attention of several query positions that each see the whole window, every head in one call, the bits of the three calls per head. The int8 and int4 products on the ESP32-S3 hold a chunk of 64 weights in vector registers for every vector of a k32_matmul_* call and store a row block's results vector by vector.

  • k32_attention_f32_i8 (docs/app-development.md): the attention of several query positions over an int8 KV cache, every head in one call, the bits of k32_matvec_f32_i8, k32_softmax_f32 and k32_vecmat_f32_i8 per head. llm-speculative and llm-pipeline attend with it: one call per layer for all positions of a pass instead of three per head and position.

  • k32_conv3x3_i8 (docs/app-development.md): a 3x3 convolution with padding 1 over several int8 maps in one call, with stride 1 or 2 and the input read as its 2x nearest-neighbour upsample, the weights read once per call in cache-sized blocks and the products on the vector instructions of the ESP32-S3; the bits of k32_matvec_i8 over each pixel's patch. k32_add_f32_i8 adds a scaled int8 map to floats, and k32_quantize_i8_static quantizes at a fixed scale. The doodle model of wordsketch draws with them: one call per chunk of output rows for both conditions of a guided step, and the step and class bias, the skip, SiLU and the next input's quantization as element-wise calls per row.

  • k32_fft_q15 (docs/app-development.md): the butterflies of an integer radix-2 FFT with Q15 twiddles, natively, for audio front ends. n from 2 to 4096 (a power of two); K32_ERR_INVAL otherwise. A 512-point frame of the keyword spotter's front end drops from 127,000 interpreter operations to 30,000 with the same bits.

  • kube32-sim --sim-mic <file> plays a recording to k32_mic_read in real time (docs/cli-reference.md).

  • kube32-sim workers follow the network (docs/cli-reference.md). --node-ip auto, the default for a worker whose API server is on another host, reports the local address that the system uses to reach the API server and patches the Node's InternalIP when the network changes. A control plane started with --node-ip announces its API address in HELLO frames, and a worker that has had no answer from its API server for 30 seconds dials the announced address in place, without restarting its pods, as boards do. A worker that dials another host also sends its HELLO to the UDP broadcast address on port 7400, so that a control plane that has lost track of it answers with its API address.

  • Blobs (docs/storage.md): named, read-only data such as model weights, which pods read in place instead of loading it into their memory. A node keeps them in a flash region mapped into the address space (a blobs data partition, or the passive app slot between firmware updates) and in kube32/blobs/ on its microSD card; kube32-sim keeps the region in a file. Pods open the blobs named in kube32.io/blobs with k32_blob_open, k32_blob_size, k32_blob_read, k32_blob_mapped and k32_blob_close, and the linear algebra imports read weights and scales from a mapped blob through K32_BLOB_ADDR. The kubelet endpoint serves /blobs through the node proxy. apps/blobbench measures a node's blob throughput and, at /check, compares the linear algebra imports with a pod's own loops, and K32_CMD:BLOBS lists the stores.

  • Cardputer audio (docs/peripherals.md): k32_audio_play plays 16-bit PCM, k32_mic_read records 16 kHz PCM from the PDM microphone (hardware.kube32.io/microphone) with its DC offset removed and a gain of 16, and the speaker never plays louder than a quarter of full scale. The two share a clock pin and take turns, the speaker first. K32_CMD:MIC:<ms> prints the microphone's levels, K32_CMD:MICGAIN:<n> sets its gain and K32_CMD:SDKHZ:<khz> sets the microSD card's SPI clock.

  • K32_CMD:HEAPMIN reports the internal heap's lowest free size over an interval, and [MEM] lines name the task of a failed allocation. Firmware built with CONFIG_HEAP_USE_HOOKS adds K32_CMD:HEAPWATCH, each task's internal allocations at the heap's low point (docs/flashing-troubleshooting.md).

  • Native linear algebra for pods (docs/app-development.md): k32_matvec_i8 and k32_matvec_i4 multiply int8 or int4 matrices with per-row scales by an int8 vector, k32_matvec_f32_i8 and k32_vecmat_f32_i8 compute attention over an int8 KV cache, and k32_softmax_f32 normalizes scores. k32_matmul_i8 and k32_matmul_i4 apply one matrix to several vectors and read each row once for all of them. k32_rmsnorm_f32, k32_quantize_i8, k32_rope_f32, k32_swiglu_f32, k32_add_f32 and k32_argmax_f32 run the element-wise steps of a transformer over many vectors in one call. On the ESP32-S3 the integer products run on the core's vector instructions (PIE). The node checks every operand against the pod's linear memory, an operand outside it traps with a message that names it, and a metered pod pays for the work from its execution budget. The results are the same bits on every node; k32_expf in k32_app_abi.h is the exp the softmax uses. The Rust SDK wraps them in the linalg feature, and kube32 check knows them. llm-pipeline computes its matrix products, attention and element-wise steps with them.

  • models/ (docs/examples/models.md): four small networks with their training code, test vectors and C inference for pods. A target language model for speculative decoding, a masked diffusion language model, a Quick, Draw! doodle denoiser in two sizes, and a Speech Commands keyword spotter trained for far-field speech with room acoustics, noise and other voices. Each model's numpy reference (models/common/k32ref.py) computes the bits a pod computes, make -C models check compares the C inference with the test vectors on the host, and models/playground.py runs the models on the host with a web page. apps/llm-pipeline/tools/spec.py runs speculative decoding as the reference and writes the speculative pod's blobs, and models/kws/farfield.py scores a keyword spotter on far-field, false-trigger and synthesized-voice tests.

  • The llm-speculative example (apps/llm-speculative, examples/llm-speculative.yaml) runs greedy speculative decoding in one pod on the control plane: a 3.6 M-parameter model trained with stories260K's tokenizer reads its int4 weights in place from the node's flash blobs, stories260K drafts k tokens from its own blob, and one pass of k32_matmul_i4 checks them all. Its HTTP API runs a prompt with the target alone, with drafts or with the draft alone, and reports tokens/s, acceptance, the time of each pass and the bytes read from flash. tools/spec.py bench checks every run against the reference, token for token and pass for pass.

  • The llm-diffusion example (apps/llm-diffusion, examples/llm-diffusion.yaml) runs a masked diffusion language model: the weights stay in a flash blob and k32_matmul_i8 reads them in place, and a request returns the window's text after every step with the time it took.

  • The llm-pipeline example (apps/llm-pipeline, examples/llm-pipeline.yaml) runs the stories260K language model as three pipeline stages: the embedding and layers 0-2 on the control plane, layer 3 in int4 in the Cardputer's 80 KiB pod pool, which shows the text on its screen, and layer 4 with the classifier on the fastest node, a kube32-sim worker. A coordinator pod runs greedy generations with an HTTP API and Server-Sent Events and reports per-stage compute and hop times. tools/llm.py quantizes and splits llama2.c models and computes the reference tokens, which the pipeline matches bit for bit. A shard given MODEL_BLOB reads its slic...

Read more