You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Dynamic workflow resume is safer. Checkpoint writes now resume lazily:
when a checkpoint memo exists, (checkpoint :key expr) returns the stored
value without evaluating expr again. Checkpoint journal events also carry
the resume content_key, so checkpoint memo files can be inspected and
invalidated with the same model as agent leaves.
Workflow memo keys now include workflow source and --args. Editing the
workflow or changing run arguments invalidates stale memo hits automatically,
while unchanged leaves still resume per-leaf.
Workflow-declared sandbox permissions are enforced.defworkflow
metadata can declare :permissions using the same syntax as --sandbox
(strict, all, none, or comma-separated denial capabilities such as no-fs-write,no-network). Workflow permissions can only tighten the
caller's sandbox; they cannot loosen a stricter CLI sandbox or --allowed-paths setting.
Crates.io publish order includes sema-workflow. The publish workflow
now publishes the workflow runtime crate before crates that depend on it.
Docs and Website
Workflow documentation caught up with the runtime. The workflow guide,
agent-facing docs, CLI sandbox reference, builtin docs, changelog, and
deferred notes now document :permissions, checkpoint resume behavior,
memo invalidation, and the complete workflow permission list. The abbreviated :perms metadata spelling is not documented or accepted.
Website rendering fixes. The notebook feature page has more stable
shortcut layout, the website logo color is fixed, and generated Open Graph
images are deterministic by using vendored fonts and blocking external font
requests during generation.
Docs/site maintenance. Architecture docs now include the sema-workflow
crate, the workflow docs have an Open Graph image, and the playground loading
screen has a small rotating Lisp-joke set.
Install sema-lang 1.28.1
Install prebuilt binaries via shell script
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/HelgeSverre/sema/releases/download/v1.28.1/sema-lang-installer.sh | sh
7 new chat/inference providers. DeepSeek (DEEPSEEK_API_KEY),
OpenRouter (OPENROUTER_API_KEY), Together AI (TOGETHER_API_KEY),
Fireworks AI (FIREWORKS_API_KEY), Cerebras (CEREBRAS_API_KEY),
SambaNova (SAMBANOVA_API_KEY), and Perplexity (PERPLEXITY_API_KEY).
All are OpenAI-compatible and auto-configured from env vars. llm/configure, llm/auto-configure, and llm/with-fallback support them.
3 new embedding/reranking providers. Nomic (NOMIC_API_KEY),
Together AI, and Fireworks AI now support embeddings and reranking via llm/configure-embeddings and llm/auto-configure. New RerankDialect
variants for each provider's wire format.
:int type validation in llm/extract. The :int type tag is now
properly validated — accepts integers and whole-number floats, rejects
other types.
[:type] list syntax in llm/extract schemas. A field spec like {:authors [:string]} is now sent to the model as "array of string" and
validated as a list where each element matches the inner type.
Fixed
llm/configure-embeddings now wires reranking. Previously, configuring
embeddings via llm/configure-embeddings for Jina, Voyage, or Cohere did
not call .with_rerank() or set_rerank_provider(), so llm/rerank would
fail with "does not support reranking". All three providers now correctly
set up reranking when configured via this path.
llm/extract and llm/classify are now sandbox-gated with Caps::LLM.
Previously only llm/extract-from-image was gated. Text-based extraction
and classification made LLM API calls even when the sandbox denied Caps::LLM.
Dynamic workflows.defworkflow / phase / step / checkpoint / parallel / pipeline — a journaled, resumable agentic-workflow runtime.
Define multi-phase LLM workflows as ordinary Sema code; the runtime journals
every event to a frozen JSONL run directory (.sema/runs/<run-id>/), enforces
budget caps (:tokens / :usd), and supports --resume via content-keyed
memo sidecars. sema workflow run / view / index / check CLI commands.
A web viewer (sema workflow view) renders live runs with a SQLite cross-run
index. sema workflow check statically validates workflow files without
evaluating them — catches arity traps and layout issues before a run.
Doc entries for all workflow builtins are available in LSP hover/completion.
Feature page at /feature/workflows.
:stack-trace on caught error maps. The VM now captures the call stack
at error time and serializes it as a :stack-trace field on caught error
maps — a list of {:name :file :line :col} frame maps, innermost first.
For inline opcodes (+, -, car, etc.), a synthetic intrinsic frame is
synthesized by decoding the opcode at the failing PC. TCO-bounded: tail
calls reuse frames, so the trace stays small even for deep recursion.
Source spans are now threaded through the main eval path
(run_exprs_on_vm) via compile_program_with_spans_and_natives, so
function frames carry :line and :col from the original source.
Fixed
pretty_print no longer double-serializes values.pretty_print called format!("{value}") to check if the compact form fit, then pp_value repeated
the same format! at indent=0 when it didn't — walking and stringifying the
entire tree twice. The redundant check is removed; pp_value handles it
directly. Affects every REPL result, DAP variable inspection, and WASM
playground output.
Runtime / now matches the constant folder for large integers. The
optimizer used exact i64 division while the runtime converted to f64
first, causing (/ 9007199254740993 1) to return different results depending
on whether the operands were compile-time constants. A two-integer fast path
in the runtime / uses exact i64 division when a % b == 0, falling back
to f64 for non-whole results. +, -, * were unaffected (both paths
already used i64).
ValueViewRef — zero-refcount trait impls.PartialEq, Hash, Ord, Display, and pp_value previously used view() which calls Rc::increment_strong_count + Rc::from_raw (and a matching decrement on
drop) for every heap-typed Value comparison. A new ValueViewRef<'a> enum
and view_ref() method use borrow_ref (raw pointer deref) instead,
eliminating refcount mutations on every ==, cmp(), hash(), and format!(). Micro-benchmarked 40–60% faster on eq/cmp for strings,
lists, and maps; 39.9% faster total across the benchmark suite.
VM arithmetic and comparison functions use view_ref().vm_add, vm_sub, vm_mul, vm_div, vm_eq, vm_lt (the handlers for every ADD/SUB/MUL/DIV/EQ/LT/GT/LE/GE opcode) and the stdlib comparison.rs, arithmetic.rs, predicates.rs, math.rs, map.rs, and list.rs modules were migrated from view() to view_ref(), eliminating
refcount churn on numeric and collection operations.
filter no longer double-clones passing items. Each item that passed the
predicate was cloned twice (once for the predicate call, once for the result
vector). Now cloned once and reused for both.
Optimizer extend_shadowed avoids allocation when nothing is shadowed.
The function previously called current.to_vec() on every let/let*/ letrec/lambda/do/try form, even when none of the new names were
foldable builtins (the common case). Now returns Cow::Borrowed when no
foldable names are added, avoiding the allocation entirely.
DAP serde_json::to_string().unwrap() replaced with error handling. Four
sites in the DAP server (event send, initialized event, send_response, send_error) now log serialization errors to stderr instead of panicking.
A malformed debug value would have crashed the entire debug session.
DAP decode_percent off-by-one fixed. The check i + 2 < bytes.len()
required at least one byte after the two hex digits, so a percent-encoded
sequence at the very end of a file URI (e.g. file%20 for a trailing space)
was not decoded. Fixed to i + 3 <= bytes.len().
DAP debug query boilerplate extracted into helpers. The sync_channel(1) + send + spawn_blocking + recv pattern duplicated
6× across setBreakpoints, stackTrace, scopes, variables, evaluate,
and setVariable is now send_cmd_and_recv / send_cmd_and_recv_result.
Formatter token_text returns Cow<str> and token_width avoids
allocation.token_text was -> String, allocating on every call including
in measure_width where only .len() was needed. Now -> Cow<'_, str>
(symbols — the most common token — return Cow::Borrowed), with a separate token_width function that computes the width without allocating.
Formatter format_top_level O(n²) → single-pass. When alignment failed
for a group of N consecutive defines, the code formatted only the first and
re-scanned the remaining N-1 on the next iteration. Now formats the entire
group in one pass and advances past it.
Span::contains and Span::contains_pos consolidated. Three
near-identical span containment functions duplicated across sema-lsp/helpers.rs and sema-lsp/scope.rs are now methods on Span in sema-core, available to all crates.
Performance
builtin_index() cached with OnceLock. The ~11K-line JSON doc index
was deserialized from scratch on every ,apropos REPL command. Now parsed
once and cached; subsequent calls return a &'static DocIndex reference.
BuiltinDocs::load() shares entries via Rc<DocEntry>. Each DocEntry
(with full markdown body, params, examples) was cloned once per alias. Now
all names for the same entry share a single Rc<DocEntry>, eliminating
hundreds of string clones at LSP startup.
expand_query O(n²) → O(n) dedup. The synonym expansion in MCP docs_search used a linear scan per token for de-duplication. Now uses a HashSet.
Install sema-lang 1.28.0
Install prebuilt binaries via shell script
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/HelgeSverre/sema/releases/download/v1.28.0/sema-lang-installer.sh | sh
Scientific / exponential number literals. Floats can now be written as <mantissa>e<exponent> — 6.022e23, 1.0e19, 1e-9, -2.5E6 (e/E, optional
exponent sign, bare-integer mantissa allowed). Out-of-range magnitudes follow
IEEE-754 (1e400 → inf). An e/E not followed by (an optional sign and) digits
is left untouched, so identifiers like e and exp are never mis-parsed.
Fixed
Breakpoints fire inside async tasks — in both the native DAP (VS Code) and the WASM playground debuggers. A breakpoint on a line that runs only inside an async/async/spawn task (or via async/map/pool-map/async/all/channels) was
silently skipped because the scheduler ran every async task step in non-debug mode.
STOP + CONTINUE and step into/over/out now work at async breakpoints (verified e2e in
the playground). Known follow-ups: stepping across the scheduler into sibling tasks,
and variable-panel scoping to the paused task's frame at a cooperative stop.
stdlib semantic-correctness sweep. A pass over divergence bugs across the standard
library: string/foldcase and string-ci=? now use full Unicode case folding
("Straße" folds to "strasse", so caseless comparison matches "STRASSE") — distinct
from string/lower; plus correctness fixes in equality (eq), shell, date/time,
typed arrays, and text helpers, with matching doc updates.
Install sema-lang 1.27.1
Install prebuilt binaries via shell script
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/HelgeSverre/sema/releases/download/v1.27.1/sema-lang-installer.sh | sh
Concurrent I/O — blocking leaves now overlap on the scheduler. Previously async/spawn could interleave channels and sleeps, but any task that hit network or a
subprocess froze the single VM thread for the whole round-trip, so spawned I/O ran
serially. Now http/*, shell, llm/embed, and llm/complete / llm/classify / llm/extractyield to the scheduler while their work runs on a background runtime
(via a new cooperative AwaitIo yield), so spawning them as tasks makes wall-clock
approach max(latency) instead of sum(latency). Verified live: 4× concurrent llm/complete ~3.4× faster than serial; 4× llm/embed ~13.6×; 5× shell 514 ms vs
2571 ms. Top-level (non-async) calls are unchanged — byte-identical synchronous behavior.
async/pool-map — bounded-concurrency fan-out: (async/pool-map f items n) maps f
over items with at most n calls in flight, results in input order. Fan a large batch
(embeddings, fetches, completions) across a rate-limited resource without launching
everything at once.
async/map and async/spawn-all — ergonomic unbounded fan-out. (async/map f items)
is a concurrent map (a task per item, results in input order); (async/spawn-all thunks)
spawns a list of zero-arg thunks and awaits them all. Both are the obvious sugar over (async/all (map #(async/spawn …) …)); reach for async/pool-map when you need a cap.
Nested-trace propagation across async/spawn. Spans opened inside a spawned task now
nest under the spawning task's active span and share its trace, so (with-span … (async/map …)) (or any nested async) renders as ONE connected trace tree in
Jaeger/Phoenix/Langfuse instead of fragmenting into N disconnected single-span traces. A
top-level spawn (no active span) still starts its own trace, so independent top-level tasks
stay isolated.
True cancellation.async/cancel and async/timeout now abort in-flight I/O for
real where the runtime allows: a cancelled/timed-out http/* request tears down its
connection, and a shell subprocess is killed (SIGKILL) instead of running to
completion in the background. llm/* cancellation stays best-effort (the blocking worker
can't be interrupted mid-call; the result is discarded). async/timeout expiry now
cancels its target task automatically (you no longer need a paired async/cancel to free
its resources).
Per-task OpenTelemetry isolation. Concurrent LLM tasks each carry their own span
stack + conversation/session/user scope (swapped on every scheduler task-switch), so
overlapping llm/embed / llm/complete spans never cross-contaminate.
Fixed
Scheduler no longer reaps still-pending tasks at an outermost exit. A task spawned in
one top-level form and awaited in a later one (e.g. a streaming-pipeline collector spawned
before an (async/all …) of the other stages) was being cleared between scheduler runs
→ "async/await: still pending after scheduler run". The reap is now terminal-only.
Sema-native tracing API. New otel/* builtins + with-span / with-session macros
let your own Sema code emit first-class spans — not just the auto-instrumented llm/* / agent/* paths. with-span / otel/span wrap a block in a generic span; otel/set-attribute(s), otel/set-status, and otel/event annotate the innermost active
span; the typed helpers otel/llm-span + otel/llm-usage, otel/tool-span, and otel/retrieval-span render a user-built LLM/tool/retrieval step as an LLM/TOOL/RETRIEVER
span in Phoenix/Traceloop/Langfuse (via the SEMA_OTEL_COMPAT layer) with gen_ai.usage.*
accounting identical to the built-ins; with-session / otel/with-session groups
non-agent code into Langfuse Sessions/Users. Every form is a no-op when tracing is off and
never changes a program's return value. Docs: Tracing & Metrics → Adding your own
spans. Test: crates/sema/tests/otel_native_test.rs.
Install sema-lang 1.26.0
Install prebuilt binaries via shell script
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/HelgeSverre/sema/releases/download/v1.26.0/sema-lang-installer.sh | sh
llm/stream now applies resilience at stream-open. Streaming was bypassing the whole
dispatch layer; it now runs through rate-limiting and provider fallback before the
first token (a primary that fails to open the stream fails over to the next; once a token
is delivered, a mid-stream failure surfaces and keeps the partial — failing over would
re-emit it). Budgets can gate streams with llm/with-budget {... :on-stream :pre-gate}
(off by default). The cache and mid-stream retry still don't apply to streams (use cassettes for deterministic stream replay).
Verified live (OpenAI bad-model → fail over to Anthropic mid-llm/stream).
Per-call :timeout (ms) on llm/complete / llm/chat / llm/send. The option
now reaches the HTTP layer as a per-request reqwest timeout for the network providers
(Anthropic / OpenAI / Gemini) — previously parsed but ignored. (Local Ollama is excluded;
streaming calls aren't capped, since a wall-clock timeout would kill a long legitimate stream.)
Fixed
OpenAI streaming dropped tool calls.stream_complete returned an empty tool_calls,
discarding tool-call deltas — streaming agents on OpenAI were broken. It now accumulates the
index-keyed id / function.name / function.arguments fragments and assembles them into
the final response (verified live against the OpenAI API).
Gemini silent empty output. A thinking model with a small :max-tokens could spend the
whole budget reasoning and return an empty string with finishReason: MAX_TOKENS (exit 0,
no signal). It now raises an actionable error telling you to raise :max-tokens / lower :reasoning-effort (verified live; normal calls unaffected).
Install sema-lang 1.25.0
Install prebuilt binaries via shell script
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/HelgeSverre/sema/releases/download/v1.25.0/sema-lang-installer.sh | sh
Stdlib ergonomics — routine text/list/number helpers. Added seven builtins that
everyday code kept hand-rolling: math/round-to (round to N decimals) and math/format-fixed (fixed-decimal display string); string/lines (split on line
endings, Clojure split-lines semantics); list/contains? (boolean membership, vs member's Scheme tail), list/nth-or (safe indexed access with a default), and list/take-last / list/drop-last (tail counterparts to take/drop). The RAG
example now leans entirely on stdlib (list/chunk, flat-map, string/take, math/round-to) instead of local helpers.
Changed
Eval test macros emit one test per case. With the tree-walker retired, eval_str
and eval_str_compiled are the same path, so eval_tests! / eval_error_tests! no
longer generate redundant _tw + _vm pairs — halving the eval test count with no
loss of coverage.
Reranking + a full RAG pipeline. New llm/rerank cross-encoder reranking over
Cohere / Jina / Voyage (the same key already used for embeddings) — (llm/rerank query documents {:top-k 5 :model "..." :provider :cohere}) returns {:index :score :document}
maps, highest relevance first. This completes the retrieve-many → rerank-to-a-few RAG
recipe with llm/embed + vector-store/* + llm/complete. The vector-store search and
rerank steps emit OpenInference RETRIEVER / RERANKER spans (retrieval.documents.*, reranker.*) so a full RAG trace renders natively in Phoenix/Arize. New worked example examples/llm/rag-docs-search.sema (indexes Sema's own docs; make rag-demo), a RAG guide, and a FakeProvider regression test.
OpenTelemetry tags, metadata & streaming time-to-first-token (compat layer). With a SEMA_OTEL_COMPAT mode on, every LLM span is now auto-tagged with operation:/provider:/model: (+ cache-hit), and you can pass :tags (a list) and :metadata (a map) to llm/complete, llm/chat, llm/stream, and agent/run — tags
merge with the auto-tags, metadata fans out to each backend's native field
(langfuse.trace.metadata.*, langsmith.metadata.*, traceloop.association.properties.*, braintrust.metadata). Streamed calls record time-to-first-token
(sema.gen_ai.server.time_to_first_token always-on; Langfuse completion_start_time +
Traceloop gen_ai.is_streaming under compat) — a signal almost no other emitter
reports. LangSmith now also gets its own langsmith.trace.session_id, and
Langfuse a langfuse.release from SEMA_OTEL_RELEASE. Verified end-to-end against a live
OTel Collector (HTTP + gRPC) and Jaeger with real provider calls; regression test in crates/sema/tests/otel_tags_test.rs.
OpenTelemetry per-direction cost split & embedding detail (OpenInference compat). LLM
spans now also carry llm.cost.prompt / llm.cost.completion next to llm.cost.total
(so Phoenix/Arize show the prompt-vs-completion cost breakdown), and embeddings spans
carry embedding.model_name plus (content-gated, capped) embedding.embeddings.{i}.embedding.text.
Documentation
Website docs audit + reference coverage. Documented the nine new builtins
(math/round-to, math/format-fixed, string/lines, list/contains?, list/nth-or, list/take-last, list/drop-last, io/read-line, io/eof?) on their stdlib pages,
added the OTel cost-split/embedding-detail attributes to the compat doc, and gave the
RAG/rerank guide a depth pass (score semantics, top-k, cost/scaling, error handling,
observability). Fixed copy-paste examples that didn't run: shell returns a map
(:stdout/:stderr/:exit-code), the web-server demo's streaming/summarize/extract
handlers (llm/stream, llm/complete, and llm/extract's schema-first argument order).
Install sema-lang 1.24.0
Install prebuilt binaries via shell script
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/HelgeSverre/sema/releases/download/v1.24.0/sema-lang-installer.sh | sh
LLM cassettes — record/replay for deterministic, keyless testing & demos. A new sema-llm::cassette layer records real LLM responses to an NDJSON tape once, then
replays them deterministically forever — no API key, no network. llm/complete, llm/chat, llm/extract, agent loops (agent/run, each turn keyed
independently), streaming (llm/stream, the chunk sequence is recorded and
replayed in order) and embeddings (llm/embed, vectors recorded and replayed)
are all covered. Modes :auto / :replay / :record; a :replay miss is a hard error that surfaces
prompt drift. Surface: (llm/with-cassette path opts thunk), llm/cassette-load/-save/-eject, and SEMA_LLM_CASSETTE / SEMA_LLM_CASSETTE_MODE for CI. Folds with the rest of the runtime: it sits below
the OpenTelemetry span + response cache + cost accounting and above the provider, so
a replay still emits its chat span and reports its recorded usage (distinct
from a cache hit's zero usage), and with-cassette disables the response cache for
its scope. The tape stores only the response keyed by a request hash — no prompt
text, key, or header touches disk (redaction by construction). Docs: website/docs/llm/cassettes.md.
CI
Publish-list guard.scripts/check-publish-list.sh (run in the release verify gate + make check-publish-list) fails if a publishable workspace crate is
missing from publish.yml's order — preventing the half-published release that hit
1.22.0 when the new sema-otel crate wasn't in the list.
The two publishes now share a single verify gate (was one full suite per
registry), CI uses Swatinem/rust-cache, and the per-crate publish sleeps are
trimmed (cargo already waits for index propagation).
Install sema-lang 1.23.0
Install prebuilt binaries via shell script
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/HelgeSverre/sema/releases/download/v1.23.0/sema-lang-installer.sh | sh
Special-form names are bindable again. A change in 1.20.4 reserved special-form names (if, fn, let, and, message, …) and rejected binding any of them — but that also rejected correct value-position use, like a function parameter named message or a variable named fn, because the scope-free lowerer can't distinguish value use from operator use. It broke real code (5 bundled examples) and slipped a CI regression past four releases. The reservation is removed: special-form names shadow correctly in value position again. In operator/head position the special form still wins (a documented, accepted footgun — docs/limitations.md#36); the proper fix (full lexical shadowing) is future work.
CI / release process
Publishing now requires the test suite to pass. The crates.io and npm publish workflows triggered on a version tag with no dependency on CI, so a red test suite never blocked a release — which is how the reserved-name regression shipped. They now gate on a reusable verify workflow that runs the full CI-equivalent suite (fmt, clippy, doc check, cargo test, example + bytecode smoke tests) before any publish. The local release runbook was updated to run the example/bytecode smoke tests too (plain cargo test skips them).
Install sema-lang 1.21.2
Install prebuilt binaries via shell script
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/HelgeSverre/sema/releases/download/v1.21.2/sema-lang-installer.sh | sh
Bugfix. Found by a live stress test of the resilience features (Ollama-down →
Mistral fallback, caching, and budgets).
Fixed
Cache hits double-charged cost and burned budget. A cached response (llm/with-cache) makes no provider call — no tokens are consumed and no money is spent — but its stored token usage was still run through track_usage, so llm/session-usage cost and llm/with-budget spend incremented on every cache hit (e.g. a repeated call reported 2× the cost, and cache hits could trip a budget). Cache hits now report zero usage, so cost/budget reflect actual spend. Verified live (cost unchanged across identical calls; a budget that allows one real call is not tripped by subsequent cache hits) and with a deterministic regression test. Provider fallback and budget enforcement were verified correct and unchanged.
Install sema-lang 1.21.1
Install prebuilt binaries via shell script
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/HelgeSverre/sema/releases/download/v1.21.1/sema-lang-installer.sh | sh
