Skip to content

sema-lisp/sema

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

1,481 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Sema β€” Stop rewriting the agent loop.

A Lisp where LLM agents are language primitives, not an SDK β€”
compiled to a fast bytecode VM, shipped as a single binary.

Playground Docs Version License

Docs Β· Playground Β· For Agents Β· Examples Β· Issues

Stop rewriting the agent loop. Every LLM script grows the same scaffolding β€” retries, caching, cost caps, rate limits, tool dispatch, conversation state. Sema makes that scaffolding the runtime: your script stays the size of its idea, ships as a single binary, and your coding agent already speaks the language.

Sema is a Scheme-like Lisp where prompts are s-expressions, conversations are persistent data structures, and LLM calls are just another form of evaluation β€” with Clojure-style keywords (:foo), map literals ({:key val}), and vector literals ([1 2 3]).

What It Looks Like

A coding agent with file tools, safety checks, and budget tracking β€” in ~40 lines:

;; Define tools the LLM can call
(deftool read-file
  "Read a file's contents"
  {:path {:type :string :description "File path"}}
  (lambda (path)
    (if (file/exists? path) (file/read path) "File not found")))

(deftool edit-file
  "Replace text in a file"
  {:path {:type :string} :old {:type :string} :new {:type :string}}
  (lambda (path old new)
    (file/write path (string/replace (file/read path) old new))
    "Done"))

(deftool run-command
  "Run a shell command"
  {:command {:type :string :description "Shell command to run"}}
  (lambda (command) (:stdout (shell "sh" "-c" command))))

;; Create an agent with tools, system prompt, and spending limit
(defagent coder
  {:system (format "You are a coding assistant. Working directory: ~a" (sys/cwd))
   :tools [read-file edit-file run-command]
   :max-turns 20})   ; no :model β†’ uses the configured default provider

;; Run it β€” budget is scoped, automatically restored after the block
(llm/with-budget {:max-cost-usd 0.50} (lambda ()
  (define result (agent/run coder "Add error handling to src/main.rs"))
  (println (:response result))
  (println (format "Cost: $~a" (:spent (llm/budget-remaining))))))

Key Features

;; Simple completion
(llm/complete "Explain monads in one sentence")

;; Structured data extraction β€” returns a map, not a string
(llm/extract
  {:vendor {:type :string} :amount {:type :number} :date {:type :string}}
  "Bought coffee for $4.50 at Blue Bottle on Jan 15")
;; => {:amount 4.5 :date "2025-01-15" :vendor "Blue Bottle"}

;; Classification
(llm/classify (list :positive :negative :neutral) "This product is amazing!")
;; => :positive

;; Multi-turn conversations as immutable data
(define conv (conversation/new {:model "claude-haiku-4-5-20251001"}))
(define conv (conversation/say conv "The secret number is 7"))
(define conv (conversation/say conv "What's the secret number?"))
(conversation/last-reply conv) ;; => "The secret number is 7."

;; Streaming
(llm/stream "Tell me a story" {:max-tokens 500})

;; Batch β€” all prompts sent concurrently
(llm/batch (list "Translate 'hello' to French"
                 "Translate 'hello' to Spanish"
                 "Translate 'hello' to German"))

;; Vision β€” extract structured data from images
(llm/extract-from-image
  {:text :string :background_color :string}
  "assets/logo.png")
;; => {:background_color "white" :text "Sema"}

;; Multi-modal chat β€” send images in messages
(define img (file/read-bytes "photo.jpg"))
(llm/chat (list (message/with-image :user "Describe this image." img)))

;; Cost tracking
(llm/set-budget 1.00)
(llm/budget-remaining) ;; => {:limit 1.0 :spent 0.05 :remaining 0.95}

;; Response caching β€” avoid duplicate API calls during development
(llm/with-cache (lambda ()
  (llm/complete "Explain monads")))

;; Cassettes β€” record real responses once, replay them in CI (no keys, no network)
(llm/with-cassette "fixtures/run.jsonl" {:mode :auto} (lambda ()
  (llm/complete "Explain monads")))

;; Fallback chains β€” automatic provider failover
(llm/with-fallback [:anthropic :openai :groq]
  (lambda () (llm/complete "Hello")))

;; In-memory vector store for semantic search (RAG)
(vector-store/create "docs")
(vector-store/add "docs" "id" (llm/embed "text") {:source "file.txt"})
(vector-store/search "docs" (llm/embed "query") 5)

;; Cross-encoder reranking β€” the retrieve-many β†’ rerank-to-a-few RAG move
(llm/rerank "how do I read a file?"
            ["file/read returns a string" "http/get fetches a URL"]
            {:top-k 3})
;; => ({:index 0 :score 0.98 :document "file/read returns a string"} ...)

;; Text chunking for LLM pipelines
(text/chunk long-document {:size 500 :overlap 100})

;; Prompt templates
(prompt/render "Hello {{name}}" {:name "Alice"})
; => "Hello Alice"

;; Persistent key-value store
(kv/open "cache" "cache.json")
(kv/set "cache" "key" {:data "value"})
(kv/get "cache" "key")

Supported Providers

All providers are auto-configured from environment variables β€” just set the API key and go.

Provider Chat Stream Tools Embeddings Vision
Anthropic βœ… βœ… βœ… β€” βœ…
OpenAI βœ… βœ… βœ… βœ… βœ…
Google Gemini βœ… βœ… βœ… β€” βœ…
Ollama βœ… βœ… βœ… β€” βœ…
Groq βœ… βœ… βœ… β€” β€”
xAI βœ… βœ… βœ… β€” β€”
Mistral βœ… βœ… βœ… β€” β€”
Moonshot βœ… βœ… βœ… β€” β€”
Jina β€” β€” β€” βœ… β€”
Voyage β€” β€” β€” βœ… β€”
Cohere β€” β€” β€” βœ… β€”
Any OpenAI-compat βœ… βœ… βœ… β€” βœ…
Custom (Lisp) βœ… β€” βœ… β€” β€”

It's Also a Real Lisp

Hundreds of built-in functions, tail-call optimization, macros, modules, error handling β€” not a toy.

;; Closures, higher-order functions, TCO
(define (fibonacci n)
  (let loop ((i 0) (a 0) (b 1))
    (if (= i n) a (loop (+ i 1) b (+ a b)))))
(fibonacci 50) ;; => 12586269025

;; Maps, keywords-as-functions, f-strings
(define person {:name "Ada" :age 36 :langs ["Lisp" "Rust"]})
(:name person) ;; => "Ada"
(println f"${(:name person)} knows ${(length (:langs person))} languages")

;; Destructuring
(let (({:keys [name age]} person))
  (println f"${name} is ${age}"))

;; Pattern matching with guards
(define (classify n)
  (match n
    (x when (> x 100) "big")
    (x when (> x 0)   "small")
    (_                 "non-positive")))

;; Functional pipelines
(->> (range 1 100)
     (filter even?)
     (map #(* % %))
     (take 5))
;; => (4 16 36 64 100)

;; Nested data access
(define config {:db {:host "localhost" :port 5432}})
(get-in config [:db :host])  ;; => "localhost"

;; Macros
(defmacro unless (test . body)
  `(if ,test nil (begin ,@body)))

;; Modules
(module utils (export square)
  (define (square x) (* x x)))

;; HTTP, JSON, regex, file I/O, crypto, CSV, datetime...
(define data (json/decode (http/get "https://api.example.com/data")))

πŸ“– Full language reference, stdlib docs, and more examples at sema-lang.com/docs

Try It Now

sema.run β€” Browser-based playground with 20+ example programs. No install required. Runs entirely in WebAssembly.

Teach Your Coding Agent Sema in One Line

Sema is new, so your agent hasn't seen it. Fix that in one command β€” append the agent crib sheet to your repo's AGENTS.md (and point CLAUDE.md at it):

curl -fsSL https://sema-lang.com/docs/for-agents.md >> AGENTS.md
ln -s AGENTS.md CLAUDE.md     # Claude Code, Cursor, etc. read this

for-agents.md is a single terse page β€” everything that differs from other Lisps β€” written for an LLM that already knows a Lisp. It links to /llms.txt, a machine index of every doc page so the agent can fetch just the page it needs (e.g. /docs/llm/tools-agents.md) on demand, instead of loading the whole manual. Every doc URL also serves raw Markdown β€” append .md to any sema-lang.com/docs/... link and your agent gets the source, not HTML.

Installation

Install pre-built binaries (no Rust required):

# macOS / Linux
curl -fsSL https://sema-lang.com/install.sh | sh

# Windows (PowerShell)
powershell -ExecutionPolicy ByPass -c "irm https://github.com/sema-lisp/sema/releases/latest/download/sema-lang-installer.ps1 | iex"

# Homebrew (macOS / Linux)
brew install helgesverre/tap/sema-lang

Or install from crates.io:

cargo install sema-lang

Or build from source:

git clone https://github.com/sema-lisp/sema
cd sema && cargo build --release
# Binary at target/release/sema
sema                          # REPL (with tab completion)
sema script.sema              # Run a file
sema -e '(+ 1 2)'             # Evaluate expression
sema --no-llm script.sema     # Run without LLM (faster startup)
sema build app.sema -o myapp  # Build standalone executable
./myapp                       # Run without sema installed

Shell Completions

Generate tab-completion scripts for your shell:

# Zsh (macOS / Linux)
mkdir -p ~/.zsh/completions
sema completions zsh > ~/.zsh/completions/_sema

# Bash
mkdir -p ~/.local/share/bash-completion/completions
sema completions bash > ~/.local/share/bash-completion/completions/sema

# Fish
sema completions fish > ~/.config/fish/completions/sema.fish

πŸ“– Full setup instructions for all shells: sema-lang.com/docs/shell-completions

πŸ“– Full CLI reference, flags, and REPL commands: sema-lang.com/docs/cli

Editor Support

Each editor plugin lives in its own repo under the sema-lisp org:

Editor Repository Install
VS Code vscode-sema ext install helgesverre.sema
Zed zed-sema Extensions β†’ search Sema
IntelliJ intellij-sema JetBrains Marketplace β†’ Sema
Neovim sema.nvim { "sema-lisp/sema.nvim" }
Vim sema.vim Plug 'sema-lisp/sema.vim'
Emacs emacs-sema MELPA β†’ sema-mode
Helix helix-sema clone + ./install.sh
Sublime Text sublime-sema Package Control β†’ Sema

All plugins provide syntax highlighting; VS Code, Zed, IntelliJ, Neovim, Emacs, Helix, and Sublime also wire up the built-in language server (sema lsp), and several (VS Code, Zed, IntelliJ, Neovim, Helix) add debugging (sema dap) β€” some also register the MCP server (sema mcp). Zed, Helix, and Neovim highlight via the shared tree-sitter-sema grammar; the others ship their own.

πŸ“– Full installation instructions and per-editor feature lists: sema-lang.com/docs/editors

Notebook

Sema includes a Jupyter-inspired notebook interface with a browser UI:

sema notebook new my-notebook.sema-nb        # Create a notebook
sema notebook serve my-notebook.sema-nb      # Open in browser (localhost:8888)
sema notebook run my-notebook.sema-nb        # Run all cells headlessly
sema notebook export my-notebook.sema-nb     # Export to Markdown

Cells share a persistent environment β€” definitions in earlier cells are visible in later ones. Notebooks are saved as .sema-nb JSON files.

πŸ“– Full notebook documentation: sema-lang.com/docs/notebook

Language Tooling

A full toolchain ships in the box β€” no plugins to assemble:

sema fmt script.sema     # Canonical code formatter
sema lsp                 # Language Server (completions, hover, go-to-def, rename)
sema dap                 # Debug Adapter (breakpoints, stepping, variable inspection)
sema mcp                 # Model Context Protocol server for LLM clients

The MCP server lets LLM clients (Claude Desktop, Cursor, Claude Code) compile, format, evaluate, and build Sema code β€” and call your own deftool Lisp tools β€” directly in your environment.

πŸ“– Formatter Β· LSP Β· Debugger Β· MCP

Example Programs

The examples/ directory has 50+ programs:

Example What it does
coding-agent.sema Full coding agent with file editing, search, and shell tools
review.sema AI code reviewer for git diffs
commit-msg.sema Generate conventional commit messages from staged changes
summarize.sema Summarize files or piped input
game-of-life.sema Conway's Game of Life
brainfuck.sema Brainfuck interpreter
mandelbrot.sema ASCII Mandelbrot set
json-api.sema Fetch and process JSON APIs
test-vision.sema Vision extraction and multi-modal chat tests
test-extract.sema Structured extraction and classification
test-batch.sema Batch/parallel LLM completions
test-pipeline.sema Caching, budgets, rate limiting, retry, fallback chains
test-text-tools.sema Text chunking, prompt templates, document abstraction
test-vector-store.sema In-memory vector store with similarity search
test-kv-store.sema Persistent JSON-backed key-value store
expr-evaluator.sema Mini calculator using match on tagged vectors
shape-geometry.sema Shape areas/perimeters with map pattern matching
http-router.sema HTTP router with match on nested maps and guards
destructuring.sema Comprehensive destructuring showcase (vector, map, lambda)
demo.sema-nb Interactive notebook demo (run with sema notebook serve)

Why Sema?

The pitch in one line: no LangChain, no provider SDK, no agent framework, no glue script β€” the agent loop, retries, caching, budgets, tracing, and tool dispatch are the language runtime, and the whole thing is one binary you can scp to a box.

  • LLMs as language primitives β€” prompts, messages, conversations, tools, and agents are first-class data types, not string templates bolted on
  • Multi-provider β€” swap between Anthropic, OpenAI, Gemini, Ollama, any OpenAI-compatible endpoint, or define your own provider in Sema
  • Pipeline-ready β€” response caching, fallback chains, rate limiting, retry with backoff, text chunking, prompt templates, vector store, and a persistent KV store
  • Cost-aware β€” built-in budget tracking with a bundled pricing snapshot (models.dev), updated per release
  • Observable β€” every LLM/agent run is auto-traced with OpenTelemetry (GenAI semantic conventions): tokens, cost, latency, and the full invoke_agent β†’ chat β†’ execute_tool tree, exportable to Jaeger, Grafana, Datadog, Langfuse, Arize Phoenix, and more β€” zero manual instrumentation, off by default
  • Practical Lisp β€” closures, TCO, macros, modules, error handling, HTTP, file I/O, regex, JSON, and a comprehensive stdlib
  • Standalone executables β€” sema build compiles programs into self-contained binaries with auto-traced imports and bundled assets
  • Embeddable β€” a Rust crate with a builder API, or @sema-lang/sema to run Sema client-side in JS via WebAssembly
  • Full toolchain β€” formatter, language server (LSP), debugger (DAP), and an MCP server for LLM clients, all built in
  • Developer-friendly β€” REPL with tab completion, structured error messages with hints, and 50+ example programs

Why Not Sema?

  • No full numeric tower (rationals, bignums, complex numbers)
  • No continuations (call/cc) or fully hygienic macros (syntax-rules) β€” has auto-gensym (foo#) for preventing variable capture
  • Single-threaded β€” Rc-based, no cross-thread sharing of values
  • No JIT β€” bytecode compiler + stack-based VM, no native code generation
  • Package manager is git-based β€” central registry not yet live
  • Young language β€” solid but not battle-tested at scale

Architecture

crates/
  sema-core/     NaN-boxed Value type, errors, environment
  sema-reader/   Lexer and s-expression parser
  sema-vm/       Bytecode compiler and virtual machine
  sema-eval/     Trampoline-based evaluator, special forms, modules
  sema-stdlib/   Built-in functions across many modules
  sema-llm/      LLM provider trait + multi-provider clients
  sema-otel/     OpenTelemetry tracing (GenAI semantic conventions)
  sema-docs/     Canonical builtin docs (powers LSP hover + REPL apropos)
  sema-lsp/      Language Server Protocol implementation
  sema-dap/      Debug Adapter Protocol server
  sema-fmt/      Source code formatter
  sema-mcp/      Model Context Protocol server
  sema-notebook/ Jupyter-inspired notebook interface with browser UI
  sema-wasm/     WebAssembly build for sema.run playground
  sema/          CLI binary: REPL + file runner + standalone builder

πŸ”¬ Deep-dive into the internals: Architecture Β· Evaluator Β· Lisp Comparison

License

MIT β€” see LICENSE.

About

A Lisp with first-class LLM primitives, implemented in Rust

Topics

Resources

License

Code of conduct

Contributing

Security policy

Stars

20 stars

Watchers

1 watching

Forks

Contributors