chore(terminal): bundle static terminal.so into release artifacts#2205
Merged
theangelperalta merged 3 commits intoJun 3, 2026
Merged
Conversation
…urce Removes the three prebuilt terminal.so shared objects from version control and replaces them with a from-source build path, addressing lem-project#1964 ("Open source software should, by definition, not be distributing binary files"). The build script and `make terminal-lib` target landed in lem-project#2182; this wires them into the editor build and removes the binaries: - git rm the linux/arm64, linux/x64 and macosx/arm64 terminal.so files. - Invoke `terminal-lib` best-effort (`-$(MAKE) terminal-lib`) from the ncurses, sdl2, sdl2-ncurses, webview and webview-ncurses targets so source installs still get a terminal. A missing libvterm/compiler is non-fatal: lem-terminal/ffi.lisp already wraps use-foreign-library in ignore-errors and silently disables itself when the library is absent. - gitignore extensions/terminal/lib/**/*.so so locally-built helpers are never re-committed. - Convert .github/workflows/build-terminal-shared-object.yml from a workflow that recompiled and re-committed terminal.so on every push (which would reintroduce exactly the binaries this removes) into a verify-only CI check that builds terminal.so from source on Linux and macOS and fails if the build breaks. Refs lem-project#1964, lem-project#2060, lem-project#2182.
build-terminal.lisp dropped into the SBCL REPL after a successful build (exiting only on stdin EOF) and trusted the compiler's exit code without checking that terminal.so was actually written. - Quit 0 after a successful build so `sbcl --load` never blocks on stdin. - Delete any stale .so before compiling, then assert the artifact exists afterwards, so a compiler that exits 0 without producing the file is treated as a failure instead of a silent no-op. - Factor the failure path into build-failed (stderr + exit 1).
Release bundles (Linux AppImage, macOS zip) never shipped or loaded terminal.so: ffi.lisp resolves it from the build machine's source tree via asdf:system-relative-pathname, and the macOS deploy library only bundles a foreign library that is loaded at build time -- which failed because the CI runner lacked libvterm. Even when bundled, a dynamic terminal.so referenced Homebrew's libvterm by absolute path and would break on a clean machine. Build a self-contained terminal.so with libvterm statically linked for releases, so the deploy library (macOS) and LD_LIBRARY_PATH=usr/lib (AppImage) can load it with no external libvterm dependency. - build-terminal.lisp: add LEM_TERMINAL_STATIC mode. macOS links libvterm.a directly (prefix via LIBVTERM_PREFIX / brew --prefix); Linux wraps -lvterm in -Wl,-Bstatic. Default stays dynamic for source installs. - macos-deploy.bash + nightly-builds.yml: install libvterm and build the static helper before asdf:make so deploy bundles it; assert it landed in the .app. - Dockerfile-AppImage: install libvterm-dev and build the static helper before asdf:make; make_appdir.sh asserts terminal.so reached usr/lib. No ffi.lisp/core changes: deploy + LD_LIBRARY_PATH handle runtime resolution once a self-contained .so is bundled. Refs lem-project#1964, lem-project#2060.
Contributor
|
✅ Code Contractor Validation: PASSED 📋 Contract Configuration: contract (Source: Repository)version: 2
trigger:
paths:
- "extensions/**"
- "frontends/**/*.lisp"
- "src/**"
- "tests/**"
- "contrib/**"
- "**/*.asd"
head_branches:
exclude:
- 'revert-*'
validation:
limits:
max_total_changed_lines: 400
max_delete_ratio: 0.5
max_files_changed: 10
severity: warning
ai:
system_prompt: |
You are a senior Common Lisp engineer reviewing code for Lem editor.
Lem is a text editor with multiple frontends (ncurses, SDL2, webview).
Focus on maintainability, consistency with existing code, and Lem-specific conventions.
rules:
# === File Structure ===
- name: defpackage_rule
prompt: |
First form must be `defpackage` or `uiop:define-package`.
Package name should match filename (e.g., `foo.lisp` → `:lem-ext/foo` or `:lem-foo`).
Extensions must use `lem-` prefix (e.g., `:lem-python-mode`).
- name: file_structure_rule
prompt: |
File organization (top to bottom):
1. defpackage
2. defvar/defparameter declarations
3. Key bindings (define-key, define-keys)
4. Class/struct definitions
5. Functions and commands
# === Style ===
- name: loop_keywords_rule
prompt: |
Loop keywords must use colons: `(loop :for x :in list :do ...)`
NOT: `(loop for x in list do ...)`
- name: naming_conventions_rule
prompt: |
Naming conventions:
- Functions/variables: kebab-case (e.g., `find-buffer`)
- Special variables: *earmuffs* (e.g., `*global-keymap*`)
- Constants: +plus-signs+ (e.g., `+default-tab-size+`)
- Predicates: -p suffix for functions (e.g., `buffer-modified-p`)
- Do NOT use -p suffix for user-configurable variables
# === Documentation ===
- name: docstring_rule
prompt: |
Required docstrings for:
- Exported functions, methods, classes
- `define-command` (explain what the command does)
- Generic functions (`:documentation` option)
Important functions should explain "why", not just "what".
severity: warning
# === Lem-Specific ===
- name: internal_symbol_rule
prompt: |
Use exported symbols from `lem` or `lem-core` package.
Avoid `lem::internal-symbol` access.
If internal access is necessary, document why.
- name: error_handling_rule
prompt: |
- `error`: Internal/programming errors
- `editor-error`: User-facing errors (displayed in echo area)
Always use `editor-error` for messages shown to users.
- name: frontend_interface_rule
prompt: |
Frontend-specific code must use `lem-if:*` protocol.
Do not call frontend implementation directly from core.
severity: warning
# === Functional Style ===
- name: functional_style_rule
prompt: |
Prefer explicit function arguments over dynamic variables.
Avoid using `defvar` for state passed between functions.
Exception: Well-documented cases like `*current-buffer*`.
- name: dynamic_symbol_call_rule
prompt: |
Avoid `uiop:symbol-call`. Rethink architecture instead.
If unavoidable, document the reason.
# === Libraries ===
- name: alexandria_usage_rule
prompt: |
Alexandria utilities allowed: `if-let`, `when-let`, `with-gensyms`, etc.
Avoid: `alexandria:curry` (use explicit lambdas)
Avoid: `alexandria-2:*` functions not yet used in codebase
# === Macros ===
- name: macro_style_rule
prompt: |
Keep macros small. For complex logic, use `call-with-*` pattern:
```lisp
(defmacro with-foo (() &body body)
`(call-with-foo (lambda () ,@body)))
```
Prefer `list` over backquote outside macros.📚 About Code ContractorDeclarative Code Standards That Learn and Improve Define domain-specific validation rules in YAML. Want this for your repo? |
theangelperalta
added a commit
that referenced
this pull request
Jun 3, 2026
PRs #2204 and #2205 merged in the reverse of their stacking order (#2205, which contained #2204's commits, merged first; #2204 merged second). Because #2205 had added static-p/vterm-flags just above build-failed, the squash of #2204 re-applied the build-failed block instead of recognizing it as already present, leaving two identical definitions in build-terminal.lisp. Remove the duplicate. No behavior change (the copies were identical); this just silences the SBCL redefinition warning.
This was referenced Jun 3, 2026
theangelperalta
added a commit
that referenced
this pull request
Jun 3, 2026
* feat(nix): build and bundle terminal.so in the flake The Nix flake never built the lem-terminal native helper: it has no libvterm input and no terminal.c build step, so on Nix the terminal extension silently disabled itself (ffi.lisp's use-foreign-library is wrapped in ignore-errors). Removing the committed binaries in #2204 did not regress this -- the prebuilt .so was dynamically linked against a libvterm that was never present in the Nix sandbox -- but it also left Nix without a working terminal. Add a terminal-so derivation that compiles extensions/terminal/terminal.c against pkgs.libvterm, mirroring the existing ts-wrapper / c-webview C derivations, and add it to the nativeLibs of lem-ncurses, lem-sdl2 and lem-webview. Dynamic linking is the idiomatic choice on Nix (unlike the AppImage/macOS bundles, which static-link in #2205): the stdenv records an RPATH to the pinned libvterm store path, so terminal.so resolves libvterm at runtime with no bundling or relinking, and nativeLibs puts terminal.so itself on the library path that ffi.lisp's "terminal.so" lookup uses at build and run time. * fix(nix): gate terminal-so on Linux (libvterm is Linux-only in nixpkgs) nixpkgs' libvterm (0.99.7) has meta.platforms = Linux only, so referencing pkgs.libvterm unconditionally made `nix flake check --all-systems` fail at evaluation on the aarch64-darwin / x86_64-darwin systems, breaking every build job. Add terminal-so to nativeLibs only when stdenv.isLinux (via lib.optionals, which doesn't force its list when the condition is false). Terminal works on Linux Nix builds; Darwin Nix is unchanged (no terminal, as before, since nixpkgs doesn't package libvterm for Darwin). * fix(nix): add glib cflags for terminal.so (nixpkgs libvterm is the neovim fork) nixpkgs' libvterm (0.99.7) is the Neovim fork, whose vterm.h #includes <glib.h>. terminal.c includes vterm.h, so the build failed with "fatal error: glib.h: No such file or directory". Add pkg-config + glib and pull GLib's compile/link flags via `pkg-config --cflags --libs glib-2.0`. * fix(nix): use libvterm-neovim, not libvterm (wrong library) The compile failed on glib.h then curses.h from libvterm-0.99.7's vterm.h because pkgs.libvterm is the old, abandoned glib/curses-based "libvterm" with an API incompatible with terminal.c. Leonerd's modern libvterm -- the one terminal.c targets -- is packaged as pkgs.libvterm-neovim. Switch to pkgs.libvterm-neovim and drop the glib/pkg-config workaround.
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes the terminal extension actually work from a downloaded AppImage and macOS zip by shipping a self-contained, statically-linked
terminal.soin both release bundles.Why
Release bundles never shipped or loaded
terminal.so:ffi.lispresolves the lib viaasdf:system-relative-pathname— the build machine's source tree, absent on a user's machine.deploy, which only bundles a foreign library that is loaded at build time. The CI runner installssbcl tree-sitterbut not libvterm, so the dlopen failed (swallowed byignore-errors) anddeploynever recorded it.terminal.soreferenced/opt/homebrew/.../libvterm.0.dylibby absolute path (verified viaotool -L) anddeploydoes not follow the transitive libvterm dependency.Approach — static libvterm
Compile
terminal.sofor releases with libvterm statically linked, producing one self-contained.sowith no external libvterm dependency. Then bundling is trivial and identical in spirit on both platforms; noinstall_name_tool/patchelf/linuxdeploy --libraryrelinking. libvterm is MIT, so static linking is license-clean. The existingextensions/terminal/Dockerfile{,.musl}already document the Linux static recipe.Key mechanic: once
terminal.sois loaded at build time,deploybundles it next to the executable and reloads it by filename at boot — so a self-contained static.so"just works". Noffi.lisp/ core changes are needed.Changes
scripts/build-terminal.lisp—LEM_TERMINAL_STATICmode. macOS linkslibvterm.adirectly (prefix fromLIBVTERM_PREFIX, elsebrew --prefix libvterm); Linux wraps-lvtermin-Wl,-Bstatic. Default stays dynamic for source installs.scripts/macos-deploy.bash,.github/workflows/nightly-builds.yml) —brew install … libvterm; build the static helper beforeasdf:make :lem; assertterminal.solanded inLem.app.docker/Dockerfile-AppImage,docker/make_appdir.sh) —apt-get install … libvterm-dev; build the static helper beforeasdf:make;AppRunalready setsLD_LIBRARY_PATH=usr/lib;make_appdir.shnow assertsterminal.soreachedusr/lib.Test plan
LEM_TERMINAL_STATIC=1 make terminal-lib→otool -Lshows no libvterm dependency (self-contained, 93 KB). Defaultmake terminal-libstill dynamic-linkslibvterm.0.dylib.nightly-builds.yml(workflow_dispatch) builds both Linux + macOS green; the bundle assertions pass.libvterm-devprovideslibvterm.afor the-Wl,-Bstaticlink (fallback: build libvterm from source asextensions/terminal/Dockerfiledoes).*terminal*buffer, confirm a shell runs.make sdl2/ncurses/webview(dynamic, source install) unaffected.Refs #1964, #2060.