Skip to content

chore(terminal): remove committed terminal.so binaries, build from source#2204

Merged
theangelperalta merged 2 commits into
lem-project:mainfrom
theangelperalta:chore/terminal-remove-committed-binaries
Jun 3, 2026
Merged

chore(terminal): remove committed terminal.so binaries, build from source#2204
theangelperalta merged 2 commits into
lem-project:mainfrom
theangelperalta:chore/terminal-remove-committed-binaries

Conversation

@theangelperalta
Copy link
Copy Markdown
Collaborator

Summary

Removes the three prebuilt terminal.so shared objects from version control and replaces them with a from-source build path. This is the follow-up to #2182 that finally addresses #1964"Open source software should, by definition, not be distributing binary files. Given the xz backdoor, these increase distrust."

#2182 added scripts/build-terminal.lisp and the make terminal-lib target but deliberately kept the binaries. This PR removes them and wires the build in.

Changes

  • Remove binariesgit rm of extensions/terminal/lib/{linux/arm64,linux/x64,macosx/arm64}/terminal.so.
  • Build on source installs — the ncurses, sdl2, sdl2-ncurses, webview and webview-ncurses targets now invoke terminal-lib best-effort via -$(MAKE) terminal-lib. A missing libvterm or C toolchain is non-fatal: lem-terminal/ffi.lisp already wraps cffi:use-foreign-library in ignore-errors and silently disables the terminal when the library is absent. Normal builds gain no hard libvterm dependency.
  • gitignore extensions/terminal/lib/**/*.so so locally-built helpers are never re-committed.
  • Convert the re-committing workflow.github/workflows/build-terminal-shared-object.yml previously recompiled terminal.so and opened a PR to commit it back on every push to main, which would reintroduce exactly the binaries this removes. It's now a verify-only CI check that builds terminal.so from source on both Linux (apt libvterm) and macOS (brew libvterm) and fails if the build breaks — no commits, no artifacts.

Why this scope / risk

  • lem-terminal is loaded best-effort, so removing the binaries cannot break make sdl2/ncurses/webview even where libvterm is unavailable.
  • Release bundles (AppImage, macOS zip) do not currently ship or load terminal.soffi.lisp resolves it from the source-tree path via asdf:system-relative-pathname, so the committed binaries only ever served source/checkout installs. Bundling the terminal into release artifacts is a separate concern, left out of scope.

Test plan

  • make terminal-lib on macOS arm64 (Homebrew libvterm) rebuilds extensions/terminal/lib/macosx/arm64/terminal.so (byte-identical size, 54232, to the removed binary).
  • The rebuilt .so is gitignored (absent from git status).
  • CI: make terminal-lib builds on Linux x64 (apt libvterm-dev) — verified by the new workflow.
  • make sdl2 / ncurses / webview still build on a host without libvterm (terminal silently disabled).
  • Open a *terminal* buffer from a fresh source build to confirm the freshly-built .so loads.

Refs #1964, #2060, #2182.

…urce

Removes the three prebuilt terminal.so shared objects from version control
and replaces them with a from-source build path, addressing lem-project#1964 ("Open
source software should, by definition, not be distributing binary files").

The build script and `make terminal-lib` target landed in lem-project#2182; this wires
them into the editor build and removes the binaries:

- git rm the linux/arm64, linux/x64 and macosx/arm64 terminal.so files.
- Invoke `terminal-lib` best-effort (`-$(MAKE) terminal-lib`) from the
  ncurses, sdl2, sdl2-ncurses, webview and webview-ncurses targets so source
  installs still get a terminal. A missing libvterm/compiler is non-fatal:
  lem-terminal/ffi.lisp already wraps use-foreign-library in ignore-errors
  and silently disables itself when the library is absent.
- gitignore extensions/terminal/lib/**/*.so so locally-built helpers are
  never re-committed.
- Convert .github/workflows/build-terminal-shared-object.yml from a workflow
  that recompiled and re-committed terminal.so on every push (which would
  reintroduce exactly the binaries this removes) into a verify-only CI check
  that builds terminal.so from source on Linux and macOS and fails if the
  build breaks.

Refs lem-project#1964, lem-project#2060, lem-project#2182.
@code-contractor-app
Copy link
Copy Markdown
Contributor

code-contractor-app Bot commented Jun 3, 2026

✅ Code Contractor Validation: PASSED

=== Contract: contract ===

✓ Code Contractor Validation Result
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

📋 Contract Source: Repository

📊 Statistics:
  Files Changed:    3
  Lines Added:      0
  Lines Deleted:    0
  Total Changed:    0
  Delete Ratio:     0.00 (0%)

Status: PASSED ✅

🤖 AI Providers:
  - codex — model: gpt-5.4

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
🎉 No violations detected. Great job!
📋 Contract Configuration: contract (Source: Repository)
version: 2

trigger:
  paths:
    - "extensions/**"
    - "frontends/**/*.lisp"
    - "src/**"
    - "tests/**"
    - "contrib/**"
    - "**/*.asd"
  head_branches:
    exclude:
      - 'revert-*'

validation:
  limits:
    max_total_changed_lines: 400
    max_delete_ratio: 0.5
    max_files_changed: 10
    severity: warning

  ai:
    system_prompt: |
      You are a senior Common Lisp engineer reviewing code for Lem editor.
      Lem is a text editor with multiple frontends (ncurses, SDL2, webview).
      Focus on maintainability, consistency with existing code, and Lem-specific conventions.
    rules:
      # === File Structure ===
      - name: defpackage_rule
        prompt: |
          First form must be `defpackage` or `uiop:define-package`.
          Package name should match filename (e.g., `foo.lisp` → `:lem-ext/foo` or `:lem-foo`).
          Extensions must use `lem-` prefix (e.g., `:lem-python-mode`).

      - name: file_structure_rule
        prompt: |
          File organization (top to bottom):
          1. defpackage
          2. defvar/defparameter declarations
          3. Key bindings (define-key, define-keys)
          4. Class/struct definitions
          5. Functions and commands

      # === Style ===
      - name: loop_keywords_rule
        prompt: |
          Loop keywords must use colons: `(loop :for x :in list :do ...)`
          NOT: `(loop for x in list do ...)`

      - name: naming_conventions_rule
        prompt: |
          Naming conventions:
          - Functions/variables: kebab-case (e.g., `find-buffer`)
          - Special variables: *earmuffs* (e.g., `*global-keymap*`)
          - Constants: +plus-signs+ (e.g., `+default-tab-size+`)
          - Predicates: -p suffix for functions (e.g., `buffer-modified-p`)
          - Do NOT use -p suffix for user-configurable variables

      # === Documentation ===
      - name: docstring_rule
        prompt: |
          Required docstrings for:
          - Exported functions, methods, classes
          - `define-command` (explain what the command does)
          - Generic functions (`:documentation` option)
          Important functions should explain "why", not just "what".
        severity: warning

      # === Lem-Specific ===
      - name: internal_symbol_rule
        prompt: |
          Use exported symbols from `lem` or `lem-core` package.
          Avoid `lem::internal-symbol` access.
          If internal access is necessary, document why.

      - name: error_handling_rule
        prompt: |
          - `error`: Internal/programming errors
          - `editor-error`: User-facing errors (displayed in echo area)
          Always use `editor-error` for messages shown to users.

      - name: frontend_interface_rule
        prompt: |
          Frontend-specific code must use `lem-if:*` protocol.
          Do not call frontend implementation directly from core.
        severity: warning

      # === Functional Style ===
      - name: functional_style_rule
        prompt: |
          Prefer explicit function arguments over dynamic variables.
          Avoid using `defvar` for state passed between functions.
          Exception: Well-documented cases like `*current-buffer*`.

      - name: dynamic_symbol_call_rule
        prompt: |
          Avoid `uiop:symbol-call`. Rethink architecture instead.
          If unavoidable, document the reason.

      # === Libraries ===
      - name: alexandria_usage_rule
        prompt: |
          Alexandria utilities allowed: `if-let`, `when-let`, `with-gensyms`, etc.
          Avoid: `alexandria:curry` (use explicit lambdas)
          Avoid: `alexandria-2:*` functions not yet used in codebase

      # === Macros ===
      - name: macro_style_rule
        prompt: |
          Keep macros small. For complex logic, use `call-with-*` pattern:
          ```lisp
          (defmacro with-foo (() &body body)
            `(call-with-foo (lambda () ,@body)))
          ```
          Prefer `list` over backquote outside macros.
📚 About Code Contractor

Declarative Code Standards That Learn and Improve

Define domain-specific validation rules in YAML.
Your contracts document team knowledge and evolve into more accurate AI enforcement.

Want this for your repo?
Install Code Contractor

Copy link
Copy Markdown
Contributor

@code-contractor-app code-contractor-app Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Contractor validation passed ✅ — see the sticky comment for full results.

build-terminal.lisp dropped into the SBCL REPL after a successful build
(exiting only on stdin EOF) and trusted the compiler's exit code without
checking that terminal.so was actually written.

- Quit 0 after a successful build so `sbcl --load` never blocks on stdin.
- Delete any stale .so before compiling, then assert the artifact exists
  afterwards, so a compiler that exits 0 without producing the file is
  treated as a failure instead of a silent no-op.
- Factor the failure path into build-failed (stderr + exit 1).
Copy link
Copy Markdown
Contributor

@code-contractor-app code-contractor-app Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Contractor validation passed ✅ — see the sticky comment for full results.

@theangelperalta theangelperalta merged commit 2f435b0 into lem-project:main Jun 3, 2026
12 checks passed
theangelperalta added a commit that referenced this pull request Jun 3, 2026
PRs #2204 and #2205 merged in the reverse of their stacking order (#2205,
which contained #2204's commits, merged first; #2204 merged second). Because
#2205 had added static-p/vterm-flags just above build-failed, the squash of
#2204 re-applied the build-failed block instead of recognizing it as already
present, leaving two identical definitions in build-terminal.lisp.

Remove the duplicate. No behavior change (the copies were identical); this
just silences the SBCL redefinition warning.
theangelperalta added a commit that referenced this pull request Jun 3, 2026
* feat(nix): build and bundle terminal.so in the flake

The Nix flake never built the lem-terminal native helper: it has no
libvterm input and no terminal.c build step, so on Nix the terminal
extension silently disabled itself (ffi.lisp's use-foreign-library is
wrapped in ignore-errors). Removing the committed binaries in #2204 did not
regress this -- the prebuilt .so was dynamically linked against a libvterm
that was never present in the Nix sandbox -- but it also left Nix without a
working terminal.

Add a terminal-so derivation that compiles extensions/terminal/terminal.c
against pkgs.libvterm, mirroring the existing ts-wrapper / c-webview C
derivations, and add it to the nativeLibs of lem-ncurses, lem-sdl2 and
lem-webview.

Dynamic linking is the idiomatic choice on Nix (unlike the AppImage/macOS
bundles, which static-link in #2205): the stdenv records an RPATH to the
pinned libvterm store path, so terminal.so resolves libvterm at runtime with
no bundling or relinking, and nativeLibs puts terminal.so itself on the
library path that ffi.lisp's "terminal.so" lookup uses at build and run time.

* fix(nix): gate terminal-so on Linux (libvterm is Linux-only in nixpkgs)

nixpkgs' libvterm (0.99.7) has meta.platforms = Linux only, so referencing
pkgs.libvterm unconditionally made `nix flake check --all-systems` fail at
evaluation on the aarch64-darwin / x86_64-darwin systems, breaking every
build job.

Add terminal-so to nativeLibs only when stdenv.isLinux (via lib.optionals,
which doesn't force its list when the condition is false). Terminal works on
Linux Nix builds; Darwin Nix is unchanged (no terminal, as before, since
nixpkgs doesn't package libvterm for Darwin).

* fix(nix): add glib cflags for terminal.so (nixpkgs libvterm is the neovim fork)

nixpkgs' libvterm (0.99.7) is the Neovim fork, whose vterm.h #includes
<glib.h>. terminal.c includes vterm.h, so the build failed with
"fatal error: glib.h: No such file or directory".

Add pkg-config + glib and pull GLib's compile/link flags via
`pkg-config --cflags --libs glib-2.0`.

* fix(nix): use libvterm-neovim, not libvterm (wrong library)

The compile failed on glib.h then curses.h from libvterm-0.99.7's vterm.h
because pkgs.libvterm is the old, abandoned glib/curses-based "libvterm" with
an API incompatible with terminal.c. Leonerd's modern libvterm -- the one
terminal.c targets -- is packaged as pkgs.libvterm-neovim.

Switch to pkgs.libvterm-neovim and drop the glib/pkg-config workaround.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant