Skip to content

v0.12.0

Choose a tag to compare

@github-actions github-actions released this 12 May 16:01
· 132 commits to main since this release

Added

  • HTML render splice (#563).
    lexd convert --to html (via
    lex_babel::serialize_to_html_with_registry) now actually splices
    handler-rendered HTML into the output for annotations whose
    registered handler returns RenderOut::String. The default
    <!-- lex:label --> ... body ... <!-- /lex:label --> rendering
    is replaced by the handler's raw HTML; the annotation's body
    events are suppressed (the handler owns the full rendering of its
    labelled node). Mechanism: AST-walk and event-walk visit
    annotations in matching document order; the HTML builder
    maintains a counter, looks up the matching plan entry on
    Event::StartAnnotation, and emits a sentinel comment that
    string-replaces with the handler's HTML after DOM serialization.
    Handler diagnostics from on_render continue to surface via
    HtmlExportOutcome::diagnostics.

    • RenderOut::WireAst outputs fall through to default rendering
      with the existing format-shape-mismatch diagnostic;
      WireAst → HTML conversion is a follow-up.
    • Document-level annotations (extracted into the IR's synthetic
      frontmatter block before events are emitted) are not splice
      targets — the existing frontmatter path handles them as
      metadata. Per-element annotations (the actual extension use
      case) splice correctly.
    • Multi-annotation documents with trailing annotations may hit
      an IR-ordering quirk that breaks the counter-based alignment.
      Single-annotation documents and clearly-separated annotations
      work; see test
      multi_annotation_splice_is_a_known_limitation for context.
  • lexd labels emit <doc> [--label X]... [--namespace N]...
    (#564). Pull-based
    NDJSON export — one record per labelled annotation / verbatim in
    the document. Output uses the wire Position / Range types so
    the same parser that consumes LSP hover and extension hook
    payloads consumes emit output unchanged. Body shapes are tagged:
    {kind: "none"}, {kind: "text", text: "…"}, or {kind: "lex", wire: [...]}. --label and --namespace are repeatable and
    intersect. No registry boot required — to_wire_node produces
    the wire form without schema lookup, so this command runs against
    documents whose namespaces aren't registered. Exit 0 on success
    (including zero matches), 2 on parse failure.

  • Resolver machinery (#546 item A, partial). lex-extension-host
    gains a pluggable [Fetcher] trait, a FetcherRegistry, and a
    content-keyed ResolverCache (24-hour TTL for mutable refs,
    indefinite for immutable). The four remote schemes
    (github:/gitlab:/https:/git+ssh:) ship as stub Fetcher
    impls that return FetchError::Unimplemented — same observable
    behaviour as before, but plugged into the new dispatch so an
    implementer's PR per lex#562
    only needs to swap the stub for a real fetcher. path: stays
    built-in (no cache, no fetcher) — it's special-cased in
    resolve_namespace_with before the registry is consulted.

  • resolve_namespace_with(uri, workspace_root, &registry, &cache)
    for callers that want explicit control over the registry + cache
    (hosts constructing one cache + registry at boot time; tests
    using tempdir caches and custom fetchers). The existing
    resolve_namespace(uri, workspace_root) is now a convenience
    wrapper using default_fetcher_registry() and
    ResolverCache::user_default() (XDG-cache-home aware).

  • End-to-end validation: tests/resolver_http_e2e.rs exercises the
    full pipeline (URI parse → registry dispatch → cache miss →
    fetch → cache hit on second resolve) with a hand-rolled HTTP
    Fetcher against a std::net::TcpListener mock server. Proves
    the machinery works for a non-stub fetcher without depending on
    any external network.

  • sha2 workspace dependency, used by ResolverCache for the
    content-key hash function (hash(scheme + body + rev + subdir)
    → 64-char hex directory name under the cache root).

  • Extract-to-include LSP command
    (#497).
    lexd-lsp registers a new workspace command lex.extractToInclude
    that splits a selected slice of a Lex document out into a new file
    referenced via :: lex.include src="…" ::. The command takes
    positional arguments [uri, range, src], validates the target
    path against the configured includes root (returns distinct
    ExtractError variants for empty / URL / absolute / root-escape /
    existing-target / missing-parent-dir cases), indent-shifts the
    selection so its shallowest non-blank line lands at column 0,
    parses the shifted text as a Lex fragment, and returns an atomic
    WorkspaceEdit containing CreateFile + target-content
    TextEdit + host-replace TextEdit. All logic lives in
    lex_lsp::features::extract so per-editor shims stay thin —
    editor wiring is tracked at
    #498. The deeper
    GeneralContainer host-policy check (e.g. rejecting a Session
    selected for extraction into a Definition body) is reserved for
    a follow-up; the ExtractError::ContainerPolicy variant stays in
    the enum for it.

Fixed

  • lexd inspect silently dropped verbatim blocks when source
    mentioned lex.include in prose

    (#505). The CLI's
    expand_includes_to_source round-trips source through
    resolve_from_sourceserialize_to_lex_with_rules → re-parse
    whenever the literal string lex.include appears anywhere — even
    in backticked prose or proposal docs describing the feature. The
    Lex serializer wrote a verbatim block's subject line directly
    after the preceding paragraph (the parser consumes the
    separating blank line as part of the verbatim's preamble, so it
    isn't represented as a BlankLineGroup in the AST and no other
    visitor emits it). On the re-parse the paragraph absorbed the
    subject and the verbatim — plus everything it bracketed — silently
    disappeared from the resolved tree. visit_verbatim_block now
    emits the leading blank line, suppressed when the verbatim is the
    first child of a Subject:-style container opener (Definition,
    verbatim group) where a blank at column 0 would terminate the
    container.

Changed

  • Trust-matrix flip (lex#528 PR 12d). lex_fmt::boot_registry
    now installs the OS-appropriate Sandbox on the trust gate (via
    the new lex_extension_host::sandbox::os_default() factory) and
    passes the same Arc<dyn Sandbox> to every
    SubprocessHandler::spawn_with_sandbox call. The effect is
    per-OS:
    • Linux: declared-pure subprocess handlers (capabilities: { fs: false, net: false }) auto-trust under the default
      engine — no prompt, no --enable-handlers flag. The kernel
      enforces the declared capability set via LinuxSandbox
      (seccomp + landlock). This is the user-visible matrix flip
      promised by §8 of the proposal.
    • macOS: pure handlers continue to prompt because
      MacosSandbox::supports is pinned to false pending a
      hardened (deny default) SBPL profile. The same prompt UX
      Windows uses.
    • Windows + other: pure handlers continue to prompt
      NullSandbox is the fallback. lex#528 PR 12c (Windows Job
      Objects + restricted tokens) is unscheduled; the trust gate
      routes those subprocesses to the prompt path until it ships.
      Full subprocess handlers (any capabilities declaring fs or
      net) always prompt regardless of OS — supports returns
      false for non-pure shapes on every impl.

Added

  • lex_extension_host::sandbox::os_default() -> Arc<dyn Sandbox>:
    factory selecting the per-target default sandbox. Used by the
    δ-phase trust-matrix flip (above); also available to embedders
    building a custom Engine who want the same per-OS dispatch.
  • lex-extension-host::sandbox::MacosSandbox: macOS implementation of
    the Sandbox trait (lex#528 PR 12b). Installs a Sandbox Profile
    Language (SBPL) policy via the libSystem sandbox_init API inside
    a pre_exec hook so it applies to the child after fork() and
    survives execve(). The v1 profile denies network* and reads of
    /etc (covering both probe-fixture targets) while keeping other
    operations permissive enough for the system loader to bring up a
    Rust binary. supports() returns false for every capability
    shape on macOS
    until a hardened (deny default) profile lands —
    the current (allow default) profile still permits writes
    anywhere on disk and reads outside /etc, so auto-trusting on it
    would let a pure-declared handler silently exfiltrate or modify
    user data. The trust gate routes pure handlers to the prompt path
    on macOS, same as Windows or no-landlock Linux, until the
    hardened profile work ships. sandbox_init is deprecated since
    macOS 10.8 but still resolvable in libSystem through Sequoia —
    same dependency Apple's sandbox-exec utility relies on.
  • lex-extension-host::sandbox::LinuxSandbox: Linux implementation of
    the Sandbox trait (lex#528 PR 12a). Combines landlock (filesystem
    allowlist — dynamic-loader paths + the handler binary itself) and
    seccompiler (network-stack syscall deny) installed via a pre_exec
    hook so the policy applies to the child after fork() and survives
    execve(). supports() returns true only for the
    Capabilities::is_pure() shape; finer capability shapes report
    unsupported until the schema grows the corresponding fields. Build
    surface remains MIT/Apache (landlock MIT/Apache, seccompiler
    Apache/BSD) — lex-extension-host and downstream consumers stay MIT.
  • lex-extension-host::sandbox: new module hosting the Sandbox
    trait (the OS-level sandbox facade), a SandboxError error type,
    and a NullSandbox no-op default that reports
    supports(_) == false for every capability set on every platform.
    Foundation for the δ-phase trust matrix flip (lex#528): per-OS
    sandbox implementations (12a Linux via seccomp+landlock, 12b macOS
    via sandbox-exec, 12c Windows via Job Objects + restricted tokens)
    plug in behind this trait; the trust gate consults
    Sandbox::supports(caps) to decide whether a declared-pure handler
    can auto-trust without a prompt.
  • SubprocessHandler::spawn_with_sandbox companion to the existing
    spawn. Takes Arc<dyn Sandbox> so the host can install one
    instance and share it with TrustGate. The worker thread calls
    apply_to(&mut cmd, capabilities) before cmd.spawn() so the
    kernel enforces declared capabilities from the child's first
    instruction. spawn (the existing entry point) now delegates to
    spawn_with_sandbox with NullSandbox — no behaviour change for
    current callers.
  • SpawnError::Sandbox(String) variant for policy-install failures
    on the host side. Same handling as other spawn errors today;
    retry-with-prompt path is future work alongside PR 12d.
  • TrustGate::set_sandbox(Arc<dyn Sandbox>) and
    TrustGate::sandbox() accessor (returns the Arc by clone). The
    default install is NullSandbox, so β/γ behaviour is preserved
    (every subprocess prompts). When PR 12a/b/c land, lex-fmt
    swaps in the OS-appropriate impl and shares the same Arc across
    the gate and the transport — guaranteeing the auto-trust decision
    is anchored on the sandbox that actually enforces policy.

Changed

  • Renamed the lex-engine crate to lex-fmt and positioned it as the
    canonical Rust embedder API for the lex document format. The
    boot_registry / ExtensionSetup / BootOutcome types move
    unchanged; only the crate name (and the use paths in lexd /
    lexd-lsp) change. Sets up PR 11's Engine::builder() to land in
    a discoverable place (use lex_fmt::Engine) rather than under a
    name that says "boot helper". lex-engine 0.11.0 stays published
    on crates.io for historical reference; future releases publish as
    lex-fmt.
  • TrustGate::evaluate now short-circuits to Trusted for the
    (transport=Subprocess, capability=Pure) pair when
    sandbox.supports(Capabilities::default()) returns true. With
    NullSandbox (the default), this branch is inactive —
    observable behaviour is unchanged. Tests cover both the
    supported and unsupported paths via a mock sandbox.