Skip to content

v0.10.0

Choose a tag to compare

@github-actions github-actions released this 04 May 19:40
· 190 commits to main since this release

Added

  • lex-core: Range.origin_path: Option<Arc<PathBuf>> field with with_origin builder and origin() accessor. Currently always None — pure additive scaffolding for the upcoming includes feature (PR 1 of 10). The field is #[serde(skip)] so existing AST JSON output is byte-identical. Range is now #[non_exhaustive]; equality and hashing ignore origin_path (positional only). See comms/specs/proposals/includes.lex for the full design.
  • lex-core: Annotation::is_include() and Annotation::include_src() accessors plus RESERVED_NAMESPACE_PREFIX ("lex.") and INCLUDE_LABEL ("lex.include") constants. The lex.* annotation label namespace is now reserved for core-defined semantics; the accessors hide the string-match on the reserved label and serve as a migration boundary if includes are later modeled as a distinct AST node type. Pure additive scaffolding for the includes feature (PR 2 of 10).
  • lex-core: new lex_core::lex::includes module skeleton — Loader trait, ResolveConfig, LoadError, IncludeError, and a stub resolve_includes that returns its input unchanged. The trait/config/error surface is stable from this PR; splice logic, container-policy validation, recursion, cycle detection, and depth limiting land in PRs 4–6. lex-core's own code does not reference std::fs; loaders are injected. New test-support cargo feature exposes MemoryLoader so downstream crates' tests can exercise APIs that take a Loader. Pure additive scaffolding (PR 3 of 10).
  • lex-core: include resolver now actually splices. resolve_from_source(source, source_path, config, loader) parses the entry-point file (without annotation attachment so includes are visible in container children), recursively walks every container looking for lex.include annotations, loads each target through the injected Loader, parses it independently, stamps Range.origin_path on every node from the loaded file, validates the splice list against the host container's policy (Sessions are rejected inside Definition / Annotation body / ListItem), and replaces the include annotation with the resolved content in-place. The included file's DocumentTitle becomes a leading Paragraph and document-level annotations become regular annotations — matching what a textual paste with indent-shift would produce. After all splices, annotation attachment runs once on the merged tree so the include annotation lands on the first spliced sibling per standard rules. Adds IncludeError::MissingSrc. PR 4 of 10.
  • lex-core: include resolver is now recursive. Each loaded file is fully resolved (its own lex.include annotations replaced) before being spliced into the host. Each level of recursion walks with that file's own directory as the host, so a relative path inside an included file resolves from the file's location — not the entry's. Cycle detection via an active-chain stack of lexically normalized absolute paths (a path already on the stack is a cycle; the resolver does not touch the filesystem, so symlink canonicalization is up to the loader). Depth limit defaults to 8 (ResolveConfig::max_depth), configurable per project. IncludeError::Cycle and IncludeError::DepthExceeded both carry the offending include site (Range) and the resolution chain at the moment of failure for diagnostics. PR 5 of 10.
  • lex-core: origin-aware reference helpers complete the includes machinery. lex_core::lex::includes::resolve_file_reference(target, ref_origin, root) resolves a ReferenceType::File target the same way the include resolver resolves include paths — relative paths from the reference's authoring directory (Range.origin_path's parent), root-absolute under root, with the same lexical-normalization + root-escape protection. Document::find_annotation_by_label_in_origin(label, origin) scopes footnote-style lookups to the file the reference was authored in, so a [1] in chapter.lex finds :: 1 :: defined in chapter.lex and not some other file's :: 1 ::. Unlike the existing find_annotation_by_label, the new walker checks both standalone annotations and attached .annotations slots on every node type that carries them (Session, Definition, ListItem, Paragraph, List, Table, VerbatimBlock) — necessary post-attachment. Both helpers are pure additive utilities; downstream wiring (CLI in PR 7, LSP in PR 8) will consume them. PR 6 of 10.
  • lex-core: Annotation::include_src() now returns Option<String> (was Option<&str>) and unquotes the parameter value. The previous return type left raw quotes on parsed sources, which broke any downstream that used the value as a path.
  • lex includes are now live for users. lexd convert and lexd inspect expand :: lex.include src="..." :: annotations by default, splicing the included file's content into the host tree before serializing. Pass --no-includes to operate on the unresolved tree (useful for inspecting a single document atom). lexd format never expands includes (per spec §11.4). (Note: the lex serializer's visitor does not currently emit attached annotations on Session/Definition/etc., which means a :: lex.include :: line that gets attached during parsing may not appear in formatter output verbatim — that's a pre-existing serializer limitation, separate from the includes feature, that will be addressed in a follow-up.) PR 7 of 10.
  • lex-lsp adds goto-definition + hover for lex.include annotations. Click on :: lex.include src="chapter.lex" :: to jump to chapter.lex; hover to see a small preview (resolved path + first non-blank lines from the target). Path resolution uses the same [includes].root precedence as the resolver. Both features short-circuit cleanly: cursor not on an include falls through to the existing in-document goto/hover; untitled buffers and broken paths return None gracefully. PR 9 of 10.
  • lex-lsp runs include resolution on did_open / did_change and surfaces include errors as diagnostics. Each IncludeError variant maps to a distinct diagnostic code (include-cycle, include-depth-exceeded, include-root-escape, include-not-found, include-parse-failed, include-container-policy, include-loader-io, include-missing-src); diagnostics that carry an include_site (cycle, depth, container policy, missing src) point at the offending annotation, others fall back to the document head. The LSP continues to store the unresolved host parse in its document store so position-based features (semantic tokens, hover, goto-definition, document symbols) keep using ranges in the host buffer's coordinate space — the merged tree is computed for diagnostic purposes only. Origin-aware position mapping (so cross-file goto / hover can land in the right buffer) lands in a follow-up. Fast path skips the resolver entirely when source contains no lex.include literal (avoids per-keystroke work and spurious include-parse-failed diagnostics on ordinary parse errors). Untitled URIs skip resolution silently. Editor packages need no changes — they pick this up via the next lexd-lsp pin bump. PR 8 of 10.
    • New --includes-root <PATH> global flag explicitly sets the resolution root. Default discovery: nearest .lex.toml walking upward from the entry file, falling back to the entry's own directory.
    • New [includes] config section with root (path) and max_depth (integer, default 8). CLI flags override config, config overrides defaults.
    • New lex_core::lex::includes::FsLoader is the production loader (filesystem-backed). Only FsLoader references std::fs — the rest of lex-core stays sandbox-clean.
    • Editor packages (vscode, nvim, lexed) pick up nothing yet — LSP integration lands in PR 8.

Changed

  • Bumped comms submodule to v0.16.0, which adds the canonical specs/elements/lex.include.lex element doc, the specs/elements/lex.include.docs/ fixture set, and formally reserves the lex.* annotation label namespace in specs/general.lex §3.1. Also archives the includes proposal to specs/proposals/done/includes.lex per the new "frozen-when-implemented" convention.