feature: ls walk glob filesystem builtins#407
Merged
Conversation
`ls` is becoming a builtin (directory listing) alongside `walk` and `glob`. Four spots in the tree still use `ls` as a local binding name - two regression-test fixtures, one example, and the dedicated builtin-rename example whose body itself uses `ls` as a list local. Rename each to a non-builtin, non-reserved short name (`xs`, `ws`, `lims`) so the upcoming reserved-namespace check stays green.
Three new Result-typed builtins for directory enumeration. Closes the categorical gap personas have hit repeatedly: `rd` exists for single files but there was no way to list a directory or recursively find files matching a pattern - the kind of thing every shell and every scripting language ships out of the box. - `ls dir > R (L t) t` lists immediate entries (filenames only, not full paths). Sorted lexicographically so output is deterministic across runs and filesystems. Empty dirs return `[]`, not Err. Missing dirs and permission errors surface as Err so the caller can branch. - `walk dir > R (L t) t` does a depth-first recursive walk, returning paths relative to `dir` with forward-slash separators (deterministic across OSes, composable with `cat`/`fmt` for downstream reads). Symlinks are not followed to avoid cycles on typical project trees. - `glob dir pat > R (L t) t` filters `walk` output by a shell-style pattern. `*`/`?`/`[abc]` within a path segment, `**` recursive across segments. Hand-rolled matcher rather than the `glob` crate to keep the build lean - it's small enough that pulling a crate isn't worth it. All three are tree-bridge eligible (`is_tree_bridge_eligible`), so VM and Cranelift inherit them through the same path `rd`/`rdb`/`sleep` already use. No new opcodes, no Cranelift helpers, no AOT changes. `tree_bridge_returns_result` picks them up so `!`-unwrap composes correctly through the bridge.
11 cross-engine cases against a per-test tempdir fixture: - ls: basic listing (sorted, files + dirs), empty dir returns `[]`, missing dir surfaces as Err. - walk: full recursive output sorted lexicographically, missing dir as Err, single-file dir (regression guard for an early DFS that pushed the file as a directory). - glob: single-segment `*`, recursive `**`, character class `[ab]`, missing dir as Err, no-matches returns `[]` (not Err). Every assertion runs against tree, VM, and Cranelift JIT. The bridge is exactly where past cross-engine drift has hidden, so the assertions care more about parity than about exhaustive matcher coverage. `examples/fs-builtins.ilo` shows the four call shapes (`ls!`, `walk!`, `glob!`, top-level Err propagation) using the repo's own `examples/` directory as a stable fixture so the engine-harness can run it without setup.
SPEC.md gets three new rows in the builtins table next to the rest of the file-I/O family, plus `ls` added to the 2-char reserved-names list and `walk` / `glob` mentioned in the longer-builtins prose. `ai.txt` and `skills/ilo/SKILL.md` regenerate from SPEC.md via build.rs.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
2 tasks
…s-builtins # Conflicts: # SPEC.md # ai.txt # skills/ilo/SKILL.md # src/builtins.rs # src/vm/mod.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three new Result-typed builtins for directory enumeration, closing the categorical gap personas have hit repeatedly:
rdexists for single files but there was no way to list a directory or recursively find files matching a pattern (a basic facility every shell and scripting language ships).ls dir > R (L t) t- non-recursive listing, filenames only, sorted lexicographically. Empty dir returns[], missing dir / permission denied surface as Err.walk dir > R (L t) t- depth-first recursive walk, paths relative todirwith forward slashes (deterministic across OSes). Symlinks not followed.glob dir pat > R (L t) t- shell-style filter.*/?/[abc]within a segment,**recursive across segments. No matches returns[], not Err.All three are tree-bridge eligible so VM and Cranelift inherit them through the same path
rd/rdb/sleepalready use - no new opcodes, no Cranelift helpers, no AOT touch.Why
Originating: filesystem-walk persona reports in
ilo_assessment_feedback.md. Token-cost framing:walk diris 8 chars vs the alternative of asking the user to pre-compute the file list and pass it as an arg, or hand-rolling a recursive list-pass withrdon each file. Plus the matcher inglobcollapses the most common shape (find . -name '*.ext') to a single call.Repro before / after
Before (no way to list a directory at all):
```
$ ilo 'main>n;len ls "."'
ILO-T004: undefined variable 'ls'
```
After:
```
$ ilo 'main>R (L t) t;ls "examples"'
[02-with-dependencies.ilo, at-float-index.ilo, ...]
$ ilo 'main>R (L t) t;glob "examples" "**/*.ilo"'
[02-with-dependencies.ilo, ..., window-listview-perf.ilo]
```
What's in the diff
Four commits, one logical change per commit:
lsbinding name ahead of fs builtins - rename four pre-existingls=bindings (two test fixtures, one example, plus the bindings inside thebuiltin-binding-name-rename.iloexample which itself useslsas a local) to non-builtin names. Needed beforelslands as a reserved short name.Builtinenum entries +from_name/name/ALL/round-trip-tests insrc/builtins.rs, signatures insrc/verify.rs, tree-walker impl + helperwalk_collect+ helperglob_matchinsrc/interpreter/mod.rs, tree-bridge eligibility insrc/vm/mod.rs.tests/regression_fs_builtins.rsusing a per-test tempdir fixture, plusexamples/fs-builtins.ilofor the engine harness (uses the repo's ownexamples/dir as a stable fixture).ai.txtandskills/ilo/SKILL.mdregenerate from SPEC via build.rs.Test plan
cargo test --release --features cranelift- whole suite greencargo fmt --checkcleancargo clippy --release --features cranelift -- -D warningscleantests/regression_fs_builtins.rs- 11 tests across--run-tree,--run-vm,--jittests/examples_engines.rs- newexamples/fs-builtins.iloruns through every enginetests/regression_reserved_names_doc.rs- SPEC reserved-names list matchesBuiltin::ALLFollow-ups
docs/builtins/data-io.mdin the ilo-lang/site repo) edit prepared at/Users/dan/code/ilo-lang/site-docs-refresh/src/content/docs/docs/builtins/data-io.mdbut couldn't push - the site main checkout is not present on this machine; needs landing in a separate PR over there.