Skip to content

glep 2.0.0

Latest

Choose a tag to compare

@anishfyi anishfyi released this 26 Oct 17:32
· 2 commits to main since this release
027a9a0

ripgrep parity and a parallel search pipeline: most of rg's flags with rg's semantics, checked byte for byte against rg on both the single-thread and the worker-pool search paths, and a first crates.io release since 0.2.3.

2.0.0 follows 0.3.1 directly; there was no 0.4.0 release.

Upgrading from 0.3.1:

  • Existing indexes rebuild once, on the first query (postings format v2 to v5).
  • Run in a subdirectory of an indexed tree, glep now uses the nearest .glep/ above it and scopes the search to the current directory; 0.3.1 built a second index in the subdirectory.
  • -l, -c and --json are last-wins output modes, as in rg; 0.3.1 rejected -l or -c with --json, and -c with -l, as a usage error.
  • Binary detection follows rg's reader (a NUL in the first 64 KiB, where 0.3.1 looked at 8 KB), and UTF-16 files with a byte-order mark are transcoded and searched instead of skipped.
  • .rgignore files apply, and --no-ignore-vcs also turns off .git/info/exclude and the global gitignore, so results can change in trees that rely on them.

Performance

  • Worker-pool search: one reusable searcher per worker thread, an in-order printer thread with a bounded window, whole-file slice searches and coalesced writes, so output stays byte-identical to rg --sort path at unordered parallel throughput. The freshness diff runs sharded in parallel over an unsorted sweep, and on macOS --files answers from a dir-mtime listing cache. Every flag below runs on this pipeline; flags whose bytes are not the file's own (--pre, -z, --binary, --no-mmap) read through the searcher's buffer inside the worker.
  • Files the sweep flags hidden are no longer re-checked against every ignore file on the way to them unless something can still show them (--hidden, a -g, -t or --ignore-file whitelist, or a hidden operand): the flag already holds every ignore file's ! rescue. A default --files listing or full-scan search over a tree with many dotfiles in many directories paid about 20 us of ignore-file syscalls per hidden file (on a tree with 2,000 .gitignore files, --files took 54 ms instead of 11 ms).
  • On macOS the search worker pool defaults to the performance-core count (hw.perflevel0.physicalcpu) instead of one worker per logical core when -j is absent or 0. A full scan is mostly kernel open, read and page-fault work whose lock contention grows faster than the thread count, so efficiency cores add contention rather than throughput: on an M3 Pro (5 performance plus 6 efficiency cores) a kernel-tree full scan now spends about a quarter of the system time and finishes in roughly half the wall time (glep q 2.44 s to 1.35 s, glep -c ab 2.70 s to 1.15 s, glep static 1.24 s to 0.78 s, against rg's 2.36 s, 2.40 s and 2.31 s on the same busy machine). GLEP_THREADS=N overrides the default and an explicit -j N still wins; elsewhere (Linux, Windows, Intel Macs) the default stays one worker per core. The rayon pool behind sweeps and listings keeps one thread per core, where a --files listing runs about 4% faster than at the performance-core count.
  • Search workers read each file of up to 1 MiB whole into one reused buffer and search it as a single slice, instead of memory-mapping every file. A file under 64 KiB now costs an open, two reads and a close; a map also cost an fstat, an mmap, a fault per page and an munmap, and mmap and munmap serialize the workers on the process's VM map lock. Bigger files are still mapped: copying them costs more than faulting them in, and reading files up to 8 MiB made a narrowed kernel search whose candidates include the multi-megabyte amdgpu register headers about a third slower (--ttl 60 PCI_EXP_LNKCTL2_TLS 25 ms to 35 ms). Each worker also searches with its own clone of the matcher, which shares the compiled program but owns its scratch space, so workers no longer queue on the regex engine's shared cache pool, and a file's path is built in a reused buffer. Output is unchanged. On the kernel tree, one interleaved hyperfine invocation per row, mean: glep -c '.*' 2.05 s to 1.80 s, -c '\w+' 1.52 s to 1.38 s, q 1.45 s to 1.36 s, static 892 ms to 848 ms (rg: 2.48 s, 2.61 s, 2.84 s, 2.68 s); at -j 4, -c '.*' takes 1.90 s against rg's 2.14 s.
  • A --files listing scoped to path operands (or run from a subdirectory) lists straight from a live sweep of its operands and never opens, builds or writes the index. The scoped sweep the index path ran already produced exactly the files it listed, so loading and diffing the manifest (about 11 ms on the kernel tree, whatever the scope) bought nothing, and on an unindexed tree a scoped listing no longer builds the whole index first. On macOS the sweep starts at each operand: the levels above it are read by name (their .gitignore, .ignore and .rgignore, then the next level's stored name and type) instead of listing every directory on the way down one after another, several operands walk in parallel, and a listing reads directories with readdir (names and types only) on a pool sized like the search pool. A scoped search's freshness sweep starts at its operands the same way. Output is unchanged. On the kernel tree, one interleaved hyperfine invocation per row, mean (min) in ms for the previous build, this one and rg: --files '**' drivers/net 22.4 (21.2), 7.9 (7.2), 10.1 (9.1); drivers 48.0 (46.4), 26.7 (24.8), 32.1 (30.8); fs/ext4 include/linux 23.0 (22.0), 6.0 (5.5), 7.3 (6.3); scripts/dtc/include-prefixes, which holds only symlinks, 16.6 (14.0), 3.2 (2.7), 4.9 (3.8).
  • The first query on an unindexed tree holding more than 64 MiB no longer waits for the index build. It answers from the sweep that measured the tree, at full-scan cost, the way --no-index answers from its walk, and once its output is flushed it starts the build in a detached child at background priority (utility QoS on macOS, nice 10 on other Unix systems). The child's stdio is the null device, so a pipe or $(...) reading the query sees the end of its output as soon as the query exits, and it runs in a process group of its own, so Ctrl-C at the terminal leaves it alone. Queries that arrive during the build answer from their own sweep; the first one after it opens the index. A smaller tree still builds inline, now from that same sweep instead of sweeping twice, and no query runs the freshness sweep right after building the index itself. GLEP_SYNC_BUILD=1 keeps the inline build and GLEP_BG_BUILD_MIN_BYTES moves the threshold. Windows keeps the inline build, since a child there inherits the query's stdout pipe and its reader would wait for the build. On the kernel tree, over fresh clones of it in one hyperfine invocation, the first query went from 26.4 s to 2.6 s for PCI_EXP_LNKCTL2_TLS and from 25.5 s to 2.6 s for static, against rg's 1.5 s and 1.8 s there. That first query is a full scan, which on this machine ran between a third of rg's time and 1.7 times it as kernel contention came and went (1.1 to 1.9 s while rg took 2.8 to 3.3 s). The build took 22 to 24 s in the background, and the query after it took 150 ms against rg's 1.57 s.
  • A root-wide --files listing on macOS takes about half as long when the dir-mtime listing cache answers it. The pruned walk no longer copies every cached leaf directory and then reassembles, compares and frees a whole new tree on each run: it records cached leaves by reference, notes whether any directory it lists live differs from its cached node, and assembles and saves a tree only then. -g, --iglob, -t/-T, --ignore-file, --max-depth and the hidden rule apply inside the listing, before any path is copied, so --files -g '*.h' copies and sorts 25k paths of the kernel tree instead of 86k, and the listing and the loaded cache are left to the process exit instead of being freed one allocation at a time. A cache built after a full listing also no longer trusts a directory or ignore-file mtime from after its sweep started: a file created, or a .gitignore edited, while the cache was being built stayed hidden from every later listing until its directory changed again. Output is unchanged. On the kernel tree, one hyperfine invocation per row, mean (min) in ms for the previous build, this one and rg: --files -g '*.h' 68.7 (64.2), 33.1 (29.2), 75.9 (67.0); --files 64.7 (62.7), 31.7 (30.4), 72.1 (70.1); --files '**/*.c' 60.8 (59.7), 30.8 (29.4), 70.3 (68.4); and with a new file created in a leaf directory before every run, so that every run rewrites the cache, --files -g '*.h' 78.0 (72.7), 54.3 (51.0), 81.1 (78.6).
  • A search whose pattern no trigram can narrow (a pattern shorter than three bytes, .*, \w+, [0-9]{2}, or any pattern under -v, --passthru, -c --include-zero, a non-UTF-8 -E or --pre) chooses its plan before the index opens, and over the whole tree on macOS no longer opens the index at all: it searches the file set the dir-mtime listing cache lists, filtered as the index's would be, instead of loading the manifest and running the full freshness sweep only to get every file back (about 170 ms of work before the first file was searched on the kernel tree, now about 45 ms). Every listed file is searched and binary files are left to the searcher's binary detection, as in rg, so on this path --stats and --json count them as searched, as rg does. The manifest and its sweep epoch are left alone, so the next narrowed query still sweeps; with --ttl such a search keeps the index path, which answers inside its window without a sweep, and on a cache miss the index answers as before. On a tree with no index yet such a search, on every platform, walks the tree as --no-index does and builds no index: the first query an index can narrow builds it. Output is unchanged. On the kernel tree, in round-robin runs of the previous build, this one and rg (8 rounds, mean ms): -c ab 1238, 1093, 2498; -c '.*' 1890, 1778, 2363; -c '[0-9]{2}' 1667, 1402, 2340; --files-without-match '.*' 1238, 1134, 1978; -c --include-zero x 1240, 1115, 2576; q 1661, 1578, 2372; and -q a, where the search ends at the first matching file, 129, 39 and 10 (rg stops walking there too). On a busier second pass the median saving per round was 63 to 204 ms across those rows and -v -c a. Narrowed searches and --files listings are unchanged, within 1%.
  • Candidate selection orders each AND group's trigrams by their stored posting-list length without decoding them, skips the group when a trigram is absent, decodes only the rarest list and merges every other list against the survivors straight from its delta varints, stopping at the last surviving id. The freshness diff keys paths by their bytes, hashed once per path in parallel. On the kernel tree, GLEP_TIMING medians: candidates for struct file_operations 4.2 ms to 1.7 ms, EXPORT_SYMBOL_GPL 1.6 ms to 1.0 ms, and sweep_diff 7.9 ms to 2.8 ms; narrowed searches about 5% faster end to end.

Matching and output

  • New flags with ripgrep semantics: -w/--word-regexp, -x/--line-regexp, -S/--smart-case, -v/--invert-match, -m/--max-count, -M/--max-columns, -o/--only-matching, -n/--line-number (no-op alias, already the default), -N/--no-line-number, --heading, and -q/--quiet. -w/-x and -i/-S are last-wins flag pairs, matching rg. Under -q, --json still emits the closing summary event and --stats still prints its block; as in rg, -q then searches every file so the counts are complete, and stops at the first deciding file only when no summary is printed. -q --files-without-match prints nothing and exits 0 when some searched file has no match.
  • The Rust-regex and PCRE2 matchers are configured knob for knob like rg's: outside -U the line terminator is \n, so a pattern that must match it is an error (exit 2); -U --crlf keeps CRLF-aware anchors; a pattern that must match NUL is an error unless -a; PCRE2 gets rg's 10 MB JIT stack. --engine auto falls back to PCRE2 per pattern whenever the default engine rejects it, and -P/--engine are a last-wins pair.
  • A repeated flag is last-wins instead of a usage error, as in rg (-n -n, --sort path --sort path, -A1 -A2); repeatable lists such as -e, -g and -t keep accumulating. rg's negated and alias spellings are accepted: --no-require-git, --unrestricted, --no-hidden, --ignore and --ignore-dot/-vcs/-exclude/-global/-parent, --no-ignore-files/--ignore-files, --no-text, --no-binary, --no-pcre2, --no-follow, --no-crlf, --no-multiline, --no-multiline-dotall, --no-fixed-strings, --no-invert-match, --no-json, --no-stats, --no-trim, --no-search-zip, --no-column, --no-byte-offset, --no-include-zero, --no-max-columns-preview, --no-line-buffered, --no-block-buffered, --no-one-file-system, --no-glob-case-insensitive, --no-ignore-file-case-insensitive, --no-pre, --unicode, --messages, --ignore-messages, --no-sort-files, and the deprecated --auto-hybrid-regex and --pcre2-unicode pairs. Flags that set the same thing act where they stand, as in rg: -uu --no-hidden hides dotfiles again, --no-ignore --ignore-vcs keeps .gitignore, -P --engine default uses the default engine.
  • A flag that takes a value takes the next word verbatim, as in rg, even one that starts with a dash: -e -foo, -e '-> Result', -g -x, -r -X, --context-separator --. The agent hooks pass every Grep pattern as -e '<pattern>', so a pattern starting with a dash used to fail as an unknown flag (exit 2).
  • -s is --case-sensitive (a last-wins trio with -i/-S), as in rg; --no-messages is long-only. --regex-size-limit, --dfa-size-limit and --max-filesize take K/M/G suffixes. -E auto and -E none (byte-order-mark sniffing off) are accepted, plus --no-encoding.
  • -f keeps empty lines (an empty pattern matches every line) and -f - reads patterns from stdin. Like rg, -m 0 or an empty pattern set (without -v) exits 1 without searching, printing, or checking path operands.
  • -p/--pretty is an order-aware alias for --color always --heading --line-number, as in rg: a later --color, --no-heading or -N overrides its part, an earlier one does not.
  • --count-matches, -c -o and the --stats match count find matches the way rg's summary printer does: each line is searched without its terminator, so a pattern that can match empty (x*, ^, $, a|) no longer counts one extra match per line (or two under --crlf), and under -U only matches that start inside the matched lines count.
  • Under -U, -c counts matches only when the pattern can really span lines; a pattern that can never match a line terminator is searched line by line and -c counts lines, as in rg.
  • -v disables trigram narrowing (a file with zero occurrences matches every line under inversion); -S always narrows case-insensitively as a safe superset.
  • -e/--regexp is repeatable and -f/--file reads patterns from a file: all patterns OR together, each arm wrapped non-capturing so anchors stay per-arm (under -F each arm is a literal). With any -e/-f present, positionals are paths.
  • Binary files behave as under rg's default reader. The index's binary sniff covers the searcher's whole first 64 KB buffer, and summary modes squash a file once binary data shows up under quit detection: no -c record (not even path:0 under --include-zero) and no --files-without-match listing, though the file still counts toward that mode's exit status. A NUL past the first 64 KB of a memory-mapped file is caught by re-reading that file through the searcher's buffer whenever the outcome is observable, so its matches stop at the NUL with rg's warning. UTF-16 files with a byte-order mark are transcoded and searched, like rg (the index never indexes them and always scans them).
  • Binary files stop exactly where rg's reader stops, in every mode. Every file searched through a reader (--no-mmap, --binary and file operands, -z, --pre, -E and byte-order-mark files, and the second search of a file with a NUL past its first 64 KiB) is read as rg 15.0 reads it, in 64 KiB buffers each filled whole before it is searched, with a fresh buffer per file. grep-searcher's own reader handed a file's first three bytes over on their own and searched after any read that ended a line, so a binary file with lines one or two bytes long printed one line more than rg before a NUL past its first 64 KiB (x.bin:32769:a after rg's last line 32768), and a stream with a NUL in its first 64 KiB could print its first line. A slice search that reached a matched line holding such a NUL right at the end of the first 64 KiB's lines printed that match's context lines, and --stats and --json counted such files' matches and bytes past the point rg stops. --files-without-match, with or without -q, now reads a matching file on to its end as rg's summary printer does, so a NUL after the match squashes the file: it is not listed, and it counts as a file without a match (-q --files-without-match exited 1 where rg exits 0). rg 15.1 and later read a file's first bytes differently; see the README's known differences.
  • -l, --files-without-match, -c, --count-matches and --json are last-wins, like rg's search modes. glep status reports skipped (binary/oversized/utf16).
  • New -a/--text and --binary (last-wins pair): binary files are searched as raw text under -a and reported with a "binary file matches" notice under --binary. Binary-flagged files join the candidates only under these flags (and --null-data, a non-UTF-8 -E, or --pre); default quit detection still suppresses them, matching rg's output and exit codes.
  • New -P/--pcre2 (and --engine pcre2|default|auto): PCRE2 with JIT, configured like rg's. PCRE2-only syntax degrades the index plan to a full scan, never a wrong answer.
  • New -E/--encoding (transcode before searching; non-UTF-8 encodings bypass index narrowing since trigrams index raw bytes), --multiline-dotall, --line-buffered, --null-data (NUL is the record separator), --crlf, --no-unicode, --dfa-size-limit, --regex-size-limit.
  • Errors follow rg: an unreadable file or directory, a dangling link or a loop under -L, a corrupt -z stream, a failing --pre command (reported with its stderr as preprocessor command failed), a missing operand and the nothing-searched warning each make the exit code 2 while the search goes on, and with -q a match still exits 0. --no-messages hides the messages, never the exit code. --pre passes the cwd-relative path as the command's argument, like rg.
  • A reader that closes the pipe early (glep pat | head -1, as the hooks' head_limit pipes do) ends the run quietly, as in rg: no Broken pipe message, and the exit status the results so far decide (0 after a match). Before, glep printed glep: Broken pipe (os error 32) and exited 2, and a search whose first file took long could hang for good: pool workers waiting on the printer's window were never released once the printer stopped.
  • New -r/--replace (captures resolve), --passthru/--passthrough, --count-matches, --files-without-match, --include-zero, --stats (rg's stats block with exact counters for content, -c and -l modes), --stop-on-nonmatch (stops reading a file at the first non-matching line after a match, per file, like rg).
  • The printers are grep-printer 0.3, rg 15's: --colors 'highlight:...' styles the non-matching text of a matching line (it was rejected as an unknown output type, exit 2), -r combines with --json, whose submatches carry the replacement, and under --crlf a rewritten line keeps its own terminator (a bare LF used to come out as CRLF).
  • Output plumbing, all verified byte-identical: --column, -b/--byte-offset, --vimgrep, --trim, -0/--null/--print0, --path-separator, -H/-I, --field-match-separator, --field-context-separator, --context-separator, -p/--pretty, --color/--colors, --hyperlink-format/--hostname-bin, --max-columns-preview. A single operand that is not a directory drops the path prefix unless -H, as in rg: the operands count as typed (a missing second operand still keeps path: prefixes), and --vimgrep always prints the path (unless -I) and never prints headings.
  • --path-separator rewrites every separator byte of a printed path, as rg's printers do, so a -l, -c, --count-matches, --files-without-match or --files record for an absolute operand keeps one leading separator (%abs%x, not /%abs%x; on Windows C:/x, not C:/\/x).
  • Under --color always (or ansi, or auto on a tty) the path of every -l, -c, --count-matches and --files-without-match record and of the --files listing carries rg's path color and, with --hyperlink-format, rg's OSC 8 link; links stay off without color, as in rg. --hostname-bin runs the named program for {host} (platform hostname on failure), and invalid --color, --colors or --hyperlink-format values exit 2.
  • Records and separators end with the searcher's line terminator, like rg's printers: under --crlf the -c, -l, --count-matches and --files-without-match records and the separators between files end in CRLF, under --null-data in NUL. --crlf/--null-data and --passthru/-A/-B/-C are last-wins pairs. Separator flags decode escapes (\t, \xNN); --context-separator "" still prints a line break, the new --no-context-separator prints none, and --heading always separates files with a bare line break. --path-separator "" keeps the default.
  • --sort/--sortr (path, modified, accessed, created, none; last-wins pair): timestamp sorts reproduce rg exactly, including ties, which keep rg's walk order. --sort-files is an alias.
  • New --pre/--pre-glob (search a command's output per file), -z/--search-zip (gzip decoded in-stream), --mmap/--no-mmap, -j/--threads, -s/--no-messages, --no-ignore-messages, --generate {man,complete-*}, --pcre2-version, --type-list, --debug/--trace, and the no-op compat flags --no-config and --block-buffered.
  • --pre-glob follows rg: it is repeatable and takes rg's override rules over the path as printed (any glob selects a file for --pre, a !glob exempts one), and without --pre it changes nothing. It used to be a usage error without --pre (exit 2), a second --pre-glob replaced the first, and globs matched the root-relative path, so a run from a subdirectory could preprocess the wrong files.

Files and the index

  • Ancestor index discovery: running glep in a subdirectory finds the nearest .glep/ upward (or GLEP_INDEX_PATH, which points at an index dir directly), scopes the search to the cwd subtree, and prints paths relative to the cwd. Operands outside the cwd print with .. components.
  • Path filters scope the freshness sweep: glep pat src/ only sweeps src/, walking from the root so every ancestor ignore rule applies and an ignored operand never enters the index. Only full sweeps advance the --ttl epoch or trigger compaction, and within the ttl a scoped query (a path operand, or a run from a subdirectory) skips its sweep like a root-wide one: it used to sweep its scope every time, about 45 ms on the kernel tree's drivers/.
  • ! whitelist rules in .gitignore/.ignore rescue dotfiles from the hidden filter, in searches, listings and the listing cache alike. Existing indexes rebuild once (postings format v2 to v5) to recompute FLAG_HIDDEN and the binary flags described above.
  • -g/--glob/--iglob honor gitignore-style ! negations with last-match-wins ordering; --iglob matches case-insensitively and, as in rg, its rules follow every -g rule. --glob-case-insensitive, -t/-T/--type-add/--type-clear, --ignore-file (rules match relative to the cwd, as in rg, with --ignore-file-case-insensitive) and -d/--max-depth filter candidates by path.
  • These filters follow rg's walker level by level below each operand: -g '!dir' (and !dir/, !a/b, !**/dir) prunes the whole directory, globs match the path relative to the cwd (so -g 'deep/**' works from a subdirectory), a -g or -t whitelist rescues a hidden file, a hidden directory operand is searched like any other, and an operand itself is never filtered. Type flags apply in command-line order (-T md -t md selects md). A missing --ignore-file is a warning, not an abort, the tree's own whitelist rules beat --ignore-file rules, and among --ignore-file rules the last file with an opinion decides. The --files glob matches the cwd-relative path too.
  • A path operand that names a file is searched unconditionally, as in rg: ignore rules, the hidden filter, the index's binary flags, -g/-t/--ignore-file and --max-depth never drop it, and binary data in it is reported (binary file matches) instead of hiding the file. Before, an ignored, hidden or binary file named on the command line silently produced nothing.
  • Several path operands behave as in rg: results group by operand in command-line order (path order within each), a file under two operands prints twice, -d counts from each operand, and --sortr path sorts across operands. When every operand is missing nothing is searched (the --json summary and --stats block still print) and glep exits 2.
  • Live-scan escape hatches that never touch the index: --no-index, -L/--follow, --one-file-system, --require-git outside a repo, the per-source toggles --no-ignore-dot/--no-ignore-vcs/--no-ignore-exclude/--no-ignore-global/--no-ignore-parent, and the -u/-uu/-uuu ladder (-uuu adds --binary, as in rg: binary files are searched and reported with a "binary file matches" notice). Each walks every operand on its own with rg's walker configuration, so --no-ignore-parent drops the ignore files above a directory operand, as in rg.
  • A file's hidden flag follows the ignore files on every sweep: adding or removing the ! rule that rescues a dotfile changes neither the dotfile nor its directory, and the index used to keep the old flag until the dotfile itself changed, so a dotfile whose rule was removed still showed up in searches and listings.
  • .rgignore files are honored like rg's: read wherever .ignore is (in the tree and in the directories above it, off under --no-ignore-dot), with rules that outrank .ignore and .gitignore at any depth, so a ! rule there brings back what those exclude and rescues dotfiles. They used to be ignored entirely. The macOS listing cache covers .rgignore files above the root in its fingerprint and re-stats a leaf directory's own .ignore, .rgignore and .gitignore (an empty one too), so an in-place edit of one is never served stale; caches written by older versions are rebuilt once.
  • --no-ignore-vcs turns off .git/info/exclude and the global gitignore (core.excludesFile) along with .gitignore, as in rg, and a later --ignore-exclude or --ignore-global does not turn either back on while VCS ignores are off. Both used to stay in force, so files only they excluded were still missing under --no-ignore-vcs.
  • A directory operand the index holds no file under is walked live with rg's rules, as rg walks any operand without ever filtering the operand itself: an ignored directory (glep pat build, or running inside one), a symlink to a directory, and a spelling that differs from the one on disk on a case-insensitive filesystem (glep pat SRC prints SRC/... like rg). Each used to print nothing and exit 1. Operands outside the index root (../../other from a subdirectory, or an absolute path elsewhere) are walked the same way.
  • rg's nothing-searched warning (exit 2) also fires from a subdirectory, where the cwd is the implicit scope, and the check stops at the first file that survives the filters instead of building the whole file list.
  • Missing path filters print glep: <path>: No such file or directory (os error 2) and exit 2; valid paths still produce their matches. Implicit-scope runs whose filters empty the walked pool print rg's "No files were searched" warning and exit 2.
  • Literal analysis is a recursive required-literal extractor over the parsed regex: adjacent literals fuse into longer substrings, required literals survive unexpandable spans ([0-9][0-9][0-9]-foo narrows on -foo), and literals on both sides of a wildcard are AND-required.
  • The required-literal planner stays linear in the pattern: a counted repetition expands at most 64 copies and requirement groups and literals are size-capped (each cut only narrows less). (?:a.){30000} spent 10 s planning before any search, and nested counts grew without bound; rg answers it in 0.07 s, and so does glep now.
  • Case-insensitive narrowing follows Unicode case folding, as rg's -i and -S match: ASCII k and s also match KELVIN SIGN (U+212A) and LONG S (U+017F), so the index looks a trigram holding them up with those spellings too (-i kfree missed a line spelled with the Kelvin sign, as it did in 0.3). A literal's ASCII trigrams also narrow next to non-ASCII text, where any non-ASCII literal used to force a full scan; --no-unicode keeps ASCII-only folding.
  • On macOS the bulk sweep and the listing cache apply .ignore files above the root themselves (ahead of every .gitignore, as in rg), and the cache's fingerprint covers them. Before, any such file, even an empty ~/.ignore, sent every query to the portable walker and bypassed the cache, which was then rewritten on every --files run: searches ran about 2x and listings about 8x slower.
  • Parallel first queries on a tree with no index yet all answer: while one glep builds the index under its lock, the others search with a live scan (what --no-index searches) instead of failing with another glep holds the index lock (exit 2). A torn index read while another glep is writing it is no longer rebuilt by the reader. glep index and glep status still report the busy lock.
  • The macOS listing cache serves root-wide listings only, since its pruned walk always covers the whole tree: a listing scoped to a path operand or to a subdirectory cwd goes through the index's scoped sweep, which walks just that subtree, and a scoped listing (or --files-without-match universe) clones only the files under its operands. On the kernel tree --files in rust/ went from 64 ms to 14 ms and --files '**/*.c' drivers/net (the Glob hook's form) from 63 ms to 25 ms.
  • The macOS listing cache is rebuilt only after a full, written sweep (never after a --ttl skip, a scoped sweep, or a lock-contended read-only pass), so a stale manifest is never re-certified by current directory mtimes.
  • A query that runs while another glep holds the index lock sweeps without writing the index, and it now answers for the tree that sweep found: a file deleted since the last written sweep is neither listed nor searched, and a changed file's old index entry no longer stands in for the file, which is searched as it is now. Such a query used to list deleted files under --files and --files-without-match and fail on them with No such file or directory (exit 2), which parallel agent searches right after a delete or a rename ran into, since one of them holds the lock for its whole query.

Agent hooks and tooling

  • The Claude Code and Cursor hooks map Grep's multiline to -U, and turn an offset of K and a head_limit of N into | tail -n +K+1 and | head -N pipes on the suggested command; all three used to be dropped. A content search with -n: false is left to the built-in tool, since the suggested command always prints line numbers. Both hooks allow a call whose tool_input is not an object instead of failing, and the Cursor hook also reads case_sensitive: false as -i and include as the glob.
  • bench/bench.sh compares against rg at its default parallelism with matching output and ignore flags, adds the -i and -C 2 rows, times fd when it is installed, and takes a listing glob argument. The new bench/compare.sh is the kernel-tree benchmark and differential harness: index build time and size, byte-exact stdout and exit-code checks against rg for each scenario and listing, GLEP_TIMING stage timings, and hyperfine runs.
  • The CLI runs on a thread with an 8 MiB stack, so Windows debug builds, whose main thread gets 1 MiB, no longer overflow it.
  • PCRE2 (new for -P) is built from source and linked statically: .cargo/config.toml sets PCRE2_SYS_STATIC=1. Without it pcre2-sys links a system libpcre2 whenever pkg-config finds one, and the macOS arm64 wheel, built on a runner with Homebrew's pcre2, would not start on a Mac without it.

pip install -U glep       # binary wheel, no Rust toolchain needed
cargo install glep        # crates.io

Standalone binaries for macOS (Apple silicon and Intel), Linux (x86_64 and aarch64, glibc; x86_64 musl) and Windows (x86_64) are attached below with SHA256SUMS: unpack and put glep on your PATH.

PyPI: https://pypi.org/project/glep/2.0.0/ · crates.io: https://crates.io/crates/glep/2.0.0 · Docs: https://velofy.co/glep/