regex: add --regextype=SHGLOB (GLOB + {a,b} brace alternation) (#129) - #334
Merged
Conversation
SHGLOB is a new match grammar: everything GLOB does, plus shell brace alternation, so
`*.{cc,h}` matches either extension. As a matcher (not a shell) a brace group becomes an
RE2 alternation - `*.{cc,h}` -> `[^/]*\.(?:cc|h)` - rather than expanding to several words.
Why a separate grammar and not a GLOB feature: GLOB and the gitignore engine must keep
matching literal `{`/`}` (a .gitignore line `foo{1}` matches literally), so brace
expansion has to be opt-in. Named SHGLOB ("shell glob"), deliberately NOT EXTGLOB - in bash
that names the pattern-lists ?(..)/@(..)/!(..), which we do not ship (the !(..) negation has
no clean RE2 form).
Semantics match bash: each alternative is itself SHGLOB-translated (nesting, `*`/`?`/`[...]`
inside; alts may contain `/` and `**`); a comma-less `{x}` or an unbalanced `{` stays a
literal brace; empty alternatives are allowed; `\{`/`\}`/`\,` escape to literals, as do
`{`/`}`/`,` inside a `[...]` class. Deferred: numeric/char sequences {1..9}/{a..z} and
extglob pattern-lists.
Implementation is DRY: xff::glob gains an internal flavor-aware TranslateInto shared by
GlobToRegex (braces off) and the new ShglobToRegex (braces on, recursive {...} parse); the
`**`/`[...]` logic (incl. the #128 class scanner) is not duplicated. Wired through
Grammar::kShglob (regex Compile), parser GrammarFromGlobals, run.cc ValidateRegextype
(SHGLOB is a core engine, always linked), and the globals/doc-renderer SOT.
Self-doc: --regextype gains a full details body documenting every grammar. It also gives
PCRE2 the note it was missing - PCRE2 is the one build-time extra, present only in a full
build, and `xff --help=extras` reports whether THIS binary has it (RE2/EXACT/FNMATCH/GLOB/
SHGLOB are always built in). Flows into --help=--regextype / --help=full / --man / --markdown.
Also resolves #122: glob.h documents why xff keeps its own gitignore-`**` translator rather
than migrating onto mbo::file::Glob2Re2 (different `**`; xff walks its own VFS engine).
Tests: glob_test (ShglobToRegex + GLOB-braces-literal cases), regex_test (kShglob compile +
match + path semantics), grep_test.sh (--regextype=SHGLOB end-to-end + GLOB literal-brace).
Full suite green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A new match grammar
--regextype=SHGLOB("shell glob"): everythingGLOBdoes, plus shell brace alternation, so*.{cc,h}matches either extension. As a matcher (not a shell) a brace group compiles to an RE2 alternation -*.{cc,h}->[^/]*\.(?:cc|h)- it matches any one alternative rather than expanding to several words.Why a separate grammar (not a GLOB feature)
GLOBand the gitignore engine must keep matching literal{/}(a.gitignorelinefoo{1}matches literally), so brace expansion has to be opt-in. Named SHGLOB, deliberately notEXTGLOB- in bash that names the pattern-lists?(..)/@(..)/!(..), which we do not ship (the!(..)negation has no clean RE2 form).Semantics (match bash)
*.{cc,h}[^/]*\.(?:cc|h){src,test}/**/x(?:src|test)/(?:.*/)?x/and**{a,{b,c}d}(?:a|(?:b|c)d){a,,b}(?:a||b){a}\{a\}\{a,b\}\{a,b\}\{ \} \,escape to literalsDeferred: numeric/char sequences
{1..9}/{a..z}, and the extglob pattern-lists.Implementation (DRY)
xff::globgains an internal flavor-awareTranslateIntoshared byGlobToRegex(braces off) and the newShglobToRegex(braces on, recursive{...}parse); the**/[...]logic (incl. the #128 class scanner) is not duplicated. Wired throughGrammar::kShglob(regexCompile), parserGrammarFromGlobals,run.ccValidateRegextype(SHGLOB is a core engine, always linked), and the globals/doc-renderer SOT.Self-documentation (+ the PCRE2 note)
--regextypegains a fulldetailsbody documenting every grammar. It also fixes a gap: the help always listed PCRE2 but never noted it may be absent. Now the details state PCRE2 is the one build-time extra, present only in a full build, and point toxff --help=extrasfor whether THIS binary has it (RE2/EXACT/FNMATCH/GLOB/SHGLOB are always built in). Flows into--help=--regextype/--help=full/--man/--markdown.Also resolves #122
glob.hdocuments why xff keeps its own gitignore-**translator rather than migrating ontombo::file::Glob2Re2(different**; xff walks its own VFS engine, needs only the pure pattern->RE2 step).Test
glob_test:ShglobToRegexalternation/nesting/degenerate cases + a GLOB-keeps-braces-literal guard.regex_test:kShglobcompile + match + retained GLOB path semantics.grep_test.sh:--regextype=SHGLOBend-to-end ({TODO,FIXME}) + GLOB literal-brace.//xff/...green.Closes #129. Resolves #122.