Peek regex #779
Merged
Peek regex #779
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
josephwright
approved these changes
Nov 5, 2020
There was a problem hiding this comment.
This all looks good to me. I think the names \peek_regex:n(TF), etc., are good. I guess \peek_analysis_map_inline:n is slightly awkward, but I don't see a better one - iot's quite specialised anyway.
Overall I think we go for this then can move to extending xparse.
This is just code reorganisation, aiming to be able to peek ahead in the input stream, for which it is more convenient to look at tokens one at a time.
The eventual aim is to let users run code while the regex is being matched. For this we need to remove all abuses of TeX registers. The only one left is toks used for storing states of the nfa.
The aim is to remove uses of max_pos, because that will not be available when peeking for a regex
These functions check what follows in the input stream, but they do not give access to it in the T and F branches. I'll also add \peek_regex_replace:nnTF that also performs a replacement on the tokens found. It might be nice to store the tokens found into some token list, but unfortunately they can very well have unbalanced braces.
josephwright
force-pushed
the
peek-regex
branch
from
ae0dfd3
to
c390337
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This branch (which branches out of the
peek-analysisbranch) implements ways to peek ahead into the input stream. The code should be ok, but I am not sure if the interface I chose is ok.I implemented
\peek_regex:nTF,\peek_regex_remove_once:nTF, and\peek_regex_replace_once:nnTF, and the same with first argument being a regex variable instead of an n-type regex. I provided\peek_regex_replace_once:nn, but not the other "no TF" versions of the other functions because they don't seem really sensible. In fact,\peek_regex:nTFand\peek_regex_remove_once:nTFcould be implemented asso maybe the
remove_oncefunctions could be omitted. We could also reasonably provide\peek_regex_replace_all:nnTFwhich would repeatedly search for the pattern.Perhaps tests I put in
m3regex011are not thorough enough.