Multiple regex replacements at once

[blefloch]

In https://tex.stackexchange.com/questions/407637/ it would have been
convenient to replace all ( by \paren{ and all ) by } at once to
make parenthesis-matching rely on TeX's matching of braces. (Instead I
abused xparse's delimiter matching.) Unfortunately, l3regex only
allows to make one replacement at a time and this would result in an
unbalanced token list, rightfully refused by l3regex. I propose
\regex_replace_case_once:nN(TF) and \regex_replace_case_all:nN(TF)
that would perform several replacements at once:

\regex_replace_case_all:nN
  {
    { \( } { \c{paren} \cB\{ }
    { \) } { \cE\} }
  }
  \l_tmpa_tl

Not sure about the order of words in the function's name. Another
choice is to provide a replacement command that x-expands the
replacement parts (after finding the matches):

\regex_replace_all_x:nnN
  { [()] } % match open or close parenthesis
  {
    \c{if:w} \( \0
      \c{paren} \cB\{
    \c{else:}
      \cE\}
    \c{fi:}
  }

I might be missing some obvious alternatives and other possible interfaces.

[eg9]

On 30 Dec 2017, at 22:37 , Bruno Le Floch ***@***.***> wrote: In https://tex.stackexchange.com/questions/407637/ <https://tex.stackexchange.com/questions/407637/> it would have been convenient to replace all ( by \paren{ and all ) by } at once to make parenthesis-matching rely on TeX's matching of braces. (Instead I abused xparse's delimiter matching.) Unfortunately, l3regex only allows to make one replacement at a time and this would result in an unbalanced token list, rightfully refused by l3regex. I propose \regex_replace_case_once:nN(TF) and \regex_replace_case_all:nN(TF) that would perform several replacements at once: \regex_replace_case_all:nN { { \( } { \c{paren} \cB\{ } { \) } { \cE\} } } \l_tmpa_tl Not sure about the order of words in the function's name. Another choice is to provide a replacement command that x-expands the replacement parts (after finding the matches): \regex_replace_all_x:nnN { [()] } % match open or close parenthesis { \c{if:w} \( \0 \c{paren} \cB\{ \c{else:} \cE\} \c{fi:} } I might be missing some obvious alternatives and other possible interfaces.

I find this an interesting idea; I’ve already fought with this problem, solving it by doing recursive replacements. There are possible race conditions, though. Ciao Enrico

[blefloch]

There are possible race conditions, though.

Could you elaborate please? Do you mean the question of whether the replacements are done one after the other through the whole token list, versus doing them simultaneously as if looking for matches of "regex_1|regex_2|...|regex_n"? I strongly prefer the latter, partly because it seems easier to implement from what I remember of the code, and partly because it seems much more natural: to exchange x and y one would then just need `{x} {y} {y} {x}` and not the more convoluted `{x} {z} {y} {x} {z} {y}` hoping that `z` does not appear in the token list.

[eg9]

On 31 Dec 2017, at 02:58 , Bruno Le Floch ***@***.***> wrote: > There are possible race conditions, though. Could you elaborate please? Do you mean the question of whether the replacements are done one after the other through the whole token list, versus doing them simultaneously as if looking for matches of "regex_1|regex_2|...|regex_n"? I strongly prefer the latter, partly because it seems easier to implement from what I remember of the code, and partly because it seems much more natural: to exchange x and y one would then just need `{x} {y} {y} {x}` and not the more convoluted `{x} {z} {y} {x} {z} {y}` hoping that `z` does not appear in the token list.

I can imagine the latter substitution restoring part of what the former is replacing; depending on the implementation this might be disallowed, or left to the user as an interesting exercise in finding where the infinite loop is popping out. ;-) I agree with {x}{y}{y}{x}, if feasible. Ciao Enrico

[blefloch]

I can imagine the latter substitution restoring part of what the former is replacing; depending on the implementation this might be disallowed, or left to the user as an interesting exercise in finding where the infinite loop is popping out. ;-) I agree with {x}{y}{y}{x}, if feasible.

It's feasible: basically use the automaton that would be built for x|y and then have a replacement that depends on which branch was taken. It's actually trickier because group numbers need to be appropriately reset for each branch.

[blefloch]

Using the same machinery we should provide (each of the regex should be allowed to be an N-type regex: the overhead from testing that is tiny compared to the regex matching machinery).

• \regex_case_match:nn { { regex1 } { code1 } { regex2 } { code 2 } ... } { tl } and TF version
• \regex_case_count:n { { regex1 } \l_A_int { regex2 } \l_B_int ... } { tl }
• \regex_case_replace_once:nN { { regex1 } { replace1 } ... } \l_my_tl and TF version
• \regex_case_replace_all:nN { { regex1 } { replace1 } ... } \l_my_tl and TF version

Maybe the following (but I don't feel that this is quite the right syntax). No analogue of \regex_extract_once:nnN nor \regex_split:nnN because I don't see how to make it different from a larger regex regex1|regex2|....

• \regex_case_extract_all:nnN { { regex1 } \l_A_seq { regex2 } \l_B_seq ... } { tl } \l_full_seq

The implementation strategy is roughly speaking to make a big regex (?| (?:regex1) | (?:regex2) | ... ) but change how the final state in the automaton is handled, so that it keeps track of which branch succeeded. This needs 3-5 hours of code/test/docu.

[blefloch]

Hum... It is clear that \regex_case_count:n { { A } \l_a_int { B } \l_B_int } { B A X } should store 1 and 1 in the two ints so the two regexes have to be searched at the same time and matches should be counted along the way as we find them. This means that \regex_case_count:n { { A . } \l_a_int { B . } \l_B_int } { B A X } should store 0 and 1 in the two ints (first BA is found as a match for the second regex, then there isn't any A left).

But for the case_match function it is much less clear what \regex_case_match:nn { { A . } { code-A } { B . } { code-B } } { B A X } should do. Should it work consistently with the case_count function, hence run code-B? Should it test regexes one after the other, hence match A . later in the string and run code-A?

[blefloch]

In my pull request #928 I'm not implementing analogues of regex_count or regex_extract_all because it is very unclear what should be the syntax: it seems too messy to require the user to use one variable to get the result of each case. No analogue of regex_extract_once or regex_split because it is even less clear what to do.