Substitution cannot proceed if "this" field, "that" field, and the text all contain "\cite"

[saxyx]

44.tex code

\cite

localSettings.yaml

replacements:
  -
this: \cite
that: ref \cite

actual/given output

E:\Personal\Desktop\rr>latexindent -w -r -sl -o=++ -l=localSettings.yaml 44.tex
INFO: latexindent.pl version 3.23.4, 2023-11-19, a script to indent .tex files
latexindent.pl lives here: C:/CTEX/MiKTeX/scripts/latexindent/
Tue Dec 19 17:08:09 2023
Filename: 44.tex
INFO: Processing switches:
-sl|--screenlog: log file will also be output to the screen
-w|--overwrite: Overwrite mode active, will make a back up before overwriting
-l|--localSettings: Read localSettings YAML file
-o|--outputfile: output to file
-r|--replacement: replacement mode
INFO: Options check: -w and -o specified
You have called latexindent.pl with both -o and -w
The -o switch will take priority, and -w (overwrite) will be ignored
INFO: Directory for backup files and indent.log:
.
INFO: Perl modules are being loaded from the following directories:
E:/program/Strawberry/perl/lib/FindBin.pm
E:/program/Strawberry/perl/vendor/lib/YAML/Tiny.pm
E:/program/Strawberry/perl/lib/File/Copy.pm
E:/program/Strawberry/perl/lib/File/Basename.pm
E:/program/Strawberry/perl/lib/Getopt/Long.pm
E:/program/Strawberry/perl/vendor/lib/File/HomeDir.pm
INFO: LatexIndent perl modules are being loaded from, for example:
C:/CTEX/MiKTeX/scripts/latexindent/LatexIndent/Document.pm
INFO: YAML settings read: defaultSettings.yaml
Reading defaultSettings.yaml from C:/CTEX/MiKTeX/scripts/latexindent/defaultSettings.yaml
INFO: YAML reading settings
Home directory is C:\Users\lenovo
latexindent.pl didn't find indentconfig.yaml or .indentconfig.yaml
see all possible locations: https://latexindentpl.readthedocs.io/en/latest/sec-appendices.html#indentconfig-options)
INFO: YAML settings read: -l switch
Adding ./localSettings.yaml to YAML read paths
INFO: YAML settings, reading from the following files:
Reading USER settings from ./localSettings.yaml
---
replacements:
-
that: 'ref \cite'
this: \cite

INFO: -o switch active: output file check
-o switch called with file name without extension: ++
Updated to ++.tex as the file extension of the input file is tex
-o switch called with file name ending with ++: ++.tex
will search for existence and increment counter, starting with 440.tex
440.tex exists, incrementing counter
441.tex exists, incrementing counter
442.tex exists, incrementing counter
443.tex exists, incrementing counter
444.tex does not exist, and will be the output file

desired or expected output

ref \cite

[cmhughes]

Sorry for the delay, everything is a bit slow at the moment but I hope to get to this soon.

[saxyx]

Take your time, there's no rush. I appreciate your attention to this issue and I'll be patiently waiting for your further updates.

[cmhughes]

Thanks for this. Yes, at present, your example highlights a known issue.

workaround

The workaround is to use

replacements:
  -
   substitution: s/\\cite/ref \\cite/sg

for future

As of 27a5bdd there's no need for the workaround. This will be part of the next release, which I hope to get released soon.

[cmhughes]

Released at https://github.com/cmhughes/latexindent.pl/releases/tag/V3.23.5, uploaded to ctan. Thanks again.

[fengzyf]

Released at https://github.com/cmhughes/latexindent.pl/releases/tag/V3.23.5, uploaded to ctan. Thanks again.

Updating to latexindent 3.23.5, I found a new problem. The details are as follows.

original .tex code

XXXaaa
XaXaXXX

yaml settings

replacements:
  -
    this: 'X'
    that: 'Y'

actual/given output

YYYaaa
YaYa 

desired or expected output

YYYaaa
YaYaYYY

[cmhughes]

I'm unable to produce your given output; I receive the desired output:

YYYaaa
YaYaYYY

[fengzyf]

I'm unable to produce your given output; I receive the desired output:

YYYaaa
YaYaYYY

My operating system is Win10, the system language is Chinese. I recorded a video and the problem I was talking about actually happened.

2024-01-02.mp4

[cmhughes]

Your local settings are not being loaded, as detailed in your log file. This means you are not calling latexindent correctly.

…

On Tue, 2 Jan 2024, 02:57 fengzyf, ***@***.***> wrote: I'm unable to produce your given output; I receive the desired output: YYYaaa YaYaYYY My operating system is Win10, the system language is Chinese. I recorded a video and the problem I was talking about actually happened. https://github.com/cmhughes/latexindent.pl/assets/154897680/82806c86-4f9e-4eec-827c-9eee4da58779 — Reply to this email directly, view it on GitHub <#503 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAQ7CYF7Y2G4MALXKXRR5X3YMNZRNAVCNFSM6AAAAABA226DXKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZTGU4DGNJSHA> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

[cmhughes]

My apologies, please ignore my previous message, I was incorrect.

[cmhughes]

Can you post your log file here please?

[fengzyf]

INFO:  latexindent.exe version 3.23.5, 2024-01-01, a script to indent .tex files
       latexindent.exe lives here: D:/rr/
       Tue Jan  2 19:33:43 2024
       Filename: 44.tex
INFO:  Processing switches:
       -s|--silent: Silent mode active (you have used either -s or --silent)
       -o|--outputfile: output to file
       -r|--replacement: replacement mode
INFO:  Directory for backup files and indent.log:
       .
INFO:  YAML settings read: defaultSettings.yaml
       Reading defaultSettings.yaml from D:/rr/defaultSettings.yaml
       Reading defaultSettings.yaml (2nd attempt) from D:/rr/../../texmf-dist/scripts/latexindent/defaultSettings.yaml
       and then, if necessary, D:/rr/LatexIndent/defaultSettings.yaml
INFO:  YAML reading settings
       Home directory is C:\Users\lenovo
       latexindent.pl didn't find indentconfig.yaml or .indentconfig.yaml
       see all possible locations: https://latexindentpl.readthedocs.io/en/latest/sec-appendices.html#indentconfig-options)
INFO:  -l switch used without filename, will search for the following files in turn:
       localSettings.yaml,latexindent.yaml,.localSettings.yaml,.latexindent.yaml
INFO:  YAML settings read: -l switch
       Multiple localSettings found, separated by commas:
       localSettings.yaml, latexindent.yaml, .localSettings.yaml, .latexindent.yaml
       Adding ./localSettings.yaml to YAML read paths
INFO:  YAML settings, reading from the following files:
       Reading USER settings from ./localSettings.yaml
       ---
       replacements:
         -
           that: Y
           this: X

INFO:  -o switch active: output file check
       -o switch called with file name without extension: ++
       Updated to ++.tex as the file extension of the input file is tex
       -o switch called with file name ending with ++: ++.tex
       will search for existence and increment counter, starting with 440.tex
       440.tex does not exist, and will be the output file
INFO:  Phase 1: searching for objects
       No objects found.
INFO:  Phase 2: finding surrounding indentation
INFO:  Phase 3: indenting objects
INFO:  Phase 4: final indentation check
INFO:  Output routine:
       Outputting to file 440.tex
       --------------
INFO:  Please direct all communication/issues to:
        https://github.com/cmhughes/latexindent.pl

[cmhughes]

How are you calling latexindent?

[fengzyf]

Run the following command in cmd.
latexindent -s -l -r -o=++ 44.tex

[cmhughes]

Can you run

latexindent -tt -s -l -r -o=++ 44.tex

and post the log file?

[fengzyf]

INFO:  latexindent.exe version 3.23.5, 2024-01-01, a script to indent .tex files
       latexindent.exe lives here: D:/rr/
       Wed Jan  3 20:43:48 2024
       Filename: 44.tex
INFO:  Processing switches:
       -tt|--ttrace: TTrace mode active (you have used either -tt or --ttrace)
       -s|--silent: Silent mode active (you have used either -s or --silent)
       -o|--outputfile: output to file
       -r|--replacement: replacement mode
INFO:  Directory for backup files and indent.log:
       .
INFO:  Perl modules are being loaded from the following directories:
       E:\Personal\Temp\par-6c656e6f766f\cache-df2935a61bebb4ec4b36991156a66d318a6f0b0e\inc\lib/FindBin.pm
       E:\Personal\Temp\par-6c656e6f766f\cache-df2935a61bebb4ec4b36991156a66d318a6f0b0e\inc\lib/YAML/Tiny.pm
       /loader/HASH(0x6dc1d8)/File/Copy.pm
       /loader/HASH(0x6dc148)/File/Basename.pm
       E:\Personal\Temp\par-6c656e6f766f\cache-df2935a61bebb4ec4b36991156a66d318a6f0b0e\inc\lib/Getopt/Long.pm
       E:\Personal\Temp\par-6c656e6f766f\cache-df2935a61bebb4ec4b36991156a66d318a6f0b0e\inc\lib/File/HomeDir.pm
INFO:  LatexIndent perl modules are being loaded from, for example:
       E:\Personal\Temp\par-6c656e6f766f\cache-df2935a61bebb4ec4b36991156a66d318a6f0b0e\inc\lib/LatexIndent/Document.pm
INFO:  YAML settings read: defaultSettings.yaml
       Reading defaultSettings.yaml from D:/rr/defaultSettings.yaml
       Reading defaultSettings.yaml (2nd attempt) from D:/rr/../../texmf-dist/scripts/latexindent/defaultSettings.yaml
       and then, if necessary, D:/rr/LatexIndent/defaultSettings.yaml
INFO:  YAML reading settings
       Home directory is C:\Users\lenovo
       latexindent.pl didn't find indentconfig.yaml or .indentconfig.yaml
       see all possible locations: https://latexindentpl.readthedocs.io/en/latest/sec-appendices.html#indentconfig-options)
INFO:  -l switch used without filename, will search for the following files in turn:
       localSettings.yaml,latexindent.yaml,.localSettings.yaml,.latexindent.yaml
INFO:  YAML settings read: -l switch
       Multiple localSettings found, separated by commas:
       localSettings.yaml, latexindent.yaml, .localSettings.yaml, .latexindent.yaml
       Adding ./localSettings.yaml to YAML read paths
INFO:  YAML settings, reading from the following files:
       Reading USER settings from ./localSettings.yaml
       ---
       replacements:
         -
           that: Y
           this: X

INFO:  -o switch active: output file check
       -o switch called with file name without extension: ++
       Updated to ++.tex as the file extension of the input file is tex
       -o switch called with file name ending with ++: ++.tex
       will search for existence and increment counter, starting with 440.tex
       440.tex does not exist, and will be the output file
TRACE: Token check
TRACE: Replacement mode *before* indentation: -r
       -
       this: X
       that: Y
       when: before
TRACE: List of items: item|myitem (see itemNames)
TRACE: Constructing specialBeginEnd regex (see specialBeginEnd)
TRACE: The special beginnings regexp is: (see specialBeginEnd)
       \$\$|(?<!\\)\\\[|(?<!\$)(?<!\\)\$(?!\$)
TRACE: The overall special regexp is: (see specialBeginEnd)
       (?^usx:
                                         \$\$
                                         (?^usx:(?:                        # cluster-only (), don't capture 
                                               (?!             
                                                   (?:\$\$|(?<!\\)\\\[|(?<!\$)(?<!\\)\$(?!\$)) # cluster-only (), don't capture
                                               ).                     # any character, but not anything in $specialBegins
                                         )*?) 
                                         \$\$
                                  )|(?^usx:
                                         (?<!\\)\\\[
                                         (?^usx:(?:                        # cluster-only (), don't capture 
                                               (?!             
                                                   (?:\$\$|(?<!\\)\\\[|(?<!\$)(?<!\\)\$(?!\$)) # cluster-only (), don't capture
                                               ).                     # any character, but not anything in $specialBegins
                                         )*?) 
                                         \\\]
                                  )|(?^usx:
                                         (?<!\$)(?<!\\)\$(?!\$)
                                         (?^u:[^$]*?) 
                                         (?<!\\)\$(?!\$)
                                  )
TRACE: Constructing headings reg exp for example, chapter, section, etc (see indentAfterThisHeading)
       Not indenting after paragraph (see indentAfterThisHeading)
       Not indenting after section (see indentAfterThisHeading)
       Not indenting after chapter (see indentAfterThisHeading)
       Not indenting after part (see indentAfterThisHeading)
       Not indenting after subparagraph (see indentAfterThisHeading)
       Not indenting after subsection* (see indentAfterThisHeading)
       Not indenting after subsection (see indentAfterThisHeading)
       Not indenting after subsubsection (see indentAfterThisHeading)
TRACE: Looping through array for commandCodeBlocks->stringsAllowedBetweenArguments
       node
       at
       to
       decoration
       \+\+
       \-\-
       \#\#\d
TRACE: Strings allowed between arguments: (?^u:node|at|to|decoration|\+\+|\-\-|\#\#\d) (see stringsAllowedBetweenArguments)
TRACE: Looping through array for commandCodeBlocks->stringsAllowedBetweenArguments
       node
       at
       to
       decoration
       \+\+
       \-\-
       \#\#\d
TRACE: Strings allowed between arguments: (?^u:node|at|to|decoration|\+\+|\-\-|\#\#\d) (see stringsAllowedBetweenArguments)
TRACE: Looping through array for commandCodeBlocks->commandNameSpecial
       @ifnextchar\[
TRACE: The special command names regexp is: (?^u:@ifnextchar\[) (see commandNameSpecial)
TRACE: Searching for NOINDENTBLOCK (see noIndentBlock)
       {
         cmhtest => 1,
         noindent => 1
       }

       looking for cmhtest:1 noIndentBlock
       looking for noindent:1 noIndentBlock
TRACE: Searching for VERBATIM commands (see verbatimCommands)
       {
         lstinline => 1,
         verb => 1
       }

       looking for verb:1 Commands
       looking for lstinline:1 Commands
TRACE: Storing trailing comments
       No trailing comments found
TRACE: Searching for VERBATIM environments (see verbatimEnvironments)
       {
         lstlisting => 1,
         minted => 1,
         verbatim => 1
       }

       looking for verbatim:1 environments
       looking for lstlisting:1 environments
       looking for minted:1 environments
TRACE: Verbatim storage:
       {}

TRACE: Horizontal space removal routine
TRACE: Searching for FILE CONTENTS environments (see fileContentsEnvironments)
       {
         filecontents => 1,
         "filecontents*" => 1
       }

       looking for filecontents* environments
       looking for filecontents environments
TRACE: Removing leading space from mainDocument (verbatim/noindentblock already accounted for)
INFO:  Phase 1: searching for objects
TRACE: looking for ENVIRONMENTS
TRACE: looking for IFELSEFI
TRACE: looking for HEADINGS (chapter, section, part, etc)
       No objects found.
INFO:  Phase 2: finding surrounding indentation
TRACE: FamilyTree before update:
       {}

TRACE: Updating FamilyTree...
TRACE: FamilyTree after update:
       {}

INFO:  Phase 3: indenting objects
       No child objects (mainDocument)
INFO:  Phase 4: final indentation check
TRACE: Horizontal space removal routine
       Removing trailing white space *after* the document is processed (see removeTrailingWhitespace: afterProcessing)
TRACE: Replacement mode *after* indentation: -r
INFO:  Output routine:
       Outputting to file 440.tex
       --------------
INFO:  Please direct all communication/issues to:
        https://github.com/cmhughes/latexindent.pl

[cmhughes]

Can you try

replacements:
  -
    this: X
    that: Y

[fengzyf]

The actual output is still

YYYaaa
YaYa

[cmhughes]

I believe this may be a windows line break issue. Try adding any character at the end of your file and run again.

…

On Wed, 3 Jan 2024, 14:13 fengzyf, ***@***.***> wrote: The actual output is still YYYaaa YaYa — Reply to this email directly, view it on GitHub <#503 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAQ7CYH54RNNIAC552JT53LYMVRPFAVCNFSM6AAAAABA226DXKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZVGQZTKNRZHA> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

[fengzyf]

Yes, that's true. After adding a space at the end of my file, the output is

YYYaaa
YaYaYYY

[fengzyf]

I believe this may be a windows line break issue. Try adding any character at the end of your file and run again.

I found the cause of the problem from the introduction of split in the official documentation

split /PATTERN/,EXPR,LIMIT

If LIMIT is negative, it is treated as if it were instead arbitrarily large; as many fields as possible are produced.

If LIMIT is omitted (or, equivalently, zero), then it is usually treated as if it were instead negative but with the exception that trailing empty fields are stripped (empty leading fields are always preserved); if all fields are empty, then all fields are considered to be trailing (and are thus stripped in this case). Thus, the following:

my @x = split(/,/, "a,b,c,,,"); # ("a", "b", "c")

produces only a three element list.

my @x = split(/,/, "a,b,c,,,", -1); # ("a", "b", "c", "", "", "")

produces a six element list.

As another special case, split emulates the default behavior of the command line tool awk when the PATTERN is either omitted or a string composed of a single space character (such as ' ' or "\x20", but not e.g. / /). In this case, any leading whitespace in EXPR is removed before splitting occurs, and the PATTERN is instead treated as if it were /\s+/; in particular, this means that any contiguous whitespace (not just a single space character) is used as a separator.

my @x = split(" ", "  Quick brown fox\n");
# ("Quick", "brown", "fox")


my @x = split(" ", "RED\tGREEN\tBLUE");
# ("RED", "GREEN", "BLUE")

---

Therefore, on line 74 of Replacement.pm

${$self}{body} = join( $that, split( $this, ${$self}{body} ) );

should be replaced with

${$self}{body} = join( $that, split( /$this/, ${$self}{body}, -1 ) );

[cmhughes]

Thanks, that's helpful. I've implemented your change as of eaed6db I'll get it released soon. thanks again

[fengzyf]

I appreciate it. Glad I could help.

[cmhughes]

released at https://github.com/cmhughes/latexindent.pl/releases/tag/V3.23.6, thanks again