[latexindent.exe] The -w option does not support Chinese filename

[fengzyf]

Please provide the following when posting an issue:

新建.tex code

test.

actual/given output

D:\test>latexindent -w "新建.tex"
FATAL Could not write to backup file ./�½�.bak0. Please check permissions.
Exiting, no indentation done.

[cmhughes]

Thanks for this.

I am unable to reproduce the problem.

On my Ubuntu machine, when I start with your file, 新建.tex and I run

latexindent.pl -w 新建.tex -g issue-505.log -y 'onlyOneBackUp:1'

then the log file is

INFO:  latexindent.pl version 3.23.4, 2023-11-19, a script to indent .tex files
       latexindent.pl lives here: /home/cmhughes/projects/latexindent/
       Wed Dec 27 10:43:53 2023
       Filename: 新建.tex
INFO:  Processing switches:
       -w|--overwrite: Overwrite mode active, will make a back up before overwriting
       -y|--yaml: YAML settings specified via command line
       -g|--logfile: logfile name
INFO:  Directory for backup files and issue-505.log:
       .
INFO:  Perl modules are being loaded from the following directories:
       /home/cmhughes/perl5/perlbrew/perls/perl-5.36.0/lib/5.36.0/FindBin.pm
       /home/cmhughes/perl5/perlbrew/perls/perl-5.36.0/lib/site_perl/5.36.0/YAML/Tiny.pm
       /home/cmhughes/perl5/perlbrew/perls/perl-5.36.0/lib/5.36.0/File/Copy.pm
       /home/cmhughes/perl5/perlbrew/perls/perl-5.36.0/lib/5.36.0/File/Basename.pm
       /home/cmhughes/perl5/perlbrew/perls/perl-5.36.0/lib/5.36.0/Getopt/Long.pm
       /home/cmhughes/perl5/perlbrew/perls/perl-5.36.0/lib/site_perl/5.36.0/File/HomeDir.pm
INFO:  LatexIndent perl modules are being loaded from, for example:
       /home/cmhughes/projects/latexindent/LatexIndent/Document.pm
INFO:  YAML settings read: defaultSettings.yaml
       Reading defaultSettings.yaml from /home/cmhughes/projects/latexindent/defaultSettings.yaml
INFO:  YAML reading settings
       Home directory is /home/cmhughes
       latexindent.pl didn't find indentconfig.yaml or .indentconfig.yaml
       see all possible locations: https://latexindentpl.readthedocs.io/en/latest/sec-appendices.html#indentconfig-options)
INFO:  YAML settings read: -y switch
       YAML setting: onlyOneBackUp:1
       Updating mainSettings with onlyOneBackUp: 1
INFO:  Backup procedure (-w flag active):
       ./新建.bak will be overwritten (see onlyOneBackUp)
       Backing up 新建.tex to ./新建.bak...
       新建.tex will be overwritten after indentation
INFO:  Phase 1: searching for objects
       No objects found.
INFO:  Phase 2: finding surrounding indentation
INFO:  Phase 3: indenting objects
INFO:  Phase 4: final indentation check
INFO:  Output routine:
       Overwriting file 新建.tex
       --------------
INFO:  Please direct all communication/issues to:
        https://github.com/cmhughes/latexindent.pl

Please can you provide more details?

[fengzyf]

My OS: Windows 10 22H2 19045.3803 (Chinese version)
My latexindent.exe version: 3.23.4, 2023-11-19

When I run

latexindent -w 新建.tex -g issue-505.log -y 'onlyOneBackUp:1'

then the log file is

INFO:  latexindent.exe version 3.23.4, 2023-11-19, a script to indent .tex files
       latexindent.exe lives here: D:/test/
       Wed Dec 27 22:32:48 2023
       Filename: �½�.tex
INFO:  Processing switches:
       -w|--overwrite: Overwrite mode active, will make a back up before overwriting
       -y|--yaml: YAML settings specified via command line
       -g|--logfile: logfile name
INFO:  Directory for backup files and issue-505.log:
       .
INFO:  YAML settings read: defaultSettings.yaml
       Reading defaultSettings.yaml from D:/test/defaultSettings.yaml
       Reading defaultSettings.yaml (2nd attempt) from D:/test/../../texmf-dist/scripts/latexindent/defaultSettings.yaml
       and then, if necessary, D:/test/LatexIndent/defaultSettings.yaml
INFO:  YAML reading settings
       Home directory is C:\Users\lenovo
       latexindent.pl didn't find indentconfig.yaml or .indentconfig.yaml
       see all possible locations: https://latexindentpl.readthedocs.io/en/latest/sec-appendices.html#indentconfig-options)
INFO:  YAML settings read: -y switch
       YAML setting: 'onlyOneBackUp:1'
       Updating mainSettings with 'onlyOneBackUp: 1'
INFO:  Backup procedure (-w flag active):
       Backing up �½�.tex to ./�½�.bak0...
       �½�.tex will be overwritten after indentation
FATAL Could not write to backup file ./�½�.bak0. Please check permissions.
      Exiting, no indentation done.
       --------------
INFO:  Please direct all communication/issues to:
        https://github.com/cmhughes/latexindent.pl

[cmhughes]

What happens if your file is named aaaa.tex?

[fengzyf]

The process completed successfully, generating aaaa.bak0.

[cmhughes]

How does the o switch behave?

Can you post the output from

latexindent.pl 新建.tex -g issue-505.log -o 新建.tex

[fengzyf]

INFO: latexindent.pl version 3.23.5, 2024-01-01, a script to indent .tex files
latexindent.pl lives here: C:/CTEX/MiKTeX/scripts/latexindent/
Sun Jan 7 22:52:12 2024
Filename: �½�.tex
INFO: Processing switches:
-o|--outputfile: output to file
-g|--logfile: logfile name
INFO: Directory for backup files and issue-505.log:
.
INFO: Perl modules are being loaded from the following directories:
E:/program/Strawberry/perl/lib/FindBin.pm
E:/program/Strawberry/perl/vendor/lib/YAML/Tiny.pm
E:/program/Strawberry/perl/lib/File/Copy.pm
E:/program/Strawberry/perl/lib/File/Basename.pm
E:/program/Strawberry/perl/lib/Getopt/Long.pm
E:/program/Strawberry/perl/vendor/lib/File/HomeDir.pm
INFO: LatexIndent perl modules are being loaded from, for example:
C:/CTEX/MiKTeX/scripts/latexindent/LatexIndent/Document.pm
INFO: YAML settings read: defaultSettings.yaml
Reading defaultSettings.yaml from C:/CTEX/MiKTeX/scripts/latexindent/defaultSettings.yaml
INFO: YAML reading settings
Home directory is C:\Users\lenovo
latexindent.pl didn't find indentconfig.yaml or .indentconfig.yaml
see all possible locations: https://latexindentpl.readthedocs.io/en/latest/sec-appendices.html#indentconfig-options)
INFO: -o switch active: output file check
INFO: Phase 1: searching for objects
No objects found.
INFO: Phase 2: finding surrounding indentation
INFO: Phase 3: indenting objects
INFO: Phase 4: final indentation check
INFO: Output routine:
Outputting to file �½�.tex
--------------
INFO: Please direct all communication/issues to:
https://github.com/cmhughes/latexindent.pl

[fengzyf]

According to the code in #263 (comment)

#!perl -w
use autodie;
use Encode;

my $file ="./测试.txt";
# We have to first decode utf-8 because perl do not use utf-8 in default.
# We can `use utf8;` to set utf-8 encoding into default.
$file = encode("gb2312", decode("utf-8", $file));
open my $in,'<',"$file";

while (<$in>){
print;
}

In the Chinese version of Windows, it appears that perl receives commands from cmd encoded in GB2312 instead of utf-8. Therefore, you need to use a command like this to convert the encoding:

decode("GB2312", ${$self}{fileName} )

And when it comes to file operations, the string needs to be converted back to GB2312 encoding. Such as:

copy( encode("GB2312", $fileName), encode("GB2312", $backupFile)

The following example confirms this.

If BackUpFileProcedure.pm is changed to the following file
BackUpFileProcedure.txt

When I run

latexindent -w 新建.tex -g issue-505.log

the bak file is generated as normal and the log file is

INFO:  latexindent.pl version 3.23.5, 2024-01-01, a script to indent .tex files
       latexindent.pl lives here: C:/CTEX/MiKTeX/scripts/latexindent/
       Sun Jan  7 23:33:34 2024
       Filename: �½�.tex
INFO:  Processing switches:
       -w|--overwrite: Overwrite mode active, will make a back up before overwriting
       -g|--logfile: logfile name
INFO:  Directory for backup files and issue-505.log:
       .
INFO:  Perl modules are being loaded from the following directories:
       E:/program/Strawberry/perl/lib/FindBin.pm
       E:/program/Strawberry/perl/vendor/lib/YAML/Tiny.pm
       E:/program/Strawberry/perl/lib/File/Copy.pm
       E:/program/Strawberry/perl/lib/File/Basename.pm
       E:/program/Strawberry/perl/lib/Getopt/Long.pm
       E:/program/Strawberry/perl/vendor/lib/File/HomeDir.pm
INFO:  LatexIndent perl modules are being loaded from, for example:
       C:/CTEX/MiKTeX/scripts/latexindent/LatexIndent/Document.pm
INFO:  YAML settings read: defaultSettings.yaml
       Reading defaultSettings.yaml from C:/CTEX/MiKTeX/scripts/latexindent/defaultSettings.yaml
INFO:  YAML reading settings
       Home directory is C:\Users\lenovo
       latexindent.pl didn't find indentconfig.yaml or .indentconfig.yaml
       see all possible locations: https://latexindentpl.readthedocs.io/en/latest/sec-appendices.html#indentconfig-options)
INFO:  Backup procedure (-w flag active):
       Backing up 新建.tex to ./新建.bak0...
       新建.tex will be overwritten after indentation
INFO:  Phase 1: searching for objects
       No objects found.
INFO:  Phase 2: finding surrounding indentation
INFO:  Phase 3: indenting objects
INFO:  Phase 4: final indentation check
INFO:  Output routine:
       Overwriting file �½�.tex
       --------------
INFO:  Please direct all communication/issues to:
        https://github.com/cmhughes/latexindent.pl

Therefore, I believe that you can refer to #263 and
#273 to add a switch to the user to specify their encoding.

[cmhughes]

That's helpful many thanks.

[cmhughes]

For my reference, this file will need updating

https://github.com/cmhughes/latexindent.pl/blob/main/.github/workflows/batch-check.yaml

[cmhughes]

Implemented as of f75a818

Please leave this open until I have released it.

[cmhughes]

Thanks again, released at https://github.com/cmhughes/latexindent.pl/releases/tag/V3.23.7