po4a

#! /usr/bin/env perl
eval 'exec perl -S $0 ${1+"$@"}'
  if $running_under_some_shell;

# po4a -- Update both the po files and translated documents in one shoot
#
# Copyright 2002-2023 by SPI, inc.
#
# This program is free software; you can redistribute it and/or modify it
# under the terms of GPL v2.0 or later (see COPYING).

=encoding UTF-8

=head1 NAME

po4a - update both the PO files and translated documents in one shot

=head1 SYNOPSIS

B<po4a> [I<options>] I<config_file>

=head1 DESCRIPTION

po4a (PO for anything) eases the maintenance of documentation translation
using the classical gettext tools. The main feature of po4a is that
it decouples the translation of content from its document structure.
Please refer to the page L<po4a(7)> for a gentle introduction to this
project.

Upon execution, B<po4a> parses all documentation files specified in its
configuration file. It updates the PO files (containing the translation) to
reflect any change to the documentation, and produce a translated documentation
by injecting the content's translation (found in the PO files) into the
structure of the original master document.

At first, the PO files only contain the strings to translate from the original
documentation. This file format allows the translators to manually provide a
translation for each paragraph extracted by B<po4a>. If the documentation is
modified after translation, B<po4a> marks the corresponding translations as
"fuzzy" in the PO file to request a manual review by the translators. The
translators can also provide so-called "addendum", that are extra content
stating for example who did the translation and how to report bugs.

 master documents ---+---->-------->---------+
  (doc authoring)    |                       |
                     V   (po4a executions)   >-----+--> translated
                     |                       |     |     documents
 existing PO files -->--> updated PO files >-+     |
      ^                            |               |
      |                            V               |
      +----------<---------<-------+               ^
       (manual translation process)                |
                                                   |
 addendum -->--------------------------------------+

The workflow of B<po4a> is asynchronous, as suited to open-source projects. The
documentation writers author the master documents at their own pace. The
translators review and update the translations in the PO files. The maintainers
rerun B<po4a> on need, to reflect any change to the original documentation to
the PO files, and to produce updated documentation translations, by injecting
the latest translation into the latest document structure.

By default, a given translated document is produced when at least 80% of its
content is translated. The untranslated text is kept in the original language.
The produced documentation thus mixes languages if the translation is not
complete. You can change the 80% threshold with the I<--keep> option described
below. Note however that discarding translations as soon as they are not 100%
may be discouraging for the translators whose work will almost never be shown to
the users, while showing "translations" that are too incomplete may be troubling
for the end users.

Storing the translated documentation files in the version control system is
probably a bad idea, since they are automatically generated. The precious files
are the PO files, that contain the hard work of your fellow translators. Also,
some people find it easier to interact with the translators through an online
platform such as S<weblate>, but this is naturally fully optional.

=head2 Quick start tutorial

Let's assume you maintain a program named B<foo> which has a man page
F<man/foo.1> written in English (the bridge language in most open-source
projects, but B<po4a> can be used from or to any language). Some times ago,
someone provided a German translation named F<man/foo.de.1> and disappeared.
This is a problem because you just got a bug report saying that your
documentation contains a gravely misleading information that must be fixed in
all languages, but you don't speak German so you can only modify the original,
not the translation. Now, another contributor wants to contribute a translation
to Japanese, a language that you don't master either.

It is time to convert your documentation to B<po4a> to solve your documentation
maintenance nightmares. You want to update the doc when needed, you want to ease
the work of your fellow translators, and you want to ensure that your users
never see any outdated and thus misleading documentation.

The conversion includes two steps: setup the po4a infrastructure, and convert
the previous German translation to salvage the previous work. This latter part
is done using S<po4a-gettextize>, as follows. As detailed in the documentation
of L<po4a-gettextize(1)>, this process rarely fully automatic, but once it's
done, the B<de.po> file containing the German translation can be integrated in
your po4a workflow.

  po4a-gettextize --format man --master foo.1 --localized foo.de.1 --po de.po

Let's now configure po4a. With the appropriate file layout, your configuration
file could be as simple as this:

 [po_directory] man/po4a/

 [type: man] man/foo.1 $lang:man/translated/foo.$lang.1

It specifies that all PO files (containing the work of the translators) are in the
F<man/po4a/> directory, and that you have only one master file, F<man/foo.1>. If
you had several master files, you would have several lines similar to the second
one. Each such line also specify where to write the corresponding translation
files. Here, the German translation of F<man/foo.1> is in
F<man/translated/foo.de.1>.

The last thing we need to complete the configuration of B<po4a> is a POT file
containing the template material that should be used to start a new translation.
Simply create an empty file with the S<.pot> extension in the specified
S<po_directory> (e.g. F<man/po4a/foo.pot>), and B<po4a> will fill it with the
expected content.

Here is a recap of the files in this setup:

  ├── man/
  │   ├── foo.1        <- The original man page, in English
  │   ├── po4a/
  │   │   ├── de.po    <- The German PO translation, from gettextization
  │   │   └── foo.pot  <- The POT template of future translations (empty at first)
  │   └── translated/  <- Directory where the translations will be created
  └── po4a.cfg         <- The configuration file

Once setup, executing B<po4a> will parse your documentation, update the POT
template file, use it to update the PO translation files, and use them to update
the documentation translation files. All in one command:

        po4a --verbose po4a.cfg

This is it. B<po4a> is now fully configured. Once you've fixed your error in
F<man/foo.1>, the offending paragraph in the German translation will be replaced by
the fixed text in English. Mixing languages is not optimal, but it's the only
way to remove errors in translations that you don't even understand, and ensure
that the content presented to the users is never misleading. Updating the German
translation is also much easier in the corresponding PO file, so the language
mix-up may not last long. Finally, when a Japanese translator wants to contribute a
new translation, she should rename the S<foo.pot> into S<ja.po> and complete the
translation. Once you have this file, just drop it in F<man/po4a/po/>. A translated 
page will appear as F<man/translated/foo.ja.1> (provided that enough content is
translated) when you run B<po4a> again.

=head1 OPTIONS

=over 4

=item B<-k>, B<--keep>

Minimal threshold for translation percentage to keep (i.e. write) the
resulting file (default: 80). I.e. by default, files have to be translated
at least at 80% to be written on disk.

=item B<-w>, B<--width>

Column at which we should wrap the resulting file if the format supports it
(default: 76).

=item B<-h>, B<--help>

Show a short help message.

=item B<-M>, B<--master-charset>

Charset of the files containing the documents to translate. Note that all
master documents must use the same charset.

=item B<-L>, B<--localized-charset>

Charset of the files containing the localized documents. Note that all
translated documents will use the same charset.

=item B<-A>, B<--addendum-charset>

Charset of the addenda. Note that all the addenda should be in the same
charset.

=item B<-V>, B<--version>

Display the version of the script and exit.

=item B<-v>, B<--verbose>

Increase the verbosity of the program.

=item B<-q>, B<--quiet>

Decrease the verbosity of the program.

=item B<-d>, B<--debug>

Output some debugging information.

=item B<-o>, B<--option>

Extra option(s) to pass to the format plugin. See the documentation of each
plugin for more information about the valid options and their meanings. For 
example, you could pass '-o tablecells' to the AsciiDoc parser, while the 
text parser would accept '-o tabs=split'.

=item B<-f>, B<--force>

Always generate the POT and PO files, even if B<po4a> considers it is
not necessary.

The default behavior (when B<--force> is not specified) is the following:

=over

If the POT file already exists, it is regenerated if a master document
or the configuration file is more recent (unless B<--no-update> is provided).
The POT file is also written in a temporary document and B<po4a> verifies
that the changes are really needed.

Also, a translation is regenerated only if its master document, the PO file,
one of its addenda or the configuration file is more recent.
To avoid trying to regenerate translations which do not pass the threshold
test (see B<--keep>), a file with the F<.po4a-stamp> extension can be created
(see B<--stamp>).

=back

If a master document includes files, you should use the B<--force> flag
because the modification time of these included files are not taken into
account.

The PO files are always re-generated based on the POT with B<msgmerge -U>.

=item B<--stamp>

Tells B<po4a> to create stamp files when a translation is not generated
because it does not reach the threshold. These stamp files are named
according to the expected translated document, with the F<.po4a-stamp>
extension.

Note: This only activates the creation of the F<.po4a-stamp> files. The stamp
files are always used if they exist, and they are removed with
B<--rm-translations> or when the file is finally translated.

=item B<--no-translations>

Do not generate the translated documents, only update the POT and PO files.

=item B<--no-update>

Do not change the POT and PO files, only the translation may be updated.

=item B<--keep-translations>

Keeps the existing translation files even if the translation doesn't meet the threshold specified by B<--keep>.
This option does not create new translation files with few content,
but it will save existing translations which decay because of changes to the master files.

WARNING: This flag changes the po4a behavior in a rather drastic way:
your translated files will not get updated at all until the
translation improves. Only use this flag if you prefer shipping an
outdated translated documentation rather than only shipping an
accurate untranslated documentation.

=item B<--rm-translations>

Remove the translated files (implies B<--no-translations>).

=item B<--no-backups>

This flag does nothing since 0.41, and may be removed
in later releases.

=item B<--rm-backups>

This flag does nothing since 0.41, and may be removed
in later releases.

=item B<--translate-only> I<translated-file>

Translate only the specified file.  It may be useful to speed up
processing if a configuration file contains a lot of files.  Note that this
option does not update PO and POT files.
This option can be used multiple times.

=item B<--variable> I<var>B<=>I<value>

Define a variable that will be expanded in the B<po4a> configuration file.
Every occurrence of I<$(var)> will be replaced by I<value>.
This option can be used multiple times.

=item B<--srcdir> I<SRCDIR>

Set the base directory for all input documents specified in the B<po4a>
configuration file. 

If both I<destdir> and I<srcdir> are specified, input files are
searched in the following directories, in order: I<destdir>, the current
directory and I<srcdir>. Output files are written to I<destdir> if
specified, or to the current directory.

=item B<--destdir> I<DESTDIR>

Set the base directory for all the output documents specified in the
B<po4a> configuration file (see B<--srcdir> above).

=back

=head2 Options modifying the POT header

=over 4

=item B<--porefs> I<type>

Specify the reference format. Argument I<type> can be one of B<never>
to not produce any reference, B<file> to only specify the file
without the line number, B<counter> to replace line number by an
increasing counter, and B<full> to include complete references (default: full).

=item B<--wrap-po> B<no>|B<newlines>|I<number> (default: 76)

Specify how the po file should be wrapped. This gives the choice between either
files that are nicely wrapped but could lead to git conflicts, or files that are
easier to handle automatically, but harder to read for humans.

Historically, the gettext suite has reformatted the po files at the 77th column
for cosmetics. This option specifies the behavior of po4a. If set to a numerical
value, po4a will wrap the po file after this column and after newlines in the
content. If set to B<newlines>, po4a will only split the msgid and msgstr after
newlines in the content. If set to B<no>, po4a will not wrap the po file at all.
The reference comments are always wrapped by the gettext tools that we use internally.

Note that this option has no impact on how the msgid and msgstr are wrapped, i.e.
on how newlines are added to the content of these strings.

=item B<--master-language>

Language of the source files containing the documents to
translate. Note that all master documents must use the same language.

=item B<--msgid-bugs-address> I<email@address>

Set the report address for msgid bugs. By default, the created POT files
have no Report-Msgid-Bugs-To fields.

=item B<--copyright-holder> I<string>

Set the copyright holder in the POT header. The default value is
"Free Software Foundation, Inc."

=item B<--package-name> I<string>

Set the package name for the POT header. The default is "PACKAGE".

=item B<--package-version> I<string>

Set the package version for the POT header. The default is "VERSION".

=back

=head2 Options to modify the PO files

=over 4

=item B<--msgmerge-opt> I<options>

Extra options for B<msgmerge>(1).

Note: B<$lang> will be extended to the current language.

=item B<--no-previous>

This option removes B<--previous> from the options passed to B<msgmerge>.
This is necessary to support versions of B<gettext> earlier than 0.16.

=item B<--previous>

This option adds B<--previous> to the options passed to B<msgmerge>.
It requires B<gettext> 0.16 or later, and is activated by default.

=back

=head1 CONFIGURATION FILE

po4a expects a configuration file as argument. This file must contain
the following elements:

=over

=item

The path to the PO files and the list of languages existing in the project;

=item

Optionally, some global options and so-called configuration aliases that are
used as templates to configure individual master files;

=item

The list of each master file to translate, along with specific parameters.

=back

All lines contain a command between square braces, followed by its parameters.
Comments begin with the char '#' and run until the end of the line. You can
escape the end of line to spread a command over several lines.

Some full examples are presented on this page, while other examples
can be found in the C<t/cfg> directory of the source distribution.

=head2 Finding the PO and POT files

The simplest solution is to explicitly give the path to POT and PO files, as follows:

 [po4a_paths] man/po/project.pot de:man/po/de.po fr:man/po/fr.po

This specifies the path to the POT file first, and then the paths to the German
and French PO files.

The same information can be written as follows to reduce the risk of copy/paste errors:

 [po4a_langs] fr de
 [po4a_paths] man/po/project.pot $lang:man/po/$lang.po

The C<$lang> component is automatically expanded using the provided languages
list, reducing the risk of copy/paste error when a new language is added.

You can further compact the same information by only providing the
path to the directory containing your translation project, as follows.

 [po_directory] man/po/

The provided directory must contain a set of PO files, each named F<XX.po> with
C<XX> the ISO 639-1 of the language used in this file. The directory must also
contain a single POT file, with the C<.pot> file extension. For the first run,
this file can be empty but it must exist (po4a cannot guess the name to use
before the extension).

Note that you must choose only one between C<po_directory> and
C<po4a_paths>. The first one (C<po_directory>) is more compact,
further reduces the risk of copy/paste error, but forces you to use
the expected project structure and file names. The second one
(C<po4a_paths>), is more explicit, probably more readable, and
advised when you setup your first project with po4a.

=head3 Centralized or split PO files?

By default, po4a produces one single PO file per target language, containing the
whole content of your translation project. As your project grows, the size of
these files may become problematic. When using weblate, it is possible to specify
priorities for each translation segment (i.e., msgid) so that the important ones get
translated first. Still, some translation teams prefer to split the content
in several files.

To have one PO file per master file, you simply have to use the string
C<$master> in the name of your PO files on the C<[po4a_paths]> line, as follows.

 [po4a_paths] doc/$master/$master.pot $lang:doc/$master/$lang.po

With this line, po4a will produce separate POT and PO files for each document to translate.
For example, if you have 3 documents and 5 languages, this will result in 3 POT files and
15 PO files. These files are named as specified on the C<po4a_paths> template, with
C<$master> substituted to the basename of each document to translate. In case of name
conflict, you can specify the POT file to use as follows, with the C<pot=> parameter.

This feature can also be used to group several translated files into the same
POT file. The following example only produces 2 POT files: F<l10n/po/foo.pot>
(containing the material from F<foo/gui.xml>) and F<l10n/po/bar.pot> (containing
the material from both F<bar/gui.xml> and F<bar/cli.xml>).

 [po4a_langs] de fr ja
 [po4a_paths] l10n/po/$master.pot $lang:l10n/po/$master.$lang.po
 [type: xml] foo/gui.xml $lang:foo/gui.$lang.xml pot=foo
 [type: xml] bar/gui.xml $lang:bar/gui.$lang.xml pot=bar
 [type: xml] bar/cli.xml $lang:bar/cli.$lang.xml pot=bar

In split mode, B<po4a> builds a temporary compendium during the PO update, to
share the translations between all the PO files. If two PO files have different
translations for the same string, B<po4a> will mark this string as fuzzy and
will submit both translations in all the PO files containing this string. When
unfuzzied by the translator, the translation is automatically used in every PO
files.

=head2 Specifying the documents to translate

You must also list the documents that should be translated. For each master
file, you must specify the format parser to use, the location of the translated
document to produce, and optionally some configuration. File names should be quoted
or escaped if they contain spaces. Here is an example:

 [type: sgml] "doc/my stuff.sgml"  "fr:doc/fr/mon truc.sgml"  de:doc/de/mein\ kram.sgml
 [type: man] script fr:doc/fr/script.1 de:doc/de/script.1
 [type: docbook] doc/script.xml fr:doc/fr/script.xml \
             de:doc/de/script.xml

But again, these complex lines are difficult to read and modify, e.g. when adding a new
language. It is much simpler to reorganize things using the C<$lang> template as follows:

 [type: sgml]    doc/my_stuff.sgml $lang:doc/$lang/my_stuff.sgml
 [type: man]     script.1          $lang:po/$lang/script.1
 [type: docbook] doc/script.xml    $lang:doc/$lang/script.xml

=head2 Specifying options

There is two types of options: I<po4a options> are default values to the po4a
command line options while I<format options> are used to change the behavior of
the format parsers. As a I<po4a options>, you could for example specify in your
configuration file that the default value of the B<--keep> command line
parameter is 50% instead of 80%. I<Format options> are documented on the
specific page of each parsing module, e.g. L<Locale::Po4a::Xml(3pm)>. You could
for example pass B<nostrip> to the XML parser to not strip the spaces around the
extracted strings.

You can pass these options for a specific master file, or even for a specific
translation of that file, using C<opt:> and C<opt_XX:> for the C<XX> language.
In the following example, the B<nostrip> option is passed to the XML parser (for
all languages), while the threshold will be reduced to 0% for the French
translation (that is thus always kept).

 [type:xml] toto.xml $lang:toto.$lang.xml opt:"-o nostrip" opt_fr:"--keep 0"

In any case, these configuration chunks must be located at the end of the line.
The declaration of files must come first, then the addendum if any (see below),
and then only the options. The grouping of configuration chunks is not very
important, since elements are internally concatenated as strings. The following
examples are all equivalent:

  [type:xml] toto.xml $lang:toto.$lang.xml opt:"--keep 20" opt:"-o nostrip" opt_fr:"--keep 0"
  [type:xml] toto.xml $lang:toto.$lang.xml opt:"--keep 20 -o nostrip" opt_fr:"--keep 0"
  [type:xml] toto.xml $lang:toto.$lang.xml opt:--keep opt:20 opt:-o opt:nostrip opt_fr:--keep opt_fr:0

Note that language specific options are not used when building the POT file. It
is for example impossible to pass B<nostrip> to the parser only when building
the French translation, because the same POT file is used to update every
languages. So the only options that can be language-specific are the ones that
are used when producing the translation, as the C<--keep> option.

=head3 Configuration aliases

To pass the same options to several files, the best is to define a type alias as
follows. In the next example, C<--keep 0> is passed to every Italian translation
using this C<test> type, that is an extension of the C<man> type.

  [po4a_alias:test] man opt_it:"--keep 0"
  [type: test] man/page.1 $lang:man/$lang/page.1

You can also extend an existing type reusing the same name for the alias as
follows. This is not interpreted as as an erroneous recursive definition.

  [po4a_alias:man] man opt_it:"--keep 0"
  [type: man] man/page.1 $lang:man/$lang/page.1

=head3 Global default options

You can also use C<[options]> lines to define options that must be used for all
files, regardless of their type.

  [options] --keep 20 --option nostrip

As with the command line options, you can abbreviate the parameters passed in
the configuration file:

  [options] -k 20 -o nostrip

=head3 Option priorities

The options of every sources are concatenated, ensuring that the default values
can easily be overridden by more specific options. The order is as follows:

=over

=item

C<[options]> lines provide default values that can be overridden by any other source.

=item

Type aliases are then used. Language specific settings override the ones
applicable to all languages.

=item

Settings that are specific to a given master file override both the default ones
and the ones coming from the type alias. In this case also, language specific
settings override the global ones.

=item

Finally, parameters provided on the B<po4a> command line override any settings
from the configuration file.

=back

=head3 Example

Here is an example showing how to quote the spaces and quotes:

 [po_directory] man/po/
 
 [options] --master-charset UTF-8
 
 [po4a_alias:man] man opt:"-o \"mdoc=NAME,SEE ALSO\""
 [type:man] t-05-config/test02_man.1 $lang:tmp/test02_man.$lang.1 \
            opt:"-k 75" opt_it:"-L UTF-8" opt_fr:--verbose

=head2 Addendum: Adding extra content in the translation

If you want to add an extra section to the translation, for example to give
credit to the translator, then you need to define an addendum to the line
defining your master file. Please refer to the page L<po4a(7)> for more details
on the syntax of addendum files.

 [type: pod] script fr:doc/fr/script.1 \
             add_fr:doc/l10n/script.fr.add

You can also use language templates as follow:

 [type: pod] script $lang:doc/$lang/script.1 \
             add_$lang:doc/l10n/script.$lang.add

If an addendum fails to apply, the translation is discarded.

=head3 Modifiers for the addendum declaration

Addendum modifiers can simplify the configuration file in the case where not all
languages provide an addendum, or when the list of addenda changes from one
language to the other. The modifier is a single char located before the file name.

=over 2

=item B<?>

Include I<addendum_path> if this file does exist, otherwise do nothing.

=item B<@>

I<addendum_path> is not a regular addendum but a file containing a list of
addenda, one by line.  Each addendum may be preceded by modifiers.

=item B<!>

I<addendum_path> is discarded, it is not loaded and will not be loaded by
any further addendum specification.

=back

The following includes an addendum in any language, but if only it exists. No
error is reported if the addendum does not exist.

 [type: pod] script $lang:doc/$lang/script.1  add_$lang:?doc/l10n/script.$lang.add

The following includes a list of addendum for every language:

 [type: pod] script $lang:doc/$lang/script.1  add_$lang:@doc/l10n/script.$lang.add

=head2 Filtering the translated strings

Sometimes, you want to hide some strings from the translation process. To
that extend, you can give a C<pot_in> parameter to your master file to specify
the name of the file to use instead of the real master when building the POT
file. Here is an example:

  [type:docbook] book.xml          \
          pot_in:book-filtered.xml \
          $lang:book.$lang.xml

With this setting, the strings to translate will be extracted from the
F<book-filtered.xml> (that must be produced before calling B<po4a>) while the
translated files will be built from F<book.xml>. As a result, any string that is
part of F<book.xml> but not in F<book-filtered.xml> will not be included in the
PO files, preventing the translators from providing a translation for them. So
these strings will be left unmodified when producing the translated documents.
This naturally decreases the level of translation, so you may need the C<--keep>
option to ensure that the document is produced anyway.

=head1 SEE ALSO

L<po4a-gettextize(1)>,
L<po4a(7)>.

=head1 AUTHORS

 Denis Barbier <barbier@linuxfr.org>
 Nicolas François <nicolas.francois@centraliens.net>
 Martin Quinson (mquinson#debian.org)

=head1 COPYRIGHT AND LICENSE

Copyright 2002-2023 by SPI, inc.

This program is free software; you may redistribute it and/or modify it
under the terms of GPL v2.0 or later (see the COPYING file).

=cut

use 5.16.0;
use strict;
use warnings;

use Getopt::Long qw(GetOptions);

use Locale::Po4a::Chooser;
use Locale::Po4a::TransTractor;
use Locale::Po4a::Common;
use Locale::Po4a::Po qw(move_po_if_needed);

use Pod::Usage qw(pod2usage);

use File::Temp;
use File::Basename;
use File::Copy;
use File::Spec;
use Fcntl;    # sysopen flags
use Cwd;      # cwd
use Config;

Locale::Po4a::Common::textdomain('po4a');

sub show_version {
    Locale::Po4a::Common::show_version("po4a");
    exit 0;
}

# keep the command line arguments
my @ORIGINAL_ARGV = @ARGV;

# Use /NUL instead of /dev/null on Windows
my $devnull = ( $^O =~ /Win/ ) ? '/NUL' : '/dev/null';

# Parse the options provided on the command line, or in argument
sub get_options {
    if ( defined $_[0] ) {
        @ARGV = @_;

        #        map {print "o: $_\n"} @ARGV;
    } else {
        @ARGV = @ORIGINAL_ARGV;
    }

    # temporary array for GetOptions
    my @verbose   = ();
    my @options   = ();
    my @variables = ();
    my $previous;
    my $noprevious;
    my $msgmerge_opt = '';

    my %opts = (
        "help"               => 0,
        "type"               => "",
        "debug"              => 0,
        "verbose"            => 0,
        "quiet"              => 0,
        "force"              => 0,
        "no-translations"    => 0,
        "no-update"          => 0,
        "keep-translations"  => 0,
        "rm-translations"    => 0,
        "no-backups"         => 0,
        "rm-backups"         => 0,
        "threshold"          => 80,
        "width"              => 76,
        "mastlang"           => "",
        "mastchar"           => "",
        "locchar"            => "",
        "addchar"            => "",
        "options"            => { "verbose" => 0, "debug" => 0 },
        "variables"          => {},
        "partial"            => [],
        "porefs"             => "full",
        "wrap-po"            => undef,
        "copyright-holder"   => undef,
        "msgid-bugs-address" => undef,
        "package-name"       => undef,
        "package-version"    => undef,
        "srcdir"             => undef,
        "destdir"            => undef,
        "calldir"            => cwd()
    );
    Getopt::Long::config( 'bundling', 'no_getopt_compat', 'no_auto_abbrev' );
    GetOptions(
        'help|h' => \$opts{"help"},

        'master-language=s'     => \$opts{"mastlang"},
        'master-charset|M=s'    => \$opts{"mastchar"},
        'localized-charset|L=s' => \$opts{"locchar"},
        'addendum-charset|A=s'  => \$opts{"addchar"},

        'verbose|v'            => \@verbose,
        'debug|d'              => \$opts{"debug"},
        'force|f'              => \$opts{"force"},
        'stamp'                => \$opts{"stamp"},
        'quiet|q'              => \$opts{"quiet"},
        'keep|k=s'             => \$opts{"threshold"},
        'width|w=s'            => \$opts{"width"},
        'no-translations'      => \$opts{"no-translations"},
        'no-update'            => \$opts{"no-update"},
        'keep-translations'    => \$opts{"keep-translations"},
        'rm-translations'      => \$opts{"rm-translations"},
        'translate-only=s'     => \@{ $opts{"partial"} },
        'no-backups'           => \$opts{"no-backups"},
        'rm-backups'           => \$opts{"rm-backups"},
        'version|V'            => \&show_version,
        'option|o=s'           => \@options,
        'variable=s'           => \@variables,
        'porefs=s'             => \$opts{"porefs"},
        'wrap-po=s'            => \$opts{"wrap-po"},
        'copyright-holder=s'   => \$opts{"copyright-holder"},
        'msgid-bugs-address=s' => \$opts{"msgid-bugs-address"},
        'package-name=s'       => \$opts{"package-name"},
        'package-version=s'    => \$opts{"package-version"},
        'no-previous'          => \$noprevious,
        'previous'             => \$previous,
        'msgmerge-opt=s'       => \$msgmerge_opt,
        'srcdir=s'             => \$opts{"srcdir"},
        'destdir=s'            => \$opts{"destdir"}
    ) or pod2usage();

    $opts{"verbose"} = scalar @verbose;
    $opts{"verbose"} = 0 if $opts{"quiet"};
    $opts{"verbose"} ||= 1 if $opts{"debug"};
    $opts{"msgmerge-opt"} = '';
    $opts{"msgmerge-opt"} .= "--previous " unless $noprevious;
    $opts{"msgmerge-opt"} .= "--add-location=file " if ( $opts{'porefs'} =~ m/^file/ );
    $opts{"msgmerge-opt"} .= $msgmerge_opt;

    # options to transmit to the modules
    $opts{"options"} = {
        "verbose" => $opts{"verbose"},
        "debug"   => $opts{"debug"},
        "wrapcol" => $opts{"width"}
    };
    foreach (@options) {
        if (m/^([^=]*)=(.*)$/) {
            $opts{"options"}{$1} = "$2";
        } else {
            $opts{"options"}{$_} = 1;
        }
    }

    foreach (@variables) {
        if (m/^([^=]*)=(.*)$/) {
            $opts{"variables"}{$1} = "$2";
        }
    }

    # The rm- options imply the no-
    $opts{"no-translations"} = 1 if $opts{"rm-translations"};

    if ( defined $opts{"srcdir"} and not -d $opts{"srcdir"} ) {
        die wrap_msg( gettext("Option %s invalid. Directory %s does not exist (current directory: %s)."),
            "srcdir", $opts{"srcdir"}, cwd() );
    }
    if ( defined $opts{"destdir"} and not -d $opts{"destdir"} ) {
        die wrap_msg( gettext("Option %s invalid. Directory %s does not exist (current directory: %s)."),
            "destdir", $opts{"destdir"}, cwd() );
    }

    #    print STDERR "msgmerge: ".$opts{"msgmerge-opt"}."\n";
    return %opts;
}

# Split a line into an array of lines. This should correctly deal with quotes and escaped spaces.
#   - Simple and double quotes can be mixed: {'"tutu truc" blah' "'blah tutu' truc"} => {"tutu truc" blah}{'blah tutu' truc}
#   - Quotes can be in the middle of words: {h ij'k l'  m} => {h}{ijk l}{m}
#   - Spaces can be escaped everywhere: {\ a b\ c d\   e} => { a}{b c}{d }{e}
#   - Quotes and space escaping can be mixed: {'a b'c\ d} => {a bc d}
#
# Submitted as https://stackoverflow.com/questions/29041011/perl-split-string-that-has-double-quotes-and-space/76404839#76404839
sub split_line {
    my ($string) = @_;

    my @result      = ();
    my @quote       = ();    # stack of quotes we've seen so far, to handle nested single and double quotes
    my $accumulated = "";    # the token we are building
    my $seen_quotes = 0
      ; # Whether we've seen at least a quote or whether we should pass a chunk of only spaces (to pass separating spaces)

    # eat the string piece by piece, separating a quote, some spaces, or a chunk w/o quotes and spaces.
    while ( $string =~ s/('|"|\s+|[^"'\s]+)(.*)$/$2/ ) {
        my $token = $1;

        #        print "Seen $token (last quote: ".(scalar @quote>0?$quote[-1]:"none")."; accumulated:$accumulated) -- ";
        if ( $token eq "'" || $token eq '"' ) {
            $seen_quotes = 1;
            if ( scalar @quote > 0 ) {
                if ( $quote[-1] eq $token ) {
                    pop @quote;
                } else {
                    $accumulated .= $token;
                }
            } else {
                push @quote, $token;
            }
        } elsif ( $token =~ /\\$/ ) {
            if ( defined( my $next_char = substr( $string, 0, 1 ) ) ) {

                #                print " steal '$next_char' ";
                $token =~ s/.$/$next_char/;
                $string = substr( $string, 1 );
            } else {
                die "Error: backslash escape at end of string\n";
            }
            $accumulated .= $token;
        } elsif ( $token =~ /^\s*$/ ) {
            if ( scalar @quote > 0 ) {
                $accumulated .= $token;
            }
        } else {
            $accumulated .= $token;
        }

        #        print " (last quote: ".(scalar @quote>0?$quote[-1]:"none")."; accumulated:$accumulated)";

        # Push the string if we're out of quotes, and if the coming chunk is a separating space
        if ( scalar @quote == 0 && ( $string eq '' || substr( $string, 0, 1 ) eq ' ' ) ) {

            #            print " Push\n";
            # Pass the separating quotes, that are space only and never contained any quote
            if ( $accumulated ne '' && $seen_quotes || $accumulated =~ /\S/ ) {
                push @result, $accumulated;
            }
            $accumulated = '';
            $seen_quotes = 0;

            #        } else {
            #            print " Pass\n";
        }
    }

    if ( scalar @quote > 0 ) {
        die "Error: unclosed quote at end of string\n";
    }

    return @result;
}

# Test the split_line function
#my $testline = "a  'b c   c d  ' 'titi' \"toto\" ' ' \" \" '\"' \"'\" '\"tutu truc\" blah' \"'blah tutu' truc\" e\\ f  g\\   h ij'k l'  m\"n \" 'l ' "
#                ."\\\" \\' '\\'' \"'\\''\" BHA\\RO \\ n \"o p\"q\\ r";
#print "$testline\n";
#print join "#", split_line($testline);
#print "#\n";
#__END__

# Parse a config line and extract the parameters that correspond to options.
# These options are appended to the options provided in argument (as a
# reference to an hash). The options are sorted by category in this hash.
# The categories are: global and the various languages.
sub parse_config_options {
    my $ref     = shift;    # a line reference for the die messages
    my $line    = shift;    # the line to parse
    my $options = shift;    # reference to an hash of options

    if ( $line =~ m/^(opt(?:_(.+?))?:)?(.*)/ ) {
        my $prefixed = $1;
        my $lang     = $2 // 'global';
        my $opt      = $3;

        if ( !$prefixed and $opt =~ m/ / ) {
            if ( $opt =~ m/'/ ) {
                $opt = qq{"$opt"};
            } else {
                $opt = qq{'$opt'};
            }
        }

        if ( !defined $options->{$lang} ) {
            $options->{$lang} = $opt;
        } else {
            $options->{$lang} .= " $opt";
        }
        return "";
    } else {
        return $line;
    }
}

my %po4a_opts = get_options(@ARGV);

sub find_input_file {
    my $filename = $_[0];
    my $debug    = $_[1];
    print STDERR "Search for input file $filename\n" if $debug;
    return $filename                                 if ( File::Spec->file_name_is_absolute($filename) );
    foreach ( ( $po4a_opts{'destdir'}, $po4a_opts{'calldir'}, $po4a_opts{'srcdir'} ) ) {
        next unless defined $_;
        my $p = File::Spec->catfile( $_, $filename );
        print STDERR "Test $p: " . ( -e $p ? "OK" : "nope" ) . "\n"
          if defined $debug;
        return $p if -e $p;
    }
    return $filename;
}

sub find_output_file {
    my $filename = $_[0];
    return $filename if ( File::Spec->file_name_is_absolute($filename) );
    foreach ( ( $po4a_opts{'destdir'}, $po4a_opts{'calldir'} ) ) {
        next unless defined $_;
        return File::Spec->catfile( $_, $filename ) if -d $_ and -w $_;
    }
    return $filename;
}

# Argument check
$po4a_opts{"help"} && pod2usage( -verbose => 1, -exitval => 0 );

sub run_cmd {
    my $cmd = shift;
    print $cmd. "\n" if $po4a_opts{"debug"};

    my $out = qx/$cmd 2>&1/;
    unless ( $? == 0 ) {
        my $err = "";
        if ( $? == -1 ) {
            $err = sprintf( gettext("failed to execute '%s': %s."), $cmd, $! );
        } elsif ( $? & 127 ) {
            if ( $? & 128 ) {
                $err = sprintf( gettext("'%s' died with signal %d, with coredump."), $cmd, $? & 127 );
            } else {
                $err = sprintf( gettext("'%s' died with signal %d, without coredump."), $cmd, $? & 127 );
            }
        } else {
            $err = sprintf( gettext("'%s' exited with value %d."), $cmd, $? >> 8 );
        }

        die wrap_msg( gettext("Error: %s"), $err );
    }
    return $out;
}

my $config_file = shift(@ARGV) || pod2usage();

# Check file existence
-e $config_file || die wrap_msg( gettext("File %s does not exist."), $config_file );

# Parse the config file
my (@langs);
my (%aliases);                # module aliases ([po4a_alias:...]
my $pot_filename     = "";    # In split mode, this is the temp file name for the big POT
my $pot_filename_cfg = "";    # This is never modified, useful in split mode to find the cfg content
my (%po_filename);            # po_files: '$lang'=>'$path'
my (%add_filename);           # Global addendum_files: '$lang'=>array of '$path'

# The document hash table contains all the information about the files to translate.
# Each key of the hash map a master file, ie a file to translate. For each such master file, we have a sub-hashmap containing:
#  'format' => '$format': string designating the po4a module to use
#  '$lang'  => '$path': one string entry per language, the path of the translated file
# Optional parts:
#  'add_$lang'=>('$path','$path'): one vector per language, the list of all addendum for that language
#  'pot_in' => '$master': The master file to use in input when building the POT (allows to filter out some things)
#  'pot'    => the filename to use when building the POT file in split mode (when not provided, basename of master is used)
my (%document);

my $doc_count = 0;
my %partial   = ( 'master' => {}, 'files' => {}, 'lang' => {} );
open CONFIG, "<", "$config_file" or die wrap_msg( gettext("Cannot open %s: %s"), $config_file, $! );
my ( $line, $nb ) = ( "", 0 );

while (<CONFIG>) {
    $nb++;
    s/#.*//;
    s/\R//g;    # Deal with CRLF
    $line .= $_;
    $line =~ s/\t/ /g;
    $line =~ s/ +/ /g;
    while ( $line =~ m/\$\((\w+)\)/ ) {
        if ( defined $po4a_opts{"variables"}{$1} ) {
            $line =~ s/\$\((\Q$1\E)\)/$po4a_opts{"variables"}{$1}/g;
        } else {
            die wrap_ref_mod( "$config_file:$nb", "", gettext("Unknown variable: %s"), $1 );
        }
    }
    $line =~ s/^ //;
    $line =~ s/ $//;
    chomp($line);
    next if ( $line =~ s/\\$// );
    next unless ( $line =~ /\S/ );

    die wrap_ref_mod( "$config_file:$nb", "", gettext("Syntax error: %s"), $line )
      unless ( $line =~ s/^\[([^\]]*)\] *// );
    my $cmd = $1;
    my (@split_args) = split_line($line);

    if (@langs) {

        # Expand the $lang templates
        my @args2 = ();
        foreach my $arg (@split_args) {
            if ( $arg =~ /\$lang\b/ ) {

                # Expand for all the langs
                foreach my $lang (@langs) {
                    my ($arg2) = $arg;
                    $arg2 =~ s/\$lang\b/$lang/g;
                    push @args2, $arg2;
                }
            } else {

                # Leave the argument as is
                push @args2, $arg;
            }
        }
        @split_args = @args2;
    }
    my $main   = $split_args[0] // "";
    my $pot_in = $main;                  # Default for POT file input is $main

    print "cmd=[$cmd]; $main / " . join( ' ', @split_args ) . "\n" if $po4a_opts{"debug"};

    if ( $cmd eq "po4a_paths" ) {
        shift @split_args;
        die wrap_ref_mod( "$config_file:$nb", "", gettext("'%s' redeclared"), "po4a_path" )
          if ( length $pot_filename );
        $pot_filename     = $main;
        $pot_filename_cfg = $main;
        warn wrap_ref_mod(
            "$config_file:$nb",
            "",
            gettext(
                "Your POT file '%s' file name does not end with the 'pot' extension. This is unusual and will trouble the translators."
            ),
            $pot_filename
        ) unless $pot_filename =~ m/\.pot$/;
        warn wrap_ref_mod(
            "$config_file:$nb",
            "",
            gettext("No PO file provided as subsequent parameters to '%s'. Your documentation will not be translated."),
            "po4a_path"
        ) unless scalar @split_args > 0;

        foreach my $arg (@split_args) {
            next unless $arg =~ /\S/;

            die wrap_ref_mod( "$config_file:$nb", "", gettext("Unparsable argument '%s'."), $arg )
              unless ( $arg =~ /^([^:]*):(.*)/ );
            my ( $component, $path ) = ( $1, $2 );
            if ( $component =~ m/^add_(.*)$/ ) {    # This is an addendum declaration
                my $lang    = $1;
                my @addlist = ($path);
                while ( scalar @addlist ) {
                    my $elem = shift @addlist;
                    my $modifiers;
                    $elem =~ s/^([@!?]+)// and $modifiers = $1;
                    if ( defined $modifiers ) {
                        next if ( $modifiers =~ m/!/ );
                        next if ( $modifiers =~ m/\?/ and not -e find_input_file($elem) );
                        if ( $modifiers =~ m/@/ ) {
                            open LIST, "<", find_input_file($elem)
                              or die wrap_msg( gettext("Cannot open %s: %s"), $elem, $! );
                            while (<LIST>) {
                                chomp;
                                s/\s+$//;
                                next if length($_) == 0 or $_ =~ m/^\s*#/;
                                while (m/\$\((\w+)\)/) {
                                    if ( defined $po4a_opts{"variables"}{$1} ) {
                                        s/\$\((\Q$1\E)\)/$po4a_opts{"variables"}{$1}/g;
                                    } else {
                                        die wrap_ref_mod( "$config_file:$nb", "", gettext("Unknown variable: %s"), $1 );
                                    }
                                }
                                push @addlist, $_;
                            }
                            close LIST;
                            next;
                        }
                    }
                    push @{ $add_filename{$lang} }, $elem;
                }

            } else {    # Not an addendum. It must be a po filename
                $po_filename{$1} = $path;
            }
        }

    } elsif ( $cmd eq "po4a_langs" ) {
        die wrap_ref_mod( "$config_file:$nb", "", gettext("'%s' redeclared"), "po4a_langs" )
          if (@langs);
        @langs = map { $_ } @split_args;    # deep copy

    } elsif ( $cmd eq "po_directory" ) {
        die wrap_ref_mod( "$config_file:$nb", "", gettext("The list of languages cannot be set twice.") )
          if scalar @langs;
        die wrap_ref_mod( "$config_file:$nb", "", gettext("The POT file cannot be set twice.") )
          if length $pot_filename;

        shift @split_args;

        # Search for the candidate PO and POT files
        # The difficulty is that the directory itself must be searched in destdir, calldir, srcdir and then the PO files must be searched in there.
        # This mechanism is easy to get wrong (see eg #960892), so let's get cautionous and verbose on error.
        # Step 1. collect the files in every candidate po_directory we find.
        my $po_directory = $main;
        my @po_files;
        my ( $destdir, $srcdir, $calldir ) = ( $po4a_opts{'destdir'}, $po4a_opts{'srcdir'}, $po4a_opts{'calldir'} );
        print STDERR "Searching for po_directory '$po_directory'."
          if $po4a_opts{"debug"};
        if ( File::Spec->file_name_is_absolute($po_directory) ) {
            print STDERR wrap_msg( gettext("Using '%s' as a %s."), $po_directory, "po_directory" )
              if $po4a_opts{"verbose"};
            die wrap_ref_mod( "$config_file:$nb", "", gettext("'%s' is not a directory (current dir: %s)"),
                $po_directory, cwd() )
              unless ( -d $po_directory );

            opendir PO_DIR, find_input_file($po_directory)
              or die wrap_ref_mod( "$config_file:$nb", "", gettext("Cannot list the '%s' directory"), $po_directory );
            map { push @po_files, $_ if -e $_; } ( sort readdir PO_DIR );
        } else {
            my $found = 0;
            foreach my $basedir ( $destdir, $calldir // ".", $srcdir ) {
                next unless defined $basedir;
                my $candidate = "$basedir/$po_directory";
                if ( -e $candidate && -d $candidate ) {
                    opendir PO_DIR,
                      $candidate
                      or die wrap_ref_mod( "$config_file:$nb", "", gettext("Cannot list directory '%s' in '%s' "),
                        $po_directory, $basedir );
                    map { push @po_files, "$po_directory/$_" if -e "$candidate/$_"; } ( sort readdir PO_DIR );

                    # Only display the message the first time we find the right directory
                    print STDERR wrap_msg( gettext("Using '%s' as a %s."), $candidate, "po_directory" )
                      if $po4a_opts{"verbose"} && $found == 0 && scalar @po_files;
                    $found = 1;
                }
            }
            unless ($found) {
                die wrap_ref_mod( "$config_file:$nb", "", gettext("Directory '%s' not found in '%s'."),
                    $po_directory, $calldir )
                  if ( not defined $destdir && not defined $srcdir );
                die wrap_ref_mod( "$config_file:$nb", "", gettext("Directory '%s' not found in '%s' nor in '%s'."),
                    $po_directory, $destdir, $calldir )
                  if ( defined $destdir && not defined $srcdir );
                die wrap_ref_mod( "$config_file:$nb", "", gettext("Directory '%s' not found in '%s' nor in '%s'."),
                    $po_directory, $calldir, $srcdir )
                  if ( not defined $destdir && defined $srcdir );
                die wrap_ref_mod( "$config_file:$nb", "",
                    gettext("Directory '%s' not found in '%s' nor in '%s' nor in '%s'."),
                    $po_directory, $destdir, $calldir, $srcdir );
            }
        }

        # Step 2: separate the PO and POT files we found in all traversed directories
        my %seen = ();    # the elements of po_files that we saw already (we need to ignore the dupplicates)
        foreach my $f (@po_files) {
            next if $seen{$f}++;    # we saw it already
            if ( $f =~ m|^(.*?)/([^/]*)\.po$| ) {
                push @langs, $2;
                die wrap_ref_mod( "$config_file:$nb", gettext("Language '%s' found twice in '%s' and '%s'."),
                    $po_filename{$2}, $f )
                  if defined $po_filename{$2} && $po_filename{$2} ne $f;
                $po_filename{$2} = $f;
                print wrap_msg( gettext("Found language '%s' in the provided po_directory: %s"), $2, $f )
                  if $po4a_opts{"verbose"};
            }
            if ( $f =~ m/\.pot$/ ) {
                die wrap_msg( gettext("%s:%d: too many POT files: '%s' and '%s'."), $config_file, $nb, $pot_filename,
                    $f )
                  if ( length $pot_filename );
                $pot_filename = $f;
                print wrap_msg( gettext("Found POT file '%s' in the provided po_directory."), $pot_filename )
                  if $po4a_opts{"verbose"};
            }
        }
        undef %seen;

        # Step 3: if no PO file found, make a sensible error message.
        # We could factorize the error message, but it would break gettext()
        if ( not @langs ) {
            if ( not defined $destdir && not defined $srcdir ) {
                warn wrap_ref_mod( "$config_file:$nb", "", gettext("No PO files found in '%s'/'%s'."), $calldir,
                    $main );
            } elsif ( defined $destdir && not defined $srcdir ) {
                warn wrap_ref_mod( "$config_file:$nb", "", gettext("No PO files found in '%s'/'%s' nor in '%s'/'%s'."),
                    $destdir, $main, $calldir, $main );
            } elsif ( not defined $destdir && defined $srcdir ) {
                warn wrap_ref_mod( "$config_file:$nb", "", gettext("No PO files found in '%s'/'%s' nor in '%s'/'%s'."),
                    $calldir, $main, $srcdir, $main );
            } else {
                warn wrap_ref_mod( "$config_file:$nb", "",
                    gettext("No PO files found in '%s'/'%s' nor in '%s'/'%s' nor in '%s'/'%s'."),
                    $destdir, $main, $calldir, $main, $srcdir, $main );
            }
            warn wrap_ref_mod( "$config_file:$nb", "",
                gettext("Your project will not be translated to any language.") );
        }

        # Step 4: if no POT file found, make a sensible error message
        if ( length $pot_filename == 0 ) {
            if ( not defined $destdir && not defined $srcdir ) {
                warn wrap_ref_mod( "$config_file:$nb", "", gettext("No POT files found in '%s'/'%s'."), $calldir,
                    $main );
            } elsif ( defined $destdir && not defined $srcdir ) {
                warn wrap_ref_mod( "$config_file:$nb", "", gettext("No POT files found in '%s'/'%s' nor in '%s'/'%s'."),
                    $destdir, $main, $calldir, $main );
            } elsif ( not defined $destdir && defined $srcdir ) {
                warn wrap_ref_mod( "$config_file:$nb", "", gettext("No POT files found in '%s'/'%s' nor in '%s'/'%s'."),
                    $calldir, $main, $srcdir, $main );
            } else {
                warn wrap_ref_mod( "$config_file:$nb", "",
                    gettext("No POT files found in '%s'/'%s' nor in '%s'/'%s' nor in '%s'/'%s'."),
                    $destdir, $main, $calldir, $main, $srcdir, $main );
            }
            warn wrap_ref_mod( "$config_file:$nb", "",
                gettext("Please create an empty file with the 'pot' extension (e.g. 'myproject.pot').") );
        }

    } elsif ( $cmd =~ m/type: *(.*)/ ) {

        shift @split_args;
        if ( defined $document{$main}{'format'} ) {
            warn wrap_ref_mod(
                "$config_file:$nb",
                "",
                gettext(
                    "The master file '%s' was specified earlier in the configuration file. This may cause problems with options."
                ),
                $main
            ) unless ( $po4a_opts{"quiet"} );
        } elsif ( not -e find_input_file($main) ) {
            die wrap_ref_mod( "$config_file:$nb", "", gettext("The master file '%s' does not exist."),
                find_input_file($main) );
        }
        if ( scalar @{ $po4a_opts{"partial"} } ) {
            foreach my $file ( @{ $po4a_opts{"partial"} } ) {
                foreach my $arg (@split_args) {
                    if ( $arg =~ m/(\S+):\Q$file\E\b/ ) {
                        $partial{'lang'}{$1}      = 1;
                        $partial{'master'}{$main} = 1;
                        $partial{'files'}{$file}  = 1;
                    }
                }
            }
        }
        $document{$main}{'format'} = $1;
        $document{$main}{'pot_in'} = $pot_in;
        $document{$main}{'pos'}    = $doc_count;
        $doc_count++;

        # Options
        my %options;

        # 1. Use the global options ([opt] ...)
        #        %options = %{ $document{''}{'options'} }
        #          if defined $document{''}{'options'};

        # 2. Use the alias options if any
        if ( defined $aliases{$1} ) {
            $document{$main}{'format'} = $aliases{$1}{"module"};
            if ( defined $aliases{$1}{"options"} ) {

                # print STDERR "Override the global options with the alias ones\n";
                %options = %{ $aliases{$1}{"options"} };    # XXX not a merge, but overwrite
            }
        }

        # 3. If this file was already specified, reuse the previous
        #    options (no merge)
        %options = %{ $document{$main}{'options'} }
          if defined $document{$main}{'options'};

        # 4. Save the name of the master doc, to handle "$master" substitution in strings

        @split_args = map {
            if (s/pot=(\S+)//) {
                $document{$main}{'pot'} = $1;
                "";    # consume that argument
            } elsif (/master:file=(\S+)/) {
                $document{$main}{'pot'} = $1;
                warn(
                    wrap_ref_mod(
                        "$config_file:$nb",
                        "",
                        gettext("Option 'master:file=' is deprecated. Please use 'pot=' instead for the same effect.")
                    )
                );
                "";
            } else {
                $_;    # Leave the other arguments unchanged
            }
        } @split_args;

        # 5. Merge the document specific options
        @split_args = grep /\S/, map {
            if (/^opt(_.+)?:(.*)/) {
                parse_config_options( "$config_file:$nb", $_, \%options );
            } else {
                $_;
            }
        } @split_args;

        %{ $document{$main}{'options'} } = %options;

        my %discarded = ();
        while (@split_args) {
            my $arg = shift(@split_args);
            die wrap_ref_mod( "$config_file:$nb", "", gettext("Unparsable argument '%s' (%s)."), $arg, $line )
              unless ( $arg =~ /^([^:]*):(.*)/ );
            my ( $lang, $trans ) = ( $1, $2 );
            die wrap_ref_mod( "$config_file:$nb", "", gettext("The translated and master file are the same.") )
              if ( $main eq $trans );

            if ( $lang =~ /^add_/ ) {
                my $modifiers;
                $trans =~ s/^([@!?]+)// and $modifiers = $1;
                next if defined( $discarded{$trans} );
                if ( defined $modifiers ) {
                    if ( $modifiers =~ m/!/ ) {
                        $discarded{$trans} = 1;
                        next;
                    }
                    next if ( $modifiers =~ m/\?/ and not -e find_input_file($trans) );
                    if ( $modifiers =~ m/@/ ) {
                        open LIST, "<", find_input_file($trans)
                          or die wrap_msg( gettext("Cannot open %s: %s"), $trans, $! );
                        my @new_list = ();
                        while (<LIST>) {
                            chomp;
                            s/\s+$//;
                            next if length($_) == 0 or $_ =~ m/^\s*#/;
                            while (m/\$\((\w+)\)/) {
                                if ( defined $po4a_opts{"variables"}{$1} ) {
                                    s/\$\((\Q$1\E)\)/$po4a_opts{"variables"}{$1}/g;
                                } else {
                                    die wrap_ref_mod( "$config_file:$nb", "", gettext("Unknown variable: %s"), $1 );
                                }
                            }
                            push( @new_list, "$lang:$_" );
                        }
                        close LIST;
                        unshift( @split_args, @new_list );
                    } else {
                        push @{ $document{$main}{$lang} }, $trans;
                    }
                } else {
                    push @{ $document{$main}{$lang} }, $trans;
                }
            } elsif ( $lang eq "pot_in" ) {
                $document{$main}{$lang} = $trans;    # override POT file input
                print " .......... divert pot_in=$trans\n" if $po4a_opts{"debug"};
            } else {
                die wrap_ref_mod( "$config_file:$nb", "", gettext("Translation of %s in %s redefined"), $main, $lang )
                  if ( defined $document{$main}{$lang} );
                $document{$main}{$lang} = $trans;
            }
        }
    } elsif ( $cmd =~ m/po4a_alias: *(.*)/ ) {
        my $name = $1;
        shift @split_args;

        if ( defined $aliases{$name} ) {    # If that alias was previously defined
            die "Alias $name redefined from " . $aliases{$name}{"module"} . " to $main\n"
              if ( $aliases{$name}{"module"} ne $main );

            # Concat the new content to the existing alias after a sanity check
            @split_args =
              map { parse_config_options( "$config_file:$nb", $_, \%{ $aliases{$name}{"options"} } ); } @split_args;

        } else {                            # Otherwise, define a new option set accordingly
            my %alias = ();
            $alias{"module"} = $main;

            my %options;
            @split_args = map { parse_config_options( "$config_file:$nb", $_, \%options ); } @split_args;
            %{ $alias{"options"} } = %options;
            %{ $aliases{$name} } = %alias;
        }
    } elsif ( $cmd eq "options" ) {
        my %options;
        %options = %{ $document{''}{"options"} } if ( defined $document{''}{"options"} );  # Add to the existing options

        map { parse_config_options( "$config_file:$nb", $_, \%options ); } @split_args;
        %{ $document{''}{"options"} } = %options;
    } else {
        die wrap_ref_mod( "$config_file:$nb", "", gettext("Unparsable command '%s'."), $cmd );
    }

    $line = "";
}
close CONFIG;    # don't care about error here
die wrap_msg( gettext("'po4a_paths' is not defined in the configuration file. Where are the POT and PO files?") )
  unless ( length $pot_filename );

# Parse a vector of parameters, and split them on spaces.
# If passed ('aa', 'bb', 'cc dd'), it splits the last parameter and returns ('aa', 'bb', 'cc', 'dd'),
sub split_opts {
    my $options      = shift;
    my $options_orig = $options;
    my @opts         = ();
    $options =~ s/\\"/"/g;
    while ( length $options ) {
        my $o = "";
        while ( length $options and $options !~ /^ /s ) {
            if (    ( $options =~ m/^(["'])/ )
                and ( $options !~ m/^(["'])(?:\\.|(?!\1)[^\\])*\1/ ) )
            {
                die wrap_msg( gettext("Cannot parse option line (missing >%s<?): %s"), $1, $options_orig );
            }

            # Extract non quoted parts
            $options =~ s/^([^\\"' ]*)//s;
            $o .= $1 if defined $1;

            # And extract quoted parts
            $options =~ s/^(["'])((?:\\.|(?!\1)[^\\])*)\1//s;
            $o .= $2 if defined $2;
        }
        $options =~ s/^ *//s;
        push @opts, $o;
    }

    return @opts;
}

# Adds the parameters passed in the global section of the config file to the ones passed on the command line
if (    defined $document{''}{"options"}
    and defined $document{''}{"options"}{"global"} )
{
    %po4a_opts = get_options( split_opts( $document{''}{"options"}{"global"} ), @ORIGINAL_ARGV );
}

# make a big pot
my $update_pot_file = 0;
if ( $pot_filename =~ m/\$master/ ) {
    printf( gettext("Split mode, creating a temporary POT:") )
      if $po4a_opts{"verbose"};
    if ( scalar @{ $po4a_opts{"partial"} } ) {
        print wrap_msg( gettext("Disabling --translate-only option, it is not supported in split mode") . "\n" );
        $po4a_opts{"partial"} = [];
    }

    # The POT needs to be generated anyway.
    $update_pot_file = 1;
    $po4a_opts{"split"} = 1;
} else {
    if ( scalar @{ $po4a_opts{"partial"} } ) {

        # Skip documents not specified, strings are read directly from POT file
        foreach my $master ( keys %document ) {
            next unless length $master;
            if ( exists $partial{'master'}{$master} ) {
                print wrap_msg( gettext("Document %s kept for update (--translate-only)."), $master )
                  if ( $po4a_opts{"verbose"} );
            } else {
                print wrap_msg( gettext("Skip the update of %s as requested (--translate-only)."), $master )
                  if ( $po4a_opts{"verbose"} );
                delete $document{$master};
            }
        }

        # Do not read PO files if no file is processed for this language
        my $skipped_langs = "";
        my $kept_langs    = "";
        foreach my $lang ( sort keys %po_filename ) {
            if ( exists $partial{'lang'}{$lang} ) {
                $kept_langs .= " $lang";
            } else {
                $skipped_langs .= " $lang";
                delete $po_filename{$lang};
            }
        }
        if ( $po4a_opts{"verbose"} ) {
            $skipped_langs =~ s/^ //;
            $kept_langs    =~ s/^ //;
            print wrap_msg( gettext("Languages skipped because of --translate-only: %s; kept languages: %s."),
                $skipped_langs, $kept_langs );
        }
    }

    if ( not scalar @{ $po4a_opts{"partial"} } ) {
        if ( -e find_input_file($pot_filename) ) {
            my @statfile = stat find_input_file($pot_filename);
            my $filesize = $statfile[7];
            my $modtime  = $statfile[9];

            # The POT files needs to be generated if it's currently empty (first run?)
            $update_pot_file = 1 if $filesize == 0;

            # The POT needs to be re-generated if a master document is more recent
            # than the POT.
            foreach my $master ( keys %document ) {
                next if ( $master eq '' );
                my $pot_in = find_input_file( $document{$master}{'pot_in'} );
                if ( ( stat $pot_in )[9] >= $modtime ) {
                    $update_pot_file = 1;
                    last;
                }
            }

            if ( ( stat $config_file )[9] > $modtime ) {

                # The configuration file was modified after the POT.
                # Maybe a new document, or new options
                $update_pot_file = 1;
            }

            if ( $po4a_opts{"force"} ) {
                $update_pot_file = 1;
            }

            if ( $po4a_opts{"no-update"} ) {
                print wrap_msg( gettext("NOT updating %s as requested (--no-update)."), $pot_filename )
                  if ( $update_pot_file and $po4a_opts{"verbose"} );
                $update_pot_file = 0;
            } else {
                printf( gettext("Updating %s:"), $pot_filename )
                  if ( $update_pot_file and $po4a_opts{"verbose"} );
            }
        } elsif ( $po4a_opts{"no-update"} ) {
            print wrap_msg( gettext("NOT creating %s as requested (--no-update)."), $pot_filename )
              if $po4a_opts{"verbose"};
        } else {
            printf( gettext("Creating %s:"), $pot_filename )
              if $po4a_opts{"verbose"};
            $update_pot_file = 1;
        }
    }
}

if ($update_pot_file) {
    my %pot_options;
    foreach (qw(porefs wrap-po msgid-bugs-address copyright-holder package-name package-version)) {
        if ( defined $po4a_opts{$_} ) {
            $pot_options{$_} = $po4a_opts{$_};
        }
    }
    $pot_options{'pot-language'} = $po4a_opts{"mastlang"};
    my $potfile = Locale::Po4a::Po->new( \%pot_options );
    foreach my $master (
        sort {
            return -1 if ( $a eq "" );
            return 1  if ( $b eq "" );
            $document{$a}{'pos'} <=> $document{$b}{'pos'}
        } keys %document
      )
    {
        next if ( $master eq '' );
        my $pot_in = $document{$master}{'pot_in'};

        my %file_opts = %po4a_opts;
        my $options =
          ( $document{''}{"options"}{"global"} // '' ) . ' ' .         # global options (if any) as default values
          ( $document{$master}{"options"}{"global"} // '' ) . ' ';     # file's global options (if any) to override them
        $options =~ s/ *$//;
        print STDERR "Options for $master while generating pot: $options\n" if $po4a_opts{'debug'};
        %file_opts = get_options( split_opts($options), @ORIGINAL_ARGV )    if ( length $options );

        my $doc = Locale::Po4a::Chooser::new( $document{$master}{'format'}, %{ $file_opts{"options"} } );

        $doc->setpoout($potfile);
        my @file_in_name;
        push @file_in_name, $pot_in;

        $doc->process(
            'file_in_name'    => \@file_in_name,
            'file_in_charset' => $file_opts{"mastchar"},
            'srcdir'          => $po4a_opts{"srcdir"},
            'destdir'         => $po4a_opts{"destdir"},
            'calldir'         => $po4a_opts{"calldir"}
        );
        $potfile = $doc->getpoout();
    }

    if ( $po4a_opts{"split"} ) {
        ( undef, $pot_filename ) = File::Temp::tempfile(
            "po4a-temp-pot-XXXX",
            DIR    => File::Spec->tmpdir(),
            SUFFIX => ".pot",
            OPEN   => 0,
            UNLINK => 0
        ) or die wrap_msg( gettext("Cannot create a temporary POT file: %s"), $! );
        $potfile->write($pot_filename);
    } else {
        if ( $po4a_opts{"force"} ) {
            $potfile->write( find_output_file($pot_filename) );
        } else {
            $potfile->write_if_needed( find_output_file($pot_filename) );
        }
    }

    print wrap_msg( gettext(" (%d entries)"), $potfile->count_entries() )
      unless ( $po4a_opts{"quiet"} );
} else {
    if ( $po4a_opts{"no-update"} ) {
        print wrap_msg( gettext("NOT updating the POT file %s as requested (--no-update)."), $pot_filename )
          if ( $po4a_opts{"verbose"} );
    } elsif ( scalar @{ $po4a_opts{"partial"} } ) {
        print wrap_msg( gettext("NOT updating the POT file %s because of --translate-only."), $pot_filename )
          if ( $po4a_opts{"verbose"} );
    } else {
        print wrap_msg( gettext("POT file %s already up to date."), $pot_filename )
          if ( $po4a_opts{"verbose"} and not $po4a_opts{"no-update"} );
    }
}

my %split_po;           # po_files: '$lang','$master' => '$path'
my %split_pot;          # pot_files: '$master' => '$path'
my %split_pot_files;    # List of files in a given POT, as a parameter to msggrep: '$potfile' => "-N file1 -N file2"

if ( $po4a_opts{"split"} ) {
    foreach my $master ( keys %document ) {
        next if ( $master eq '' );
        my $m          = $document{$master}{'pot'} || basename $master;
        my $master_pot = $pot_filename_cfg;
        $master_pot =~ s/\$master/$m/g;
        $split_pot{$master} = $master_pot;
        my $searched_file = $document{$master}{'pot_in'} // $master;
        $split_pot_files{$master_pot} //= '';
        $split_pot_files{$master_pot} .= " -N \"$searched_file\"";
    }

    # Generate a .pot for each document by msggrep on the big POT file.
    # These files are used to update the PO files later on (and also by humans to start new translations)
    if ( not $po4a_opts{"no-update"} ) {
        foreach my $master_pot ( keys %split_pot_files ) {
            my $tmp_file;

            # Create a temporary POT, and check if the old one needs to be
            # updated (unless --force was specified).
            unless ( $po4a_opts{"force"} ) {
                ( undef, $tmp_file ) = File::Temp::tempfile(
                    "po4aXXXX",
                    DIR    => File::Spec->tmpdir(),
                    SUFFIX => ".pot",
                    OPEN   => 0,
                    UNLINK => 0
                ) or die wrap_msg( gettext("Cannot create a temporary POT file: %s"), $! );
            }

            my $dir = dirname( find_output_file($master_pot) );
            if ( not -d $dir ) {
                mkdir $dir or die wrap_msg( gettext("Cannot create directory '%s': %s"), $dir, $! );
            }
            my $outfile = $po4a_opts{"force"} ? find_output_file($master_pot) : $tmp_file;
            my $cmd     = "msggrep$Config{_exe} $split_pot_files{$master_pot} -o $outfile $pot_filename --force-po";
            run_cmd($cmd);

            die wrap_msg(
                gettext(
                    "msggrep did not create '%s'. Used command:\n  %s\nPlease report that bug, along the information allowing to reproduce it.\n"
                ),
                $outfile, $cmd
            ) unless ( -e $outfile );

            move_po_if_needed( $tmp_file, find_output_file($master_pot), 0 )
              unless ( $po4a_opts{"force"} );
        }
    }

    # Generate a complete .po for each language, with the translations of all master files
    # That's done with msgcat on all existing PO files of a given language
    foreach my $lang ( sort keys %po_filename ) {
        my $tmp_bigpo;
        ( undef, $tmp_bigpo ) = File::Temp::tempfile(
            "po4aXXXX",
            DIR    => File::Spec->tmpdir(),
            SUFFIX => "-$lang.po",
            OPEN   => 0,
            UNLINK => 0
        ) or die wrap_msg( gettext("Cannot create a temporary PO file: %s"), $! );
        my $cmd_cat = "";
        foreach my $master ( sort keys %document ) {
            next if ( $master eq '' );
            my $m         = $document{$master}{'pot'} || basename $master;
            my $master_po = $po_filename{$lang};
            $master_po =~ s/\$master/$m/g;
            if ( -e find_input_file($master_po) ) {
                $cmd_cat .= " " . find_input_file($master_po);
            } else {
                print wrap_msg(
                    gettext("The translation of master file '%s' in language '%s' is missing (file: %s) -- skipping."),
                    $master, $lang, $master_po )
                  if ( $po4a_opts{"verbose"} );
            }
            $split_po{$lang}{$master} = $master_po;
        }
        if ( length $cmd_cat ) {
            $cmd_cat = "msgcat" . $Config{_exe} . " -o $tmp_bigpo $cmd_cat";
            run_cmd($cmd_cat);
        }

        # We do not need to keep the original name with $master
        $po_filename{$lang} = $tmp_bigpo;
    }
}

if ( not $po4a_opts{"no-update"} ) {

    # update all po files
    my $lang;
    if ( not scalar @{ $po4a_opts{"partial"} } ) {
        foreach $lang ( sort keys %po_filename ) {
            my ( $infile, $outfile ) =
              ( find_input_file( $po_filename{$lang} ), find_output_file( $po_filename{$lang} ) );
            my $updated_potfile = find_input_file($pot_filename);

            my $dir = dirname($outfile);
            if ( not -d $dir ) {
                mkdir $dir or die wrap_msg( gettext("Cannot create directory '%s': %s"), $dir, $! );
            }

            my $usable_pofile = 0;
            if ( -e $infile ) {

                # Check that the po file is not empty, as msgmerge has issues in this case (see Debian's #1022216 and GitHub's #442)

                my $pofile = Locale::Po4a::Po->new();
                $pofile->read( $infile, undef, 0 );    # Guess the charset ; Don't fail if msgfmt spits an error
                $usable_pofile = 1 if ( $pofile->count_entries() > 0 );
            }
            if ($usable_pofile) {
                my $msgmerge_opt = $po4a_opts{"msgmerge-opt"};
                $msgmerge_opt =~ s/\$lang\b/$lang/g if scalar @langs;
                my $cmd = "msgmerge" . $Config{_exe} . " \"$infile\" \"$updated_potfile\" " . $msgmerge_opt;
                if ( $infile eq $outfile ) {           # in place
                    $cmd .= " --backup=none --update";
                } else {
                    $cmd .= " --output \"$outfile\"";
                }
                run_cmd($cmd);

                if ( $po4a_opts{"verbose"} ) {
                    if ( $po4a_opts{'split'} ) {
                        printf( gettext("Updating the translation in language %s:"), $lang );
                    } else {
                        printf( gettext("Updating %s:"), $po_filename{$lang} );
                    }

                    my $stat = qx(msgfmt$Config{_exe} --statistics -v -o $devnull \"$outfile\" 2>&1);
                    $stat =~ s/^[^:]*://;
                    print $stat;
                }

            } else {
                if ( $po4a_opts{"verbose"} ) {
                    if ( $po4a_opts{'split'} ) {
                        printf( gettext("Creating an empty PO file for the language %s.\n"), $lang );
                    } else {
                        printf( gettext("Creating an empty PO file in %s.\n"), $outfile );
                    }
                }

                my $read_pot_filename = find_input_file($pot_filename);
                my $cmd =
                  "msginit$Config{_exe} -i \"$read_pot_filename\" --locale $lang -o \"$outfile\" --no-translator >$devnull";
                run_cmd($cmd);
            }
        }

        if ( $po4a_opts{"split"} ) {

            # Split the complete PO in multiple POs
            foreach $lang ( sort keys %po_filename ) {
                foreach my $master ( keys %document ) {
                    next if ( $master eq '' );
                    my $tmp_file;

                    # Create a temporary PO, and check if the old one needs to be
                    # updated (unless --force was specified).
                    ( undef, $tmp_file ) = File::Temp::tempfile(
                        "po4aXXXX",
                        DIR    => File::Spec->tmpdir(),
                        SUFFIX => ".po",
                        OPEN   => 0,
                        UNLINK => 0
                    ) or die wrap_msg( gettext("Cannot create a temporary POT file: %s"), $! );

                    my $cmd;

                    # Create an empty PO or copy the original PO header (to keep the header)
                    if ( -f find_input_file( $split_po{$lang}{$master} ) ) {

                        # Pass msggrep to C (english), but not under windows as this syntax is not understood by the shell
                        my $env = ( $^O =~ /Win/ ) ? "" : "LC_ALL=C.UTF-8";

                        $cmd =
                            $env
                          . " msggrep$Config{_exe}"
                          . " --force-po --invert-match --msgid --regexp '.'"
                          . " --output \"$tmp_file\" "
                          . find_input_file( $split_po{$lang}{$master} );
                    } else {
                        $cmd =
                            "msginit$Config{_exe} "
                          . "--no-translator -l $lang --input " . '"'
                          . find_output_file( $split_pot{$master} ) . '"'
                          . " --output $tmp_file  >$devnull";
                    }
                    run_cmd($cmd);

                    # Update the PO according to the new POT and to the big PO (compendium).
                    $cmd =
                        "msgmerge$Config{_exe} "
                      . " --compendium " . '"'
                      . $po_filename{$lang} . '"'
                      . " --update --backup=none "
                      . $po4a_opts{"msgmerge-opt"}
                      . " \"$tmp_file\" " . '"'
                      . find_output_file( $split_pot{$master} ) . '"';
                    run_cmd($cmd);

                    my $dir = dirname( $split_po{$lang}{$master} );
                    if ( not -d $dir ) {
                        mkdir $dir
                          or die wrap_msg( gettext("Cannot create directory '%s': %s"), $dir, $! );
                    }
                    unless ( $po4a_opts{"force"} ) {
                        move_po_if_needed( $tmp_file, $split_po{$lang}{$master}, 0 );
                    } else {
                        move $tmp_file,
                          find_output_file( $split_po{$lang}{$master} )
                          or die wrap_msg(
                            dgettext( "po4a", "Cannot move %s to %s: %s." ), $tmp_file,
                            find_output_file( $split_po{$lang}{$master} ),   $!
                          );
                    }
                }
            }
        }    # if ( $po4a_opts{"split"} )
    }    # if ( not scalar @{ $po4a_opts{"partial"} } )
}    # not $po4a_opts{"no-update"}

if ( $po4a_opts{"split"} ) {

    # We don't need the tmp big POT anymore
    unlink($pot_filename);
    print STDERR "Unlink the big POT file $pot_filename\n" if $po4a_opts{'debug'};
}

if ( not $po4a_opts{"no-translations"} ) {

    # update all translations

  LANG: foreach my $lang ( sort keys %po_filename ) {

        # Read the $lang PO once, no options for the creation
        my $po     = Locale::Po4a::Po->new();
        my $pofile = find_input_file( $po_filename{$lang} );

        if ( not -e $pofile ) {
            if ( $po4a_opts{"verbose"} ) {
                if ( File::Spec->file_name_is_absolute( $po_filename{$lang} ) ) {
                    print wrap_msg( gettext("PO file for language %s is missing -- skipping."), $lang );
                } else {
                    print wrap_msg( gettext("PO file %s for language %s is missing -- skipping."),
                        $po_filename{$lang}, $lang );
                }
            }
            next LANG;
        }
        print "Reading PO file $pofile for language $lang: "
          if ( $po4a_opts{"debug"} );
        $po->read($pofile);
        system( "msgfmt" . $Config{_exe} . " --statistics -v -o $devnull \"$pofile\"" )
          if ( $po4a_opts{"debug"} );

      DOC: foreach my $master (
            sort {
                return -1 if ( $a eq "" );
                return 1  if ( $b eq "" );
                $document{$a}{'pos'} <=> $document{$b}{'pos'}
            } keys %document
          )
        {
            next if ( $master eq '' );
            next unless defined $document{$master}{$lang};
            if ( scalar @{ $po4a_opts{"partial"} } ) {
                unless ( defined $partial{'files'}{ $document{$master}{$lang} } ) {
                    print wrap_msg( gettext("Skip master file %s in language %s because of --translate-only."),
                        $master, $lang )
                      if ( $po4a_opts{"verbose"} );
                    next;
                }
            }

            unless ( $po4a_opts{"force"} ) {
                my $outfile   = find_input_file( $document{$master}{$lang} );
                my $stampfile = ( -e $outfile ? $outfile : "$outfile.po4a-stamp" );

                $stampfile = File::Spec->rel2abs($stampfile);

                my @files = ( $master, $po_filename{$lang}, File::Spec->rel2abs($config_file) );
                if ( defined $document{$master}{"add_$lang"} ) {
                    push @files, @{ $document{$master}{"add_$lang"} };
                }
                push @files, @{ $add_filename{$lang} } if ( defined $add_filename{$lang} );

                unless ( is_older( $stampfile, @files ) ) {
                    print wrap_msg( gettext("%s doesn't need to be updated."), $document{$master}{$lang} )
                      if ( $po4a_opts{"verbose"} );
                    next DOC;
                }
            }

            my %file_opts = %po4a_opts;
            my $options =
              ( $document{''}{"options"}{"global"} // '' ) . ' ' .         # global options (if any) as default values
              ( $document{$master}{"options"}{"global"} // '' ) . ' '
              .                                                 # file's global options (if any) to override them
              ( $document{$master}{"options"}{$lang} // '' );   # file's language options (if any) overriding everything
            $options =~ s/ *$//;
            print STDERR "Options for $master in $lang: $options\n"          if $po4a_opts{'debug'};
            %file_opts = get_options( split_opts($options), @ORIGINAL_ARGV ) if ( length $options );

            my $doc = Locale::Po4a::Chooser::new( $document{$master}{'format'}, %{ $file_opts{"options"} } );

            my @file_in_name;
            push @file_in_name, $master;

            # Reuse the already parsed PO, do not use the po_in_name option of process.
            $doc->{TT}{po_in} = $po;
            $doc->{TT}{po_in}->stats_clear();

            my $translation_out;
            if ( $po4a_opts{"keep-translations"}
                && -e find_output_file( $document{$master}{$lang} ) )
            {
                ( undef, $translation_out ) = File::Temp::tempfile(
                    "$document{$master}{$lang}.new-XXXX",
                    OPEN   => 0,
                    UNLINK => 0
                );
            } else {
                $translation_out = $document{$master}{$lang};
            }

            $doc->process(
                'file_in_name'     => \@file_in_name,
                'file_out_name'    => $translation_out,
                'file_in_charset'  => $file_opts{"mastchar"},
                'file_out_charset' => $file_opts{"locchar"},
                'addendum_charset' => $file_opts{"addchar"},
                'srcdir'           => $po4a_opts{"srcdir"},
                'destdir'          => $po4a_opts{"destdir"},
                'calldir'          => $po4a_opts{"calldir"}
            );

            my ( $percent, $hit, $queries ) = $doc->stats();
            if ( $percent < $file_opts{"threshold"} ) {
                print wrap_msg(
                    gettext("Discard %s (%s of %s strings; only %s%% translated; need %s%%)."),
                    $document{$master}{$lang},
                    $hit, $queries, $percent, $file_opts{"threshold"}
                );
                unlink( find_output_file( $document{$master}{$lang} ) )
                  if ( !$po4a_opts{"keep-translations"} );
                unlink( find_output_file($translation_out) )
                  if ( $po4a_opts{"keep-translations"} );
                if ( $po4a_opts{"stamp"} && ( not $po4a_opts{"force"} ) ) {
                    touch( find_output_file( $document{$master}{$lang} ) . ".po4a-stamp" );
                    print wrap_msg( gettext("Timestamp %s created."), $document{$master}{$lang} . ".po4a-stamp" )
                      if ( $po4a_opts{"verbose"} );
                }
                next DOC;

            } else {    # The translated document reached the threshold and was kept
                rename( $translation_out, find_output_file( $document{$master}{$lang} ) );

                unless ( $po4a_opts{"force"} ) {
                    if ( -e $document{$master}{$lang} . ".po4a-stamp" ) {
                        unlink find_output_file( $document{$master}{$lang} ) . ".po4a-stamp";
                        print wrap_msg( gettext("Timestamp %s removed."), $document{$master}{$lang} . ".po4a-stamp" )
                          if ( $po4a_opts{"verbose"} );
                    }
                }
            }

            my @addlist;
            push @addlist, @{ $document{$master}{"add_$lang"} }
              if ( defined( $document{$master}{"add_$lang"} ) );
            push @addlist, @{ $add_filename{$lang} }
              if ( defined( $add_filename{$lang} ) );

            foreach my $add (@addlist) {
                if ( !$doc->addendum( find_input_file($add) ) ) {
                    die wrap_msg( gettext("Addendum %s does NOT apply to %s (translation discarded)."),
                        $add, $document{$master}{$lang} );
                    my $outfile = find_output_file( $document{$master}{$lang} );
                    unlink($outfile) if ( -e $outfile );
                    next DOC;
                }
            }
            if ( $file_opts{"verbose"} ) {
                if ( $percent == 100 ) {
                    print wrap_msg( gettext("%s is 100%% translated (%s strings)."),
                        $document{$master}{$lang}, $queries );
                } else {
                    print wrap_msg(
                        gettext("%s is %s%% translated (%s of %s strings)."),
                        $document{$master}{$lang},
                        $percent, $hit, $queries
                    );
                }
            }

            $doc->write( find_output_file( $document{$master}{$lang} ), $file_opts{"locchar"} // '' );
        }
    }
}

if ( $po4a_opts{"split"} ) {

    # We don't need the tmp big POs anymore
    foreach my $lang ( keys %po_filename ) {
        unlink $po_filename{$lang};
    }
}

if ( $po4a_opts{"rm-translations"} ) {

    # Delete the translated documents
    foreach my $lang ( keys %po_filename ) {
        foreach my $master ( keys %document ) {
            next if ( $master eq '' );
            my $outfile = find_output_file( $document{$master}{$lang} );
            unlink $outfile;
            unlink "$outfile.po4a-stamp";
        }
    }
}

sub touch {
    my $file = shift;
    if ( -e $file ) {
        utime undef, undef, $file;
    } else {
        sysopen( FH, $file, O_WRONLY | O_CREAT | O_NONBLOCK | O_NOCTTY )
          or croak("Cannot create $file : $!");
        close FH or croak("Cannot close $file : $!");
    }
}

sub is_older {
    my $file  = shift;
    my @files = @_;
    return 1 unless ( -e $file );
    my $older = 0;

    my $modtime = ( stat $file )[9];

    for my $f (@files) {
        if ( -e $f and ( stat $f )[9] > $modtime ) {
            $older = 1;
            last;
        }
    }

    return $older;
}

__END__

#  LocalWords: Charset charset AsciiDoc tablecells po UTF gettext msgid nostrip
#  LocalWords:  xml docbook