Render XInclide failures, small changes on configure.php and translation qaxml sync tools #277

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open

alfsb wants to merge 4 commits into php:master from alfsb:cleanup-qaxml

configure.php

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -1,9 +1,8 @@
  
    #!/usr/bin/env php

    <?php // vim: ts=4 sw=4 et tw=78 fdm=marker

    /*

      +----------------------------------------------------------------------+

      | Copyright (c) 1997-2023 The PHP Group                                |

      | Copyright (c) 1997-2026 The PHP Group                                |

      +----------------------------------------------------------------------+

      | This source file is subject to version 3.01 of the PHP license,      |

      | that is bundled with this package in the file LICENSE, and is        |

    @@ -16,7 +15,7 @@
  
      | Authors:    Dave Barr <dave@php.net>                                 |

      |             Hannes Magnusson <bjori@php.net>                         |

      |             Gwynne Raskind <gwynne@php.net>                          |

      |             André L F S Bacci <gwynne@php.net>                       |

      |             André L F S Bacci <ae@php.net>                           |

      +----------------------------------------------------------------------+

    */

    @@ -27,16 +26,27 @@
  
    echo "configure.php on PHP " . phpversion() . ", libxml " . LIBXML_DOTTED_VERSION . "\n\n";

    // init_argv()

    // init_checks()

    // init_clean()

    // xml_configure()

    // xml_parse()

    // xml_xinclude()

    // xml_validate()

    // phd_sources()

    // phd_version()

    // php_history()

    // gereral structure/ordeing for refactoring this code

    // init_parse()     // todo: argv parsing into a typed static class, remove all global

    // init_check()     // todo: move all checks in one place

    // init_usage()

    // git_clean()                  partial done

    // git_status()                 partial done

    // dtd_conf_entities()          done

    // dtd_file_entities()          done

    // dom_load/save()              done

    // xinclude_byid()              done

    // xinclude_xpointer()          done

    // xinclude_residua()           done

    // xml_partial_output

    // xml_validation

    // phd_acronym()                done

    // phd_sources()                done

    // phd_version()                done

    // php_history()                done

    // ugly: make_scripts_executable

    const RNG_SCHEMA_DIR = __DIR__ . DIRECTORY_SEPARATOR . 'docbook' . DIRECTORY_SEPARATOR . 'docbook-v5.2-os' . DIRECTORY_SEPARATOR . 'rng' . DIRECTORY_SEPARATOR;

    const RNG_SCHEMA_FILE = RNG_SCHEMA_DIR . 'docbook.rng';

    @@ -573,9 +583,10 @@ function find_xml_files($path) // {{{
  
        $output = str_replace( "\n\n" , "\n" , $output );

    echo "\n" , trim( $output ) . "\n\n";

    // DTD configuration before first loading

    xml_configure();

    function xml_configure()

    dtd_conf_entities();

    function dtd_conf_entities()

    {

        global $ac;

        $lang = $ac["LANG"];

    @@ -617,21 +628,28 @@ function xml_configure()
  
    globbetyglob("{$ac['basedir']}/scripts", 'make_scripts_executable');

    {   # file-entities.php

        $cmd = array();

        $cmd[] = $ac['PHP'];

        $cmd[] = __DIR__ . "/scripts/file-entities.php";

        if ( $ac["LANG"] != "en" )

            $cmd[] = $ac["LANG"];

        if ( $ac['CHMENABLED'] == 'yes' )

            $cmd[] = '--chmonly';

        foreach ( $cmd as & $part )

    dtd_file_entities();

    function dtd_file_entities()

    {

        global $ac;

        $lang = $ac["LANG"];

        $withphp = $ac['PHP'];

        $withchm = $ac['CHMENABLED'] == 'yes';

        $parts = array();

        $parts[] = $withphp;

        $parts[] = __DIR__ . "/scripts/file-entities.php";

        if ( $lang != "en" )

            $parts[] = $lang;

        if ( $withchm )

            $parts[] = '--chmonly';

        foreach ( $parts as & $part )

            $part = escapeshellarg( $part );

        $cmd = implode( ' ' , $parts );

        $ret = 0;

        $cmd = implode( ' ' , $cmd );

        passthru( $cmd , $ret );

        if ( $ret != 0 )

        {

            echo "doc-base/scripts/file-entities.php FAILED.\n";

    @@ -656,6 +674,23 @@ function xml_configure()
  
    checking('whether to save an invalid .manual.xml');

    checkvalue($ac['FORCE_DOM_SAVE']);

    echo "Loading and parsing {$ac["INPUT_FILENAME"]}... ";

    $dom = new DOMDocument();

    if ( dom_load( $dom , "{$ac['srcdir']}/{$ac["INPUT_FILENAME"]}" ) )

    {

        echo "1 ";

        dom_saveload( $dom ); // correct file/line/column on error messages

        echo "2 done.\n";

    }

    else

    {

        echo "failed.\n";

        print_xml_errors();

        dom_broken_files();

        errors_are_bad(1);

    }

    function dom_load( DOMDocument $dom , string $filename , string $baseURI = "" ) : bool

    {

        $filename = realpath( $filename );

    @@ -674,24 +709,7 @@ function dom_saveload( DOMDocument $dom , string $filename = "" ) : string
  
        return $filename;

    }

    echo "Loading and parsing {$ac["INPUT_FILENAME"]}... ";

    $dom = new DOMDocument();

    if ( dom_load( $dom , "{$ac['srcdir']}/{$ac["INPUT_FILENAME"]}" ) )

    {

        echo "1 ";

        dom_saveload( $dom ); // correct file/line/column on error messages

        echo "2 done.\n";

    }

    else

    {

        echo "failed.\n";

        print_xml_errors();

        individual_xml_broken_check();

        errors_are_bad(1);

    }

    function individual_xml_broken_check()

    function dom_broken_files()

    {

        $cmd = array();

        $cmd[] = $GLOBALS['ac']['PHP'];

    @@ -724,7 +742,7 @@ function individual_xml_broken_check()
  
    function xinclude_run_byid( DOMDocument $dom )

    {

        $total = 0;

        $maxrun = 10; //LIBXML_VERSION >= 21100 ? 1 : 10;

        $maxrun = 10;

        for( $run = 0 ; $run < $maxrun ; $run++ )

        {

            echo "$run ";

    @@ -767,7 +785,7 @@ function xinclude_run_byid( DOMDocument $dom )
  
            if ( ! $changed )

                return $total;

        }

        echo "XInclude nested too deeply (xml:id).\n";

        echo "XInclude nested too deeply (by xml:id). Update `configure.php`.\n";

        errors_are_bad( 1 );

    }

    @@ -791,7 +809,7 @@ function xinclude_run_xpointer( DOMDocument $dom ) : int
  
            if ( $was === $now )

                return $total;

        }

        echo "XInclude nested too deeply (xpointer).\n";

        echo "XInclude nested too deeply (xpointer). Update `configure.php`.\n";

        errors_are_bad( 1 );

    }

    @@ -800,52 +818,54 @@ function xinclude_residual_fixup( DOMDocument $dom )
  
        // XInclude failures are soft errors on translations, so remove

        // residual XInclude tags on translations to keep them building.

        $debugFile = "temp/xinclude-debug.xml";

        $debugPath = __DIR__ . "/{$debugFile}";

        $nodes = xinclude_residual_list( $dom );

        $debugFile1 = "temp/xinclude-fixup-before.xml";

        $debugFile2 = "temp/xinclude-fixup-result.xml";

        $nodes = xinclude_residual_list( $dom );

        if ( count( $nodes ) > 0 )

        {

            unset( $nodes );

            dom_saveload( $dom , $debugPath );

            $nodes = $nodes = xinclude_residual_list( $dom );

            dom_saveload( $dom , __DIR__ . "/{$debugFile1}" );

            $nodes = xinclude_residual_list( $dom );

        }

        $count = 0;

        $fixups = 0;

        $explain = false;

        foreach( $nodes as $node )

        {

            if ( $count === 0 )

            {

                echo "\nFailed XIncludes, manual parts will be missing.";

                echo " Inspect {$debugFile} for context. Failed targets are:\n";

            }

            echo "- {$node->getAttribute("xpointer")}\n";

            $count++;

            $fixup = null;

            $parent = $node->parentNode;

            $tagName = $parent->nodeName;

            switch( $tagName )

            $parent = $node->parentNode->nodeName;

            $target = $node->getAttribute("xpointer");

            $alert = "[Failed XInclude '$target']";

            if ( $fixups === 0 )

                echo "\nFailed XIncludes, manual parts will be missing. Failed XInclude targets:\n";

            echo "- {$target}\n";

            $fixups++;

            switch( $parent )

            {

                case "listitem":

                    $fixup = "<para>$alert</para>";

                case "refentry":

                    $fixup = "";

                    break;

                case "refsect1":

                    $fixup = "<title>_</title><simpara>_</simpara>"; // https://github.com/php/phd/issues/181

                    $fixup = "<title>_</title><simpara>$alert</simpara>"; // https://github.com/php/phd/issues/181

                    break;

                case "tbody":

                    $fixup = "<row><entry></entry></row>";

                    $fixup = "<row><entry>$alert</entry></row>";

                    break;

                case "variablelist":

                    $fixup = "<varlistentry><term></term><listitem><simpara></simpara></listitem></varlistentry>";

                    $fixup = "<varlistentry><term></term><listitem><simpara>$alert</simpara></listitem></varlistentry>";

                    break;

                default:

                    echo "Unknown parent of failed XInclude: $tagName\n";

                    echo "  (Unknown parent of failed XInclude: $parent)\n";

                    $explain = true;

                    continue 2;

            }

            if ( $fixup !== null )

            {

                $other = new DOMDocument( '1.0' , 'utf8' );

    @@ -861,22 +881,32 @@ function xinclude_residual_fixup( DOMDocument $dom )
  
        if ( $explain )

        {

            dom_saveload( $dom , __DIR__ . "/{$debugFile2}" );

            echo <<<MSG

    \nIf you are seeing this message on a translation, this means that

    XInclude/XPointers failures reported above are so many or unknown,

    that configure.php cannot patch the translated manual into a validating

    state. Please report any "Unknown parent" messages on the doc-base

    repository, and focus on fixing XInclude/XPointers failures above.\n\n

    If you are seeing this message in a translation, it means that

    the XInclude/XPointers failures reported above are so numerous or unknown,

    that configure.php cannot fix up the translated manual to a validating state.

    Please report any "Unknown parent" messages to the doc-base repository

    and focus on fixing all the XInclude/XPointers failures listed above.

    Dumped {$debugFile1} .

    Dumped {$debugFile2} .

    MSG;

            exit( 1 ); // stop here, do not let more messages further confuse the matter

        }

        if ( $count > 0 )

        if ( $fixups > 0 )

            echo "\n";

        // XInclude by xml:id never duplicates xml:id, horever, also using

        // XInclude by XPath/XPointer may start causing duplications

        // (see docs/structure.md). Crude and ugly fixup ahead, beware!

        // XInclude by xml:id never duplicates xml:id. Horever, using

        // XInclude by XPath/XPointer with a XInclude 1.0 library will cause

        // xml:id duplication, as xml:id have no special tratament in this version.

        // See docs/structure.md for details.

        // Crude and ugly fixup ahead, beware!

        $list = [];

        $see = false;

    @@ -897,12 +927,18 @@ function xinclude_residual_fixup( DOMDocument $dom )
  
            $list[ $id ] = $id;

        }

        if ( $see )

            echo "  See: https://github.com/php/doc-base/blob/master/docs/structure.md#xmlid-structure\n";

        {

            echo "\n  See: https://github.com/php/doc-base/blob/master/docs/structure.md#xmlid-structure";

            echo "\n  And: https://github.com/php/doc-base/tree/master/scripts/translation";

            echo "\n  In special, qaxml-attributes.php and qaxml-entities.php, with and without --urgent.";

            echo "\n\n";

        }

        $fatal = $GLOBALS['ac']['LANG'] == 'en';

        // Duplicated strucutral xml:ids are fatal on doc-en

        $fatal = $GLOBALS['ac']['LANG'] == 'en';

        if ( $see && $fatal )

            errors_are_bad( 1 ); // Duplicated strucutral xml:ids are fatal on doc-en

            errors_are_bad( 1 );

    }

    function xinclude_residual_list( DOMDocument $dom ) : DOMNodeList

    @@ -1156,5 +1192,4 @@ function phd_version()
  
    CAT;

    individual_xml_broken_check();

    exit(0); // Finished successfully.

scripts/translation/README.md

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -6,13 +6,13 @@ for inconsistencies. These tools check for structural differences
  
    that may cause translation build failures or non-validating DocBook XML

    results, and fixing these issues will help avoid build failures.

    Some checks are less structural, and as not all translations are identical,

    Some checks are not strictly structural, and as not all translations are identical,

    or use the same conventions, they may not be entirely applicable in all

    languages. Even two translators working on one language may have different

    opinions on how much synchronization is wanted, so not all scripts will be of

    use for all translations.

    Because of the above, it's possible to silence each alert indempendly. These

    Because of the above, it's possible to silence each output indempendly. These

    scripts will output `--add-ignore` commands that, if executed, will omit the

    specific alerts in future executions.

    @@ -25,8 +25,8 @@ or if XML contents are not
  
    Unbalanced XML contents are invalid XML and will result in a broken build.

    BOM and CR marks may not result in broken builds, but *will* cause several

    tools below to misbehave, as `libxml` behaviour changes if XML text contains

    these bytes.

    tools below to misbehave, as *behaviour* of `libxml` itself changes if XML

    text contains these bytes.

    ## qaxml-attributes.php

    @@ -77,7 +77,7 @@ mismatched tag, to facilitate the work on big files.
  
    This script also accepts an `--content=` option, that will check the

    *contents* of tags, to inspect tags where the contents are expected *not* to

    be translated. Example below.

    be translated. See example below.

    ## qaxml-ws.php

    @@ -95,13 +95,13 @@ locally generated `revcheck.php` status pages.
  
    ## Suggested execution

    The first execution of these scripts may generate an inordinate amount of

    alerts. It's advised to initially run each command separately, and work the

    The first execution of these scripts may generate an inordinate amount of

    alerts. It's advised to initially run each command separately, and work the

    alerts on a case by case basis. After all interesting cases are fixed,

    it's possible to rerun the command and `grep` the output for `--add-ignore`

    lines, run these commands, and by so, mass ignore the residual alerts.

    Structural checks:

    Strict structural checks:

    ```

    php doc-base/scripts/broken.php

    @@ -114,7 +114,7 @@ php doc-base/scripts/translation/qaxml-tags.php --detail
  
    php doc-base/scripts/translation/qaxml-ws.php

    ```

    Tags where is expected no translations:

    Tags where is expected **no** translations:

    ```

    php doc-base/scripts/translation/qaxml-tags.php --content=acronym

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Render XInclide failures, small changes on configure.php and translation qaxml sync tools #277

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!

Uh oh!

Render XInclide failures, small changes on configure.php and translation qaxml sync tools #277

Are you sure you want to change the base?

Uh oh!

Render XInclide failures, small changes on configure.php and translation qaxml sync tools #277

Uh oh!

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!

Uh oh!