Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

strip empty starting/ending php tags #1329

Merged
merged 42 commits into from
Dec 3, 2021
Merged

Conversation

staabm
Copy link
Contributor

@staabm staabm commented Nov 28, 2021

@TomasVotruba
Copy link
Member

TomasVotruba commented Nov 28, 2021

Thank you for working on it 👍

We'll need a test for this too to avoid regressions.

@staabm
Copy link
Contributor Author

staabm commented Nov 28, 2021

@TomasVotruba where to put the test?

I cannot use the regular fixture format because its important that the AST starts with a Stmt\InlineHTML-node. also the problem is not related to any specific rule.

the error happens only in the 3rd and 4th phase of the rector run:
(I guess the regular rector unit tests do not have those multiple-phase processing?)

$ vendor/bin/rector process view.php -n --debug
[parsing] view.php
[refactoring] view.php
[post rectors] view.php
    [post rector] Rector\PostRector\Rector\NodeToReplacePostRector
    [post rector] Rector\PostRector\Rector\NodeAddingPostRector
    [post rector] Rector\PostRector\Rector\PropertyAddingPostRector
    [post rector] Ssch\TYPO3Rector\Rector\PostRector\FullQualifiedNamePostRector
    [post rector] Rector\PostRector\Rector\NodeRemovingPostRector
    [post rector] Rector\PostRector\Rector\ClassRenamingPostRector
    [post rector] Rector\PostRector\Rector\NameImportingPostRector
    [post rector] Rector\PostRector\Rector\UseAddingPostRector
[print] view.php
[refactoring] view.php
[post rectors] view.php
    [post rector] Rector\PostRector\Rector\NodeToReplacePostRector
    [post rector] Rector\PostRector\Rector\NodeAddingPostRector
    [post rector] Rector\PostRector\Rector\PropertyAddingPostRector
    [post rector] Ssch\TYPO3Rector\Rector\PostRector\FullQualifiedNamePostRector
    [post rector] Rector\PostRector\Rector\NodeRemovingPostRector
    [post rector] Rector\PostRector\Rector\ClassRenamingPostRector
    [post rector] Rector\PostRector\Rector\NameImportingPostRector
    [post rector] Rector\PostRector\Rector\UseAddingPostRector
[print] view.php
[refactoring] view.php
[post rectors] view.php
    [post rector] Rector\PostRector\Rector\NodeToReplacePostRector
    [post rector] Rector\PostRector\Rector\NodeAddingPostRector
    [post rector] Rector\PostRector\Rector\PropertyAddingPostRector
    [post rector] Ssch\TYPO3Rector\Rector\PostRector\FullQualifiedNamePostRector
    [post rector] Rector\PostRector\Rector\NodeRemovingPostRector
    [post rector] Rector\PostRector\Rector\ClassRenamingPostRector
    [post rector] Rector\PostRector\Rector\NameImportingPostRector
    [post rector] Rector\PostRector\Rector\UseAddingPostRector
[print] view.php
[refactoring] view.php
[post rectors] view.php
    [post rector] Rector\PostRector\Rector\NodeToReplacePostRector
    [post rector] Rector\PostRector\Rector\NodeAddingPostRector
    [post rector] Rector\PostRector\Rector\PropertyAddingPostRector
    [post rector] Ssch\TYPO3Rector\Rector\PostRector\FullQualifiedNamePostRector
    [post rector] Rector\PostRector\Rector\NodeRemovingPostRector
    [post rector] Rector\PostRector\Rector\ClassRenamingPostRector
    [post rector] Rector\PostRector\Rector\NameImportingPostRector
    [post rector] Rector\PostRector\Rector\UseAddingPostRector
[print] view.php

I think I need some kind of end2end test

@samsonasik
Copy link
Member

There is e2e dir https://github.com/rectorphp/rector-src/tree/main/e2e

You can add more subdir and register subdir to e2e workflow

@staabm
Copy link
Contributor Author

staabm commented Nov 29, 2021

hey,

thanks for the pointer. I just added a e2e test, but it does not reproduce the problem yet.

in my example repo I can see rector running 4 times.. when doing the same with the very same files within the e2e dir, rector is only running 2 times and the error does not reproduce.

my example repo:

$ vendor/bin/rector process view.php -n --debug
[parsing] view.php
[refactoring] view.php
[post rectors] view.php
    [post rector] Rector\PostRector\Rector\NodeToReplacePostRector
    [post rector] Rector\PostRector\Rector\NodeAddingPostRector
    [post rector] Rector\PostRector\Rector\PropertyAddingPostRector
    [post rector] Ssch\TYPO3Rector\Rector\PostRector\FullQualifiedNamePostRector
    [post rector] Rector\PostRector\Rector\NodeRemovingPostRector
    [post rector] Rector\PostRector\Rector\ClassRenamingPostRector
    [post rector] Rector\PostRector\Rector\NameImportingPostRector
    [post rector] Rector\PostRector\Rector\UseAddingPostRector
[print] view.php
[refactoring] view.php
[post rectors] view.php
    [post rector] Rector\PostRector\Rector\NodeToReplacePostRector
    [post rector] Rector\PostRector\Rector\NodeAddingPostRector
    [post rector] Rector\PostRector\Rector\PropertyAddingPostRector
    [post rector] Ssch\TYPO3Rector\Rector\PostRector\FullQualifiedNamePostRector
    [post rector] Rector\PostRector\Rector\NodeRemovingPostRector
    [post rector] Rector\PostRector\Rector\ClassRenamingPostRector
    [post rector] Rector\PostRector\Rector\NameImportingPostRector
    [post rector] Rector\PostRector\Rector\UseAddingPostRector
[print] view.php
[refactoring] view.php
[post rectors] view.php
    [post rector] Rector\PostRector\Rector\NodeToReplacePostRector
    [post rector] Rector\PostRector\Rector\NodeAddingPostRector
    [post rector] Rector\PostRector\Rector\PropertyAddingPostRector
    [post rector] Ssch\TYPO3Rector\Rector\PostRector\FullQualifiedNamePostRector
    [post rector] Rector\PostRector\Rector\NodeRemovingPostRector
    [post rector] Rector\PostRector\Rector\ClassRenamingPostRector
    [post rector] Rector\PostRector\Rector\NameImportingPostRector
    [post rector] Rector\PostRector\Rector\UseAddingPostRector
[print] view.php
[refactoring] view.php
[post rectors] view.php
    [post rector] Rector\PostRector\Rector\NodeToReplacePostRector
    [post rector] Rector\PostRector\Rector\NodeAddingPostRector
    [post rector] Rector\PostRector\Rector\PropertyAddingPostRector
    [post rector] Ssch\TYPO3Rector\Rector\PostRector\FullQualifiedNamePostRector
    [post rector] Rector\PostRector\Rector\NodeRemovingPostRector
    [post rector] Rector\PostRector\Rector\ClassRenamingPostRector
    [post rector] Rector\PostRector\Rector\NameImportingPostRector
    [post rector] Rector\PostRector\Rector\UseAddingPostRector
[print] view.php

1 file with changes
===================

1) view.php:0

    ---------- begin diff ----------
@@ @@
 #Warning: Strings contain different line endings!
+<?php
+
+?>

        <div>
-       <?php echo 1 ?>
-       </div>
+       <?php
+echo 1 ?>
+
+?>
+       </div><?php
    ----------- end diff -----------


 [OK] 1 file would have changed (dry-run) by Rector

after I copied my repro data into the rector-src I get only

$ cd rector/e2e/plain-views/
$ rector  -n --debug
[parsing] src/view.php
[refactoring] src/view.php
[post rectors] src/view.php
    [post rector] Rector\PostRector\Rector\NodeToReplacePostRector
    [post rector] Rector\PostRector\Rector\NodeAddingPostRector
    [post rector] Rector\PostRector\Rector\PropertyAddingPostRector
    [post rector] Ssch\TYPO3Rector\Rector\PostRector\FullQualifiedNamePostRector
    [post rector] Rector\PostRector\Rector\NodeRemovingPostRector
    [post rector] Rector\PostRector\Rector\ClassRenamingPostRector
    [post rector] Rector\PostRector\Rector\NameImportingPostRector
    [post rector] Rector\PostRector\Rector\UseAddingPostRector
[print] src/view.php
[refactoring] src/view.php
[post rectors] src/view.php
    [post rector] Rector\PostRector\Rector\NodeToReplacePostRector
    [post rector] Rector\PostRector\Rector\NodeAddingPostRector
    [post rector] Rector\PostRector\Rector\PropertyAddingPostRector
    [post rector] Ssch\TYPO3Rector\Rector\PostRector\FullQualifiedNamePostRector
    [post rector] Rector\PostRector\Rector\NodeRemovingPostRector
    [post rector] Rector\PostRector\Rector\ClassRenamingPostRector
    [post rector] Rector\PostRector\Rector\NameImportingPostRector
    [post rector] Rector\PostRector\Rector\UseAddingPostRector
[print] src/view.php

1 file with changes
===================

1) src/view.php:0

    ---------- begin diff ----------
@@ @@
-
-       <div>
+<div>
        <?php echo 1 ?>
        </div>
    ----------- end diff -----------


 [OK] 1 file would have changed (dry-run) by Rector

so rector is only working in 2 phases but in my repo it works in 4?

@staabm
Copy link
Contributor Author

staabm commented Nov 29, 2021

ohh I just found out, that its important to the repro that there is no newline at the end of the view.php file.

so my final question is: where/how to implement the assertion of the expected behaviour in the e2e test?

@samsonasik
Copy link
Member

No need to make assertion, just ensure that e2e dry run doesn't apply diff changes

As you work on branch; you can update workflow to run on current with update command to something like:

../../bin/rector process src --dry-run

@staabm
Copy link
Contributor Author

staabm commented Nov 29, 2021

No need to make assertion, just ensure that e2e dry run doesn't apply diff changes

when changing the view.php until rector does not longer want to apply changes, the actual bug is no longer reproducible

I need the view.php to look like

	<div>
	<?php echo 1 ?>
	</div>

without the fix from this PR rector tries to insert some php starting/ending tags:

+<?php
+
+?>

        <div>
-       <?php echo 1 ?>
-       </div>
+       <?php
+echo 1 ?>
+
+?>
+       </div><?php

with this PRs fix applied, rector only tries to mangle the whitespacing:

        <div>
-       <?php echo 1 ?>
-       </div>
+       <?php
+echo 1 ?>
+
+?>
+       </div>

if I change the view.php to what rector tries to change, the initial problem is not reproducable anymore, because to reproduce the problem I need these whitespaces arround in view.php

@samsonasik
Copy link
Member

samsonasik commented Nov 29, 2021

You can try change line:

run: vendor/bin/rector process src --dry-run --ansi

to use existing bin/rector instead of by composer:

 run: ./../../bin/rector process src --dry-run --ansi 

@staabm
Copy link
Contributor Author

staabm commented Nov 29, 2021

to use existing bin/rector instead of by composer:

I tried adding this change with d691594

the github actions are now running into errors, which look weird:

PHP Fatal error:  Uncaught Error: Class "Rector\Core\Bootstrap\RectorConfigsResolver" not found in /home/runner/work/rector-src/rector-src/bin/rector.php:43
Stack trace:
#0 /home/runner/work/rector-src/rector-src/bin/rector(4): require_once()
#1 {main}
  thrown in /home/runner/work/rector-src/rector-src/bin/rector.php on line 43
Fatal error: Uncaught Error: Class "Rector\Core\Bootstrap\RectorConfigsResolver" not found in /home/runner/work/rector-src/rector-src/bin/rector.php:43
Stack trace:
#0 /home/runner/work/rector-src/rector-src/bin/rector(4): require_once()
#1 {main}
  thrown in /home/runner/work/rector-src/rector-src/bin/rector.php on line 43

@samsonasik
Copy link
Member

Composer install in root rector-src seems required

@samsonasik
Copy link
Member

@staabm please rebase in favor of e2e autoload and rector path update #1351

@staabm
Copy link
Contributor Author

staabm commented Nov 30, 2021

@samsonasik thank you for the support <3.

just disabled the actual fix of the PR with 61bac73 to make sure the unit test is working as expected.

without the fix
grafik


when the fix is re-enabled with ac5017a we can see the unit test is working as expected
grafik


final question: how to get a green build now, as we cannot change the view.php .... the bug would not be reproducible otherwise.

@samsonasik
Copy link
Member

It seems there is double ?> even with the fix

@staabm
Copy link
Contributor Author

staabm commented Dec 2, 2021

It seems there is double ?> even with the fix

very good point.

when I run the current rector master across my input view, I already get

+<?php
+
+?>

        <div>
-       <?php echo 1 ?>
-       </div>
+       <?php
+echo 1 ?>
+
+?>
+       </div><?php

so it seems the error of adding 2 times ?> is not caused by the fix but is already existing.

any further hints, where this might come frome?

@samsonasik
Copy link
Member

that's probably internal php-parser bug, probably better to send failing test case there.

@staabm
Copy link
Contributor Author

staabm commented Dec 2, 2021

btw: I just verified that this bug does not occur when working with plain nikic/php-parser:

<?php
use PhpParser\ParserFactory;

require_once 'vendor/autoload.php';

$code = file_get_contents('view.php');

$parser = (new ParserFactory)->create(ParserFactory::PREFER_PHP7);
$stmts = $parser->parse($code);

$prettyPrinter = new PhpParser\PrettyPrinter\Standard;
$newCode = $prettyPrinter->prettyPrintFile($stmts);

var_dump($newCode);

leads to

$ php repro.php
string(36) "
        <div>
        <?php
echo 1;
?>
        </div>"

@staabm
Copy link
Contributor Author

staabm commented Dec 2, 2021

another investigation:

after reducing the BetterStandardPrinter to the very minimum

/**
 * @see \Rector\Core\Tests\PhpParser\Printer\BetterStandardPrinterTest
 */
final class BetterStandardPrinter extends Standard
{
    /**
     * @param mixed[] $options
     */
    public function __construct(
        private IndentCharacterDetector $indentCharacterDetector,
        private DocBlockUpdater $docBlockUpdater,
        array $options = []
    ) {
        parent::__construct($options);
    }

    /**
     * @param Node|Node[]|null $node
     */
    public function print(Node | array | null $node): string
    {
        if ($node === null) {
            $node = [];
        }

        if (! is_array($node)) {
            $node = [$node];
        }

        return $this->prettyPrint($node);
    }


    public function pFileWithoutNamespace(FileWithoutNamespace $fileWithoutNamespace): string
    {
        $content = $this->pStmts($fileWithoutNamespace->stmts, false);

        return ltrim($content);
    }
}

the error is still reproducible on rector-master.
it seems its a AST thing then

@samsonasik
Copy link
Member

samsonasik commented Dec 2, 2021

@staabm I cherry-picked your commits at PR #1368

@staabm
Copy link
Contributor Author

staabm commented Dec 2, 2021

hmm I think I am getting closer.

when using --debug one can see that the FIleWithoutNamespace node is getting nested and nested with every process-phase:

[post rectors] src/view.php
    [post rector] Rector\PostRector\Rector\NodeToReplacePostRector
    [post rector] Rector\PostRector\Rector\NodeAddingPostRector
    [post rector] Rector\PostRector\Rector\PropertyAddingPostRector
    [post rector] Ssch\TYPO3Rector\Rector\PostRector\FullQualifiedNamePostRector
    [post rector] Rector\PostRector\Rector\NodeRemovingPostRector
    [post rector] Rector\PostRector\Rector\ClassRenamingPostRector
    [post rector] Rector\PostRector\Rector\NameImportingPostRector
    [post rector] Rector\PostRector\Rector\UseAddingPostRector
[print] src/view.php
AST
array(
    0: FileWithoutNamespace(
        stmts: array(
            0: Stmt_InlineHTML(
                value:
                <div>

            )
            1: Stmt_Echo(
                exprs: array(
                    0: Scalar_LNumber(
                        value: 1
                    )
                )
            )
            2: Stmt_InlineHTML(
                value:  </div>
            )
        )
    )
)


[refactoring] src/view.php
[post rectors] src/view.php
    [post rector] Rector\PostRector\Rector\NodeToReplacePostRector
    [post rector] Rector\PostRector\Rector\NodeAddingPostRector
    [post rector] Rector\PostRector\Rector\PropertyAddingPostRector
    [post rector] Ssch\TYPO3Rector\Rector\PostRector\FullQualifiedNamePostRector
    [post rector] Rector\PostRector\Rector\NodeRemovingPostRector
    [post rector] Rector\PostRector\Rector\ClassRenamingPostRector
    [post rector] Rector\PostRector\Rector\NameImportingPostRector
    [post rector] Rector\PostRector\Rector\UseAddingPostRector
[print] src/view.php
AST
array(
    0: FileWithoutNamespace(
        stmts: array(
            0: FileWithoutNamespace(
                stmts: array(
                    0: Stmt_InlineHTML(
                        value:
                        <div>

                    )
                    1: Stmt_Echo(
                        exprs: array(
                            0: Scalar_LNumber(
                                value: 1
                            )
                        )
                    )
                    2: Stmt_InlineHTML(
                        value:  </div>
                    )
                )
            )
        )
    )
)


[refactoring] src/view.php
[post rectors] src/view.php
    [post rector] Rector\PostRector\Rector\NodeToReplacePostRector
    [post rector] Rector\PostRector\Rector\NodeAddingPostRector
    [post rector] Rector\PostRector\Rector\PropertyAddingPostRector
    [post rector] Ssch\TYPO3Rector\Rector\PostRector\FullQualifiedNamePostRector
    [post rector] Rector\PostRector\Rector\NodeRemovingPostRector
    [post rector] Rector\PostRector\Rector\ClassRenamingPostRector
    [post rector] Rector\PostRector\Rector\NameImportingPostRector
    [post rector] Rector\PostRector\Rector\UseAddingPostRector
[print] src/view.php
AST
array(
    0: FileWithoutNamespace(
        stmts: array(
            0: FileWithoutNamespace(
                stmts: array(
                    0: FileWithoutNamespace(
                        stmts: array(
                            0: Stmt_InlineHTML(
                                value:
                                <div>

                            )
                            1: Stmt_Echo(
                                exprs: array(
                                    0: Scalar_LNumber(
                                        value: 1
                                    )
                                )
                            )
                            2: Stmt_InlineHTML(
                                value:  </div>
                            )
                        )
                    )
                )
            )
        )
    )
)

samsonasik added a commit that referenced this pull request Dec 2, 2021
@samsonasik
Copy link
Member

Thank you @staabm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants