Making the Lexer initialize itself lazily, to avoid loading the extension set early

[weaverryan]

Hi!

Over on Symfony UX, we are pushing Twig in some crazy, "unconventional" ways by overriding the lexer to add a new HTML-like syntax - e.g. <twig:Alert type="success"> - https://github.com/symfony/ux/blob/2.x/src/TwigComponent/src/Twig/ComponentLexer.php

Normally, the internal Lexer is not instantiated until it's needed - i.e. when a template is rendered. However, when overriding the lexer, our ComponentLexer needs to be instantiated early and set onto the Environment. And since it needs to extend Lexer, Lexer::__construct() is called much earlier than normal. One line in the constructor -

Twig/src/Lexer.php

Line 108 in f5ee1b6

'operator' => $this->getOperatorRegex(),

- causes the "extension set" to be loaded inside Environment. Some applications rely on being able to set extra Twig extensions after Environment is created, but before a template is rendered and our custom lexer currently breaks that.

So, the suggestion is to make the work inside __construct() done later. Everything is private, so I believe this is safe.

Cheers!

[weaverryan]

fabbot failure is on an existing line - so leaving that alone

[fabpot]

Symfony UX supports Twig 2.x, so maybe we should fix this on 2.x instead?

[javiereguiluz]

@weaverryan we hit this bug this morning when upgrading an app (which uses UX) to Symfony 6.3. We call $twig->addGlobal() in a listener and we saw the error message "Unable to add global "foobar" as the runtime or the extensions have already been initialized.".

It was really tricky to debug this ... but Nicolas point me to here 🙌 So, thanks for taking care of this.

[fabpot]

Thank you @weaverryan.

[fabpot]

@weaverryan New Twig versions released.