[Yaml] Fix Yaml Parser with quote end in a new line #48022

maxbeckers · 2022-10-28T12:45:21Z

Q	A
Branch?	5.4
Bug fix?	yes
New feature?	no
Deprecations?	no
Tickets	Fix #33082
License	MIT
Doc PR	N/A

This is a fix for issue #33082.

The bug described in the ticket breaks on a ending quote in a new line:

foo:
  bar: 'baz

'
  baz: 'Lorem'

Before the fix:
Symfony\Component\Yaml\Exception\ParseException: Malformed inline YAML string: 'baz at line 4.

There was already a PR #33119, which was closed because of problems.

carsonbot · 2022-10-29T16:36:49Z

Hey!

I think @mamazu has recently worked with this code. Maybe they can help review this?

Cheers!

Carsonbot

src/Symfony/Component/Yaml/Tests/YamlTest.php

src/Symfony/Component/Yaml/Parser.php

nicolas-grekas

LGTM but I'd like to ping @xabbuh also :)

src/Symfony/Component/Yaml/Tests/YamlTest.php

fabpot

I've tried many online parser, written in different languages, which all behave differently.
I would not change the current behavior if the spec is unclear on this, as this would be a BC break.

So, that's a 👎 from me for this change.

fabpot · 2022-12-16T06:49:26Z

src/Symfony/Component/Yaml/Tests/YamlTest.php

+foo:
+  bar: 'baz
+
+'


This is not valid YAML AFAIU. The quote must be indented.

Then we should close this PR and the issue with "Won't fix", because this is exactly the case, what is requested to be fixed!

If I am not mistaken, this is indeed valid YAML.

Hi @xabbuh and @fabpot,

I have taken some time to read up on this YAML topic. In the internet are two kinds of YAML parser for this topic. Either the content must be indented even if it is in quotes or everything in quotes can be indented free, because the part in quotes is simply considered as a text block.

What I want to say is that the PR should not be merged as it is and we should decide here how to proceed.

Solution A: everything should be how it is and we only support indented quotes, seems to be how it's described in the spec https://yaml.org/spec/1.2.2/#8111-block-indentation-indicator.

foo: bar: 'baz '

Solution B: be a bit more generous and allow what requested in the issue and how a lot of online parsers/validators see the quoted part as a string and allow stuff like that:

foo: bar: 'baz1 baz2 baz3 '

Based on that I'd prefer solution A and keep it how it is as @fabpot mentioned. What do you think?

nicolas-grekas

I think we should merge this since this is YAML compliant, isn't it?

But there's still an issue since this fails:

--- a/src/Symfony/Component/Yaml/Tests/YamlTest.php
+++ b/src/Symfony/Component/Yaml/Tests/YamlTest.php
@@ -30,6 +30,7 @@ class YamlTest extends TestCase
         $yaml = <<<YAML
 foo:
   bar: 'baz
+biz
 
 '
   baz: 'Lorem
@@ -38,7 +39,7 @@ foo:
   foobar: 'foobar'
 YAML;
 
-        $this->assertSame(['foo' => ['bar' => "baz\n", 'baz' => "Lorem\nipsum", 'foobar' => 'foobar']], Yaml::parse($yaml));
+        $this->assertSame(['foo' => ['bar' => "baz biz\n", 'baz' => "Lorem\nipsum", 'foobar' => 'foobar']], Yaml::parse($yaml));
     }

maxbeckers · 2023-07-07T07:45:56Z

@nicolas-grekas you're right ... this is solution b, I described here ... this is not possible with the current implementation and this would be a bigger change of the yaml component. Because for that the component needs to understand the context (this is part of a quoted multiline string) ... for now it just reads the yaml line by line at this point of the yaml parser ... I'll have a look on that and see whats possible.

maxbeckers · 2023-07-11T05:54:14Z

Added support for a colon in the unquoted key

xabbuh · 2023-11-07T14:28:35Z

src/Symfony/Component/Yaml/Tests/YamlTest.php

+YAML;
+
+        $this->assertSame(['foo' => [
+            'bar' => "baz biz\n",


Suggested change

'bar' => "baz biz\n",

'bar' => 'baz biz ',

Is this the behavior you expect @xabbuh? Currently with this input we get this result without your suggestion.

Yes, as far as I know that should be the outcome.

In the YAML spec I didn't find sth. for this case so it might be a very special case. This is the ChatGPT answer for that:

An empty line is treated as a line break (\n)

This is also the way it works in different online interpreters which are supporting this (like this one: http://www.yaml-online-parser.appspot.com/.
This one also interpretes

baz: 'Lorem ipsum'

As {"baz": "Lorem\nipsum"}.
While other interpreters like https://codebeautify.org/yaml-parser-online interpret this as {"baz": "Lorem ipsum"}.

So it seems to be not really clear how to deal with this and we should decide how to deal with it. My suggestion would be "keep it simple for this very special case" and keep it how it is atm in this PR with the \n. What do you think?

xabbuh · 2023-11-07T14:28:51Z

src/Symfony/Component/Yaml/Tests/YamlTest.php

+
+        $this->assertSame(['foo' => [
+            'bar' => "baz biz\n",
+            'baz' => "Lorem\nipsum",


Suggested change

'baz' => "Lorem\nipsum",

'baz' => 'Lorem ipsum',

xabbuh · 2023-11-07T14:31:42Z

src/Symfony/Component/Yaml/Tests/YamlTest.php

+            'baz' => "Lorem\nipsum",
+            'error' => "Une erreur s'est produite.",
+            'trialMode' => "période d'essai",
+            'double_line' => "Les utilisateurs sélectionnés n'ont pas d'email.\n",


Suggested change

'double_line' => "Les utilisateurs sélectionnés n'ont pas d'email.\n",

'double_line' => "Les utilisateurs sélectionnés n'ont pas d'email. ",

xabbuh · 2023-11-07T14:33:54Z

src/Symfony/Component/Yaml/Parser.php

+            }
+        }
+
+        if (!str_starts_with($value, "'")) {


Suggested change

if (!str_starts_with($value, "'")) {

if (0 !== strpos($value, "'")) {

xabbuh · 2023-11-07T14:36:37Z

src/Symfony/Component/Yaml/Parser.php

+                continue;
+            } elseif ($isInMultiLineQuote) {
+                $data[] = $this->currentLine;
+                if (str_ends_with(rtrim($this->currentLine, ' '), "'")) {


Suggested change

if (str_ends_with(rtrim($this->currentLine, ' '), "'")) {

if ("'" === (rtrim($this->currentLine)[-1] ?? '')) {

fabpot

Merging it as a new feature

fabpot · 2024-02-03T17:58:29Z

Thank you @maxbeckers.

…n a new line (maxbeckers)" This reverts commit 40a2cfb, reversing changes made to d2d36b5.

xabbuh · 2024-02-03T18:47:54Z

I suggest to revert this for now as the changes to the parser break existing tests in the SecurityBundle (see #53747).

…te end in a newline (maxbeckers)" (xabbuh) This PR was merged into the 7.1 branch. Discussion ---------- [Yaml] Revert "feature #48022 Fix Yaml Parser with quote end in a newline (maxbeckers)" | Q | A | ------------- | --- | Branch? | 7.1 | Bug fix? | yes | New feature? | no | Deprecations? | no | Issues | | License | MIT Unfortunately, the features as implemented breaks tests in other bundles (see https://github.com/symfony/symfony/actions/runs/7768721075/job/21186968515?pr=53745#step:8:3092). Commits ------- d709af0 Revert "feature #48022 [Yaml] Fix Yaml Parser with quote end in a new line (maxbeckers)"

…n a new line (maxbeckers)" This reverts commit 40a2cfb, reversing changes made to d2d36b5.

maxbeckers requested a review from xabbuh as a code owner October 28, 2022 12:45

carsonbot added Status: Needs Review Bug Yaml labels Oct 28, 2022

carsonbot added this to the 4.4 milestone Oct 28, 2022

xabbuh reviewed Oct 31, 2022

View reviewed changes

src/Symfony/Component/Yaml/Tests/YamlTest.php Outdated Show resolved Hide resolved

src/Symfony/Component/Yaml/Parser.php Outdated Show resolved Hide resolved

maxbeckers force-pushed the patch-33082 branch 2 times, most recently from 67e40af to b3cc966 Compare November 1, 2022 06:22

fabpot modified the milestones: 4.4, 5.4 Nov 23, 2022

nicolas-grekas changed the base branch from 4.4 to 5.4 December 13, 2022 10:51

nicolas-grekas force-pushed the patch-33082 branch from b3cc966 to 50250dc Compare December 13, 2022 10:51

nicolas-grekas approved these changes Dec 13, 2022

View reviewed changes

carsonbot added Status: Reviewed and removed Status: Needs Review labels Dec 13, 2022

javiereguiluz reviewed Dec 15, 2022

View reviewed changes

src/Symfony/Component/Yaml/Tests/YamlTest.php Outdated Show resolved Hide resolved

fabpot requested changes Dec 16, 2022

View reviewed changes

carsonbot added Status: Needs Work and removed Status: Reviewed labels Dec 16, 2022

xabbuh approved these changes Dec 17, 2022

View reviewed changes

carsonbot added Status: Reviewed and removed Status: Needs Work labels Dec 17, 2022

nicolas-grekas requested changes Jul 7, 2023

View reviewed changes

carsonbot added Status: Needs Work and removed Status: Reviewed labels Jul 7, 2023

maxbeckers force-pushed the patch-33082 branch 3 times, most recently from c315183 to 1bc16e5 Compare July 7, 2023 10:25

maxbeckers force-pushed the patch-33082 branch 3 times, most recently from 864ccf2 to 0c8ef52 Compare July 11, 2023 05:53

nicolas-grekas approved these changes Jul 13, 2023

View reviewed changes

carsonbot added Status: Reviewed and removed Status: Needs Work labels Jul 13, 2023

xabbuh reviewed Nov 7, 2023

View reviewed changes

xabbuh requested changes Nov 7, 2023

View reviewed changes

carsonbot added Status: Needs Work and removed Status: Reviewed labels Nov 7, 2023

maxbeckers force-pushed the patch-33082 branch from 0c8ef52 to dd0070c Compare November 8, 2023 06:59

fabpot approved these changes Feb 3, 2024

View reviewed changes

carsonbot added Status: Reviewed and removed Status: Needs Work labels Feb 3, 2024

fabpot modified the milestones: 5.4, 7.1 Feb 3, 2024

[Yaml] Fix Yaml Parser with quote end in a new line

21cec3f

fabpot changed the base branch from 5.4 to 7.1 February 3, 2024 17:57

fabpot force-pushed the patch-33082 branch from dd0070c to 21cec3f Compare February 3, 2024 17:57

fabpot mentioned this pull request Feb 3, 2024

[Yaml] Fix Yaml Parser with quote end in a new line symfony/symfony-docs#19498

Closed

fabpot merged commit 40a2cfb into symfony:7.1 Feb 3, 2024
3 of 9 checks passed

xabbuh added a commit to xabbuh/symfony that referenced this pull request Feb 3, 2024

Revert "feature symfony#48022 [Yaml] Fix Yaml Parser with quote end i…

d709af0

…n a new line (maxbeckers)" This reverts commit 40a2cfb, reversing changes made to d2d36b5.

symfonyaml pushed a commit to symfonyaml/symfony that referenced this pull request Feb 4, 2024

Revert "feature symfony#48022 [Yaml] Fix Yaml Parser with quote end i…

c48c033

…n a new line (maxbeckers)" This reverts commit 40a2cfb, reversing changes made to d2d36b5.

fabpot mentioned this pull request May 2, 2024

Release v7.1.0-BETA1 #54820

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Yaml] Fix Yaml Parser with quote end in a new line #48022

[Yaml] Fix Yaml Parser with quote end in a new line #48022

maxbeckers commented Oct 28, 2022 •

edited by nicolas-grekas

Loading

carsonbot commented Oct 29, 2022

nicolas-grekas left a comment

fabpot left a comment

fabpot Dec 16, 2022

maxbeckers Dec 16, 2022

xabbuh Dec 17, 2022

maxbeckers Feb 10, 2023 •

edited

Loading

nicolas-grekas left a comment

maxbeckers commented Jul 7, 2023

maxbeckers commented Jul 11, 2023

xabbuh Nov 7, 2023

maxbeckers Nov 7, 2023

xabbuh Nov 7, 2023

maxbeckers Nov 8, 2023

xabbuh Nov 7, 2023

xabbuh Nov 7, 2023

xabbuh Nov 7, 2023

xabbuh Nov 7, 2023

fabpot left a comment

fabpot commented Feb 3, 2024

xabbuh commented Feb 3, 2024

	'double_line' => "Les utilisateurs sélectionnés n'ont pas d'email.\n",
	'double_line' => "Les utilisateurs sélectionnés n'ont pas d'email. ",

	if (!str_starts_with($value, "'")) {
	if (0 !== strpos($value, "'")) {

	if (str_ends_with(rtrim($this->currentLine, ' '), "'")) {
	if ("'" === (rtrim($this->currentLine)[-1] ?? '')) {

[Yaml] Fix Yaml Parser with quote end in a new line #48022

[Yaml] Fix Yaml Parser with quote end in a new line #48022

Conversation

maxbeckers commented Oct 28, 2022 • edited by nicolas-grekas Loading

carsonbot commented Oct 29, 2022

nicolas-grekas left a comment

Choose a reason for hiding this comment

fabpot left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

maxbeckers Feb 10, 2023 • edited Loading

Choose a reason for hiding this comment

nicolas-grekas left a comment

Choose a reason for hiding this comment

maxbeckers commented Jul 7, 2023

maxbeckers commented Jul 11, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fabpot left a comment

Choose a reason for hiding this comment

fabpot commented Feb 3, 2024

xabbuh commented Feb 3, 2024

maxbeckers commented Oct 28, 2022 •

edited by nicolas-grekas

Loading

maxbeckers Feb 10, 2023 •

edited

Loading