Skip to content

Add test case for multiline string #610

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 29, 2020
Merged

Add test case for multiline string #610

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ee7e823
Add test case for multiline string
jon-ht May 15, 2020
661acb0
remove unnecessary `\n` adding/removing
jon-ht May 24, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 51 additions & 5 deletions src/Util/YamlSourceManipulator.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -449,8 +449,14 @@ private function changeValueInYaml($value)

$endValuePosition = $this->findEndPositionOfValue($originalVal);

$newYamlValue = $this->convertToYaml($value);
if (!\is_array($originalVal) && \is_array($value)) {
$isMultilineValue = null !== $this->findPositionOfMultilineCharInLine($this->currentPosition);

// In case of multiline, $value is converted as plain string like "Foo\nBar"
// We need to keep it "as is"
$newYamlValue = $isMultilineValue ? rtrim($value, "\n") : $this->convertToYaml($value);
Copy link
Member

@weaverryan weaverryan May 23, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to rtrim() here? I can see that if I remove it the tests fail, but I can't figure out why it's needed.

Copy link
Contributor Author

@jon-ht jon-ht May 23, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing I noticed is, in this test case multiline_string_before_property.test, $value ends with an \n

image

This will cause indentMultilineYamlArray to add an extra line that's not wanted.

Copy link
Member

@weaverryan weaverryan May 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I understand. I think the implementation is what's confusing me. All multiline arrays must end in a line break, by their very nature. If anything, in this spot, (just thinking about the logical flow of the code) I would expect you to be guaranteeing that the $newYamlValue does end in \n, instead of stripping it. After all, if we made sure that $newYamlValue ended in 1 line break... then that should work perfectly with YAML, which also wants the line break :). I'm wondering if we're trimming and adding \n in so many places that we're doing and undoing extra work.

There is the edge-case of not needing the "\n" if the multiline string is at the end of the file. That should be handled (somewhere) with an if statement specifically for it. Something like:

// maybe down before the `$newContents = substr(...)` line
if (false === $this->findNextLineBreak($newPosition) && $isMultilineValue) {
    $newYamlValue = trim($newYamlValue, "\n");
}

Copy link
Contributor Author

@jon-ht jon-ht May 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I won't lie to you, I also think there is a lot of trimming to handle multiline string...

Actually, it seems that re-adding original \n (after $newYamlValue = "\n".$this->indentMultilineYamlArray($newYamlValue);) breaks test case because, as mentionned above, YamlDumper will add an extra line

image

Maybe this bug is related to Yaml component and not YamlSourceManipulator

if ((!\is_array($originalVal) && \is_array($value)) ||
($this->isMultilineString($originalVal) && $this->isMultilineString($value))
) {
// we're converting from a scalar to a (multiline) array
// this means we need to break onto the next line

Expand All @@ -468,7 +474,13 @@ private function changeValueInYaml($value)
++$newPosition;
}

if ($isMultilineValue) {
// strlen(" |")
$newPosition -= 2;
}

$newContents = substr($this->contents, 0, $this->currentPosition)
.($isMultilineValue ? ' |' : '')
.$newYamlValue
/*
* If the next line is a comment, this means we probably had
Expand Down Expand Up @@ -771,16 +783,26 @@ private function findEndPositionOfValue($value, $offset = null)
}

if (is_scalar($value) || null === $value) {
$offset = null === $offset ? $this->currentPosition : $offset;

if (\is_bool($value)) {
// (?i) & (?-i) opens/closes case insensitive match
$pattern = sprintf('(?i)%s(?-i)', $value ? 'true' : 'false');
} elseif (null === $value) {
$pattern = '(~|NULL|null|\n)';
} else {
$pattern = sprintf('\'?"?%s\'?"?', preg_quote($value, '#'));
}
// Multiline value ends with \n.
// If we remove this character, the next property will ne merged with this value
$quotedValue = preg_quote(rtrim($value, "\n"), '#');
$patternValue = $quotedValue;

// Iterates until we find a new line char or we reach end of file
if (null !== $this->findPositionOfMultilineCharInLine($offset)) {
$patternValue = str_replace(["\r\n", "\n"], '\r?\n\s*', $quotedValue);
}

$offset = null === $offset ? $this->currentPosition : $offset;
$pattern = sprintf('\'?"?%s\'?"?', $patternValue);
}

// a value like "foo:" can simply end a file
// this means the value is null
Expand Down Expand Up @@ -1164,7 +1186,31 @@ private function indentMultilineYamlArray(string $yaml): string
// also need to be indented artificially by the same amount
$yaml = str_replace("\n", "\n".$this->getCurrentIndentation(), $yaml);

if ($this->isMultilineString($yaml)) {
// Remove extra indentation in case of blank line in multiline string
$yaml = str_replace("\n".$this->getCurrentIndentation()."\n", "\n\n", $yaml);
}

// now indent this level
return $this->getCurrentIndentation().$yaml;
}

private function findPositionOfMultilineCharInLine(int $position): ?int
{
$cursor = $position;
while (!$this->isCharLineBreak($currentChar = substr($this->contents, $cursor + 1, 1)) && !$this->isEOF($cursor)) {
if ('|' === $currentChar) {
return $cursor;
}

++$cursor;
}

return null;
}

private function isMultilineString($value): bool
{
return \is_string($value) && false !== strpos($value, "\n");
}
}
8 changes: 6 additions & 2 deletions tests/Util/YamlSourceManipulatorTest.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,13 +58,17 @@ private function getYamlDataTests()
foreach ($finder as $file) {
list($source, $changeCode, $expected) = explode('===', $file->getContents());

// Multiline string ends with an \n
$source = rtrim($source, "\n");
$expected = ltrim($expected, "\n");

$data = Yaml::parse($source);
eval($changeCode);

yield $file->getFilename() => [
'source' => rtrim($source, "\n"),
'source' => $source,
'newData' => $data,
'expectedSource' => ltrim($expected, "\n"),
'expectedSource' => $expected,
];
}

Expand Down
16 changes: 16 additions & 0 deletions tests/Util/yaml_fixtures/multiline_string.test
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
nelmio_api_doc:
documentation:
info:
description: |
Multiline string
Second line
===
$data['nelmio_api_doc']['documentation']['info']['description'] = "Multiline string\nSecond line\nThird line";
===
nelmio_api_doc:
documentation:
info:
description: |
Multiline string
Second line
Third line
Copy link
Member

@weaverryan weaverryan May 23, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

beautiful test cases

18 changes: 18 additions & 0 deletions tests/Util/yaml_fixtures/multiline_string_before_property.test
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
nelmio_api_doc:
documentation:
info:
description: |
Multiline string
Second line
foo: bar
===
$data['nelmio_api_doc']['documentation']['info']['description'] = "Multiline string\nSecond line\nThird line\n";
===
nelmio_api_doc:
documentation:
info:
description: |
Multiline string
Second line
Third line
foo: bar
18 changes: 18 additions & 0 deletions tests/Util/yaml_fixtures/multiline_string_remove_line.test
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
nelmio_api_doc:
documentation:
info:
description: |
Multiline string
Second line
Third line
foo: bar
===
$data['nelmio_api_doc']['documentation']['info']['description'] = "Multiline string\nSecond line\n";
===
nelmio_api_doc:
documentation:
info:
description: |
Multiline string
Second line
foo: bar
20 changes: 20 additions & 0 deletions tests/Util/yaml_fixtures/multiline_string_with_blank_line.test
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
nelmio_api_doc:
documentation:
info:
description: |
Multiline string

Second line
foo: bar
===
$data['nelmio_api_doc']['documentation']['info']['description'] = "Multiline string\n\nSecond line\nThird line\n";
===
nelmio_api_doc:
documentation:
info:
description: |
Multiline string

Second line
Third line
foo: bar
15 changes: 15 additions & 0 deletions tests/Util/yaml_fixtures/multiline_string_without_update.test
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
nelmio_api_doc:
documentation:
info:
description: |
Multiline string
Second line
===

===
nelmio_api_doc:
documentation:
info:
description: |
Multiline string
Second line