Validating static site generator data file: pyyaml errors out, while Ruby's YAML doesn't

[wkoszek]

Hello,

Originally I've filled this bug in here: adrienverge/yamllint#13

@adrienverge narrowed down a simplified case and we couldn't figure out whose fault it is and why yamllint is complaining. Basic example input file:

cat > zsample.yaml

---
This is paragraph one.
# comment
This is paragraph two.

The result is:

python -c 'import sys, yaml; yaml.dump(yaml.load(sys.stdin), sys.stdout)' < zsample.yaml
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/yaml/__init__.py", line 71, in load
    return loader.get_single_data()
  File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 37, in get_single_data
    node = self.get_single_node()
  File "/usr/local/lib/python2.7/site-packages/yaml/composer.py", line 39, in get_single_node
    if not self.check_event(StreamEndEvent):
  File "/usr/local/lib/python2.7/site-packages/yaml/parser.py", line 98, in check_event
    self.current_event = self.state()
  File "/usr/local/lib/python2.7/site-packages/yaml/parser.py", line 174, in parse_document_start
    self.peek_token().start_mark)
yaml.parser.ParserError: expected '<document start>', but found '<scalar>'
  in "<stdin>", line 4, column 1

My little Ruby utility however doesn't complain:

wget https://raw.githubusercontent.com/wkoszek/me/master/scripts/yamllint.rb
chmod 755 yamllint.rb
./yamllint.rb zsample.yaml

I'm trying to understand whose issue is that, as I intended to use yamllint for my purposes.

[sigmavirus24]

By default, Ruby's YAML library will actually use YAML 1.2. PyYAML only uses 1.1 and does not support 1.2. Everything in 1.1 can be parsed by 1.2 but not vice versa. This could be a 1.2 change that 1.1 cannot handle (and therefore PyYAML cannot handle it).

[adrienverge]

Repeat from adrienverge/yamllint#13 (comment):

Sorry @wkoszek, after re-reading the YAML specifications (both 1.1 and 1.2), I realize this is not a PyYAML bug, but a problem with your YAML document.

As stated in paragraph 3.2.3.3 of the spec:

Comments must not appear inside scalars, but may be interleaved with such scalars inside collections.

This explains why the following document, that only contains one big string scalar, is parsed OK:

---
This is paragraph one.

This is paragraph two.

but the following doesn't, because the string scalar has a comment in the middle:

---
This is paragraph one.
# comment
This is paragraph two.

[perlpunk]

Indeed, the first example is invalid, and I just tested with ruby, which also detects it as invalid.
Closing.