Example YAML appears to be invalid

[kmcquade]

First of all, thanks for such a cool tool! This is awesome.

Description

I was following the tutorial and tinkering a bit and realized that the example YAML output is actually invalid YAML. See the screenshot from the README of the repo below.

To Reproduce

It's the main README - but for the sake of conforming to the issue template 🙂 here's the command from the root of the repo:

oasdiff -base data/openapi-test1.yaml -revision data/openapi-test2.yaml

Expected behavior
I would expect the YAML to be valid. I wasn't sure if this was just a diff format thing - but I would assume that we'd want a way to generate valid YAML.

I'm excited to get this working! 😄 Cool stuff!

[reuvenharrison]

Hi @kmcquade.
This YAML is actually valid but not very common. You can try pasting it into a YAML validator to confirm this.

The YAML spec refers to this as a "complex map key" which we leverage here to denote an Endpoint which is a combination of a Path and a Method.

The YAML spec explains it but not very well unfortunately:
A question mark and space (“? ”) indicate a complex mapping key.

You can also see this related thread.

I hope this clarifies it.
Reuven

[kmcquade]

Oh wow I didn't know that. I guess you do learn something new every day.

I'm trying to load that YAML into Python but I don't think the Python YAML module supports this. Any suggestions? Is there a way for oasdiff to toggle that feature by chance?

[reuvenharrison]

This seems to work:

from pathlib import Path
import ruamel.yaml

file_name = Path('data/test.yaml')

yaml = ruamel.yaml.YAML()
data = yaml.load(file_name)
yaml.dump(data['endpoints']['modified'] , sys.stdout)

Installation:
pip install ruamel.yaml

[kmcquade]

Sorry, I should have been more specific. I'm trying to load that YAML into a python dictionary. That code snippet above loads data as a CommentedMap and the yaml.dump method prints the same YAML string.

Any ideas on reading it as a dictionary?

[reuvenharrison]

How about this?

from yaml import load
import io
import json

try:
    from yaml import CLoader as Loader
except ImportError:
    from yaml import Loader

stream=io.open('data/test.yaml', mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True)
data = load(stream, Loader=Loader)
print(json.dumps(data, indent=4, sort_keys=True))

Edit:
I noticed that this works on some versions of PyYAML but not on others but I'm really not sure how to determine the one that is working properly.

[reuvenharrison]

I didn't want to leave this mystery unsolved so I did some more digging.
Apparently, it isn't the PyYAML version which determines success or failure but it's the code snippet below that does the magic. I got it from this blog.

from yaml import load
import io
import json

try:
    from yaml import CLoader as Loader
except ImportError:
    from yaml import Loader

class Pairs(list):
    def __repr__(self):
        return f"{type(self).__name__}({super().__repr__()})"

def construct_mapping(self, node):
    value = self.construct_pairs(node)
    try:
        return dict(value)
    except TypeError:
        return Pairs(value)

Loader.construct_mapping = construct_mapping
Loader.add_constructor('tag:yaml.org,2002:map', Loader.construct_mapping)

stream = io.open('data/test.yaml', mode='r', buffering=-1,
                 encoding=None, errors=None, newline=None, closefd=True)
data = load(stream, Loader=Loader)
print(json.dumps(data, indent=4, sort_keys=True))

[kmcquade]

Yes!!! Thank you so much. I never would have figured all that out on my own. You're awesome. I really appreciate it.