FEATURE: augment constructed (loaded) document with location marks

[kshpytsya]

strictyaml does schema based validation and reports exact text positions of erroneous input. However in addition to schema validation, the application may perform other checks which are impossible to express in terms of schema. It would be nice to be able to report to user exact text position of offending data.
I was able to come up with a quick-and-dirty proof of concept (tested only on python 3.5) that uses type(name, bases, dict) to construct subclasses of built-in types (dict, list, int, str) that contain start_mark and end_mark of nodes producing the corresponding data. I create those sub-classed instances in construct_object.
I can send you the above-mentioned prototype should you find it useful.

[crdoconnor]

Sure, I'd love to see it.

Custom scalar parameters were something I was going to work on soon - is that kind of what you were talking about?

[kshpytsya]

I am not sure what you mean by "custom scalar parameter", just see my code here.

[colmoconnor-loveholidays]

Ok, I think I understand. Yes, I very much want something like this.

Ideally, stricyaml.load(str) would return an object rather than a dict/list. You could treat this object purely as a dict or a list or a scalar (by overriding the relevant magic methods), but you could also use it to look up the line number of a scalar (for instance) as well as being able to use it to trivially load, change a value and roundtrip the YAML with comments included.

[crdoconnor]

Hi @kshpytsya

This feature is now more or less implemented on the master branch.

The way it functions is documented in this test: https://github.com/crdoconnor/strictyaml/blob/master/tests/whatline.test

I haven't fully decided on the API so I haven't released it on to pypi yet. Any feedback would be very welcome.

[kshpytsya]

As of this commit the functionality seems to be non-obvious or inconsistent.

> s = '\n\na:\n  -\n    A: 1\n  -\n    B: 2\n'
> v = strictyaml.load(s)
> [i.start_line for i in (v, v["a"], v["a"][0], v["a"][1])]
[1, 1, 3, 4]
> print(v.lines())

a:
- A: '1'
- B: '2'
>

So, the start_line values does not seem to correspond to line numbers in the input string but rather in some transformed string.

Apparently missing is the way to access position of mapping keys:

> [i[0].start_line for i in v.items()]
AttributeError: 'str' object has no attribute 'start_line'

Also, I would like to be able to get info on columns and not just line numbers. Moreover, please take a look at my feature request for ruamel.yaml.

To reiterate: I want to be able to obtain, for any YAML element (mapping, sequence or scalar), a precise position in the input string (i.e. line and column) that can be used e.g. to position cursor in an editor. For string scalars I would like to be able to obtain a precise position corresponding to any character (identified by its index) of the string, and not just the start and end positions. Please see the above-mentioned feature request for rationale.

The author of ruamel.yaml appears to be unconvinced that the feature belongs to the library despite my explanations that doing so outside the library would require considerable piece of library code to be copied.

Of course, I could implement all this functionality I desire (see also issue 16) myself, but where should it go? A fork of strictyaml? Yet another something-yaml library having strictyaml as a dependency? Definitely not as a part of the final project using this functionality, as there may be more than one such project.

[crdoconnor]

Ah, I think you found a bug with ruamel.yaml::

ruamelyaml.dump(ruamelyaml.load(s, Loader=ruamelyaml.RoundTripLoader), Dumper=ruamelyaml.RoundTripDumper)

a:
- A: 1
- B: 2

[crdoconnor]

To reiterate: I want to be able to obtain, for any YAML element (mapping, sequence or scalar), a precise position in the input string (i.e. line and column) that can be used e.g. to position cursor in an editor.

Fair enough. This only goes half way in that case.

For string scalars I would like to be able to obtain a precise position corresponding to any character (identified by its index) of the string, and not just the start and end positions.

Ok. In that case if you had start_line and start_column, deriving the precise character position outside of strictyaml would be straightforward.

Of course, I could implement all this functionality I desire (see also issue 16) myself, but where should it go? A fork of strictyaml?

I'd be very happy to merge in a PR that did start_column, end_column, start_position (character position in the YAML string) and end_position provided it came with a test.

From that point, I think your specific use case (getting the exact position of o in x: hello) should be pretty trivial (I think that part should probably live outside of strictyaml).

[crdoconnor]

Apparently missing is the way to access position of mapping keys

Noted. I will fix this. Thanks for pointing it out.

[crdoconnor]

Access position of mapping keys is now in 4b38cc6