Skip to content

v0.5.0

Choose a tag to compare

@jlevy jlevy released this 25 Oct 20:50
a9c3dc0

What's Changed

Fixes

  • Per-instance YAML representers and constructors: Each instance that new_yaml()
    returns now registers its representers and constructors on private subclasses of
    ruamel’s classes. Before, every call rewrote the shared ruamel representer class, so
    building one writer changed the suppress_vals, key_sort and stringify_unknown
    policy of every writer built earlier, and on threads a dump could switch policy
    partway through a document. An instance’s policy now stays what it was given, however
    many other instances are built later, on any thread.

Performance

  • C-first YAML reading: When ruamel’s optional C extension (ruamel.yaml.clib) is
    installed, fmf_read, fmf_read_frontmatter, from_yaml_string and read_yaml_file
    parse with libyaml wherever it is known to read a text as pure-Python ruamel does, and
    pure ruamel’s reading defines every result: value, types, key order, sharing and
    errors. The gain grows with document size: a few times faster on typical frontmatter
    blocks and more on large ones, but slower than pure ruamel on a text libyaml rejects,
    which is read twice.
  • Routing guards and fallback: libyaml implements YAML 1.1 and pure ruamel YAML 1.2,
    so texts with a shape on which libyaml was found to read differently go straight to
    pure ruamel: a line starting with %, a tab, NEL, LS, PS or byte order mark, a ...
    line, any tag, anchor or alias, and a few block scalar and flow forms. A text that
    libyaml fails on, or that composes to a document nested more than 100 levels deep or
    sharing a node, is read again by pure ruamel, and that reading is the result. A text
    that may nest past 150 levels skips libyaml, whose composer can overflow a small
    thread stack.
  • Build gate: The guards rest on a differential search over millions of generated,
    mutated and real documents, not on a proof, so the C parser is used only with libyaml
    0.1.7 on ruamel.yaml 0.18 or 0.19, the build and releases that search covered. With
    any other build, the readers use pure ruamel.
  • Pure-mode switch and report: Set FRONTMATTER_FORMAT_PURE_YAML=1 (the name is
    exported as PURE_YAML_ENV) to read with pure ruamel only, for diagnosis; it is read
    on every load. The new yaml_parser_mode() returns "c-first" when the readers will
    use libyaml and "pure" otherwise.

Behavior Changes

These follow from the changes above, but each can turn code that worked by accident into
an error or different output after upgrading.

The registration changes apply to every install:

  • Registrations on one instance stay with it: y.representer.add_representer(T, f)
    or y.register_class(T) on a returned instance no longer reaches later
    to_yaml_string and fmf_write calls, which now raise RepresenterError for T.
    Use add_default_yaml_representer or add_default_yaml_customizer to register for
    every instance.
  • stringify_unknown=True no longer leaks: Once any writer had used it, other rt
    and safe writers stringified unknown objects too. They now raise RepresenterError,
    as configured.
  • Plain ruamel objects are left alone: A YAML(typ=...) object built elsewhere in
    the process no longer picks up this library’s representers. After
    to_yaml_string({"x": 1}), YAML(typ="rt").dump({"a": None, "t": "l1\nl2\n"}) used
    to drop a and write t as a | block; it now writes ruamel’s defaults.
  • Default registrations apply to instances built after them:
    add_default_yaml_representer and add_default_yaml_customizer apply to every
    instance new_yaml builds after the call, and only to those.
  • Customizer constructors stay with the instance: A customizer such as
    lambda y: y.register_class(Foo) no longer makes a plain YAML(typ="safe") elsewhere
    in the process able to load !Foo. typ="full" has no constructor class and is
    unchanged.

The reading and dump changes apply only where ruamel’s C extension is installed.
ruamel.yaml 0.18.x declares it as a dependency on CPython, so many environments have it
without asking for it. Without it, reading and safe dumps are as before.

  • new_yaml() returns pure handles: Every instance it returns parses and emits in
    pure Python for every typ, whether or not the C extension is installed, so direct
    new_yaml().load() calls use pure ruamel. The library’s own readers keep libyaml’s
    speed through C-first reading. rt instances are unchanged, since ruamel never used C
    for round-trip.
  • Safe dumps write pure ruamel’s bytes: new_yaml(typ="safe") and
    to_yaml_string(..., typ="safe") no longer escape astral characters (an emoji was
    written as "\U0001F600") or break long lines as libyaml’s emitter did, and lose that
    emitter’s speed. rt dumps, the default for to_yaml_string, write_yaml_file and
    fmf_write, are unchanged.
  • Readers load what pure ruamel loads: Texts libyaml refused now load, such as a URL
    or a time in a flow sequence (sources: [https://example.com/a],
    t: [10:30, 11:00]), which made a Metaproc process spec fail to load with
    ScannerError ... found unexpected ':'. %YAML 1.1 documents resolve as YAML 1.1.
    Texts libyaml accepted and pure ruamel refuses, such as a tab after a value,
    ["a":1] or [[]:], now raise with pure ruamel’s message.
  • Deep documents: A document nested more than 100 levels deep is read by pure
    ruamel, which raises RecursionError at about 490 levels under the default recursion
    limit. libyaml alone read such texts and could crash the process, at about 30,000
    levels on an 8 MB stack or about 400 on a 128 KB thread stack.
  • Two visible differences that change no value remain for texts libyaml reads: a
    warning issued during the read carries libyaml’s source marks, and a raising tag
    constructor registered through a customizer runs on the libyaml attempt and again on
    the pure re-read.

Packaging

  • No runtime dependency changes: The package still depends only on
    ruamel-yaml>=0.18.10. The C extension is not a dependency and no extra is added; to
    use C-first reading, install ruamel.yaml.clib in the same environment. Only the
    development group adds it, so the test suite reads C-first by default, and CI also
    runs the suite without it.

Full commit history: v0.4.0...v0.5.0