Incompatible changes in version 5.1 #265
Description
The PyYAML release 5.1 adds some incompatible changes.
If you are getting problems with the new version, that can have several reasons, which are listed here.
Also note that there are things that still need to be fixed. Please wait for release 5.2 and see details at the bottom
The full list of merged PRs can be found here: https://github.com/yaml/pyyaml/blob/master/announcement.msg#L30
The two incompatible changes are:
- Deprecate yaml.load and add FullLoader and UnsafeLoader classes #257 -- Deprecate yaml.load and add FullLoader and UnsafeLoader classes
- Make default_flow_style=False #256 -- Make default_flow_style=False
load: Deprecate yaml.load and add FullLoader and UnsafeLoader classes
For the full story read #257 and #207
YAML is a serialization language, and the yaml.load method allowed to execute arbitrary function calls and import modules, which made it unsafe when loading untrusted YAML.
With this release, it is deprecated to use yaml.load without a Loader argument and will issue a warning:
data = yaml.load("some yaml") # deprecated, warns. It uses FullLoader behind the scenes
data = yaml.load("some yaml", Loader=yaml.SafeLoader) # ok
use safe_load/SafeLoader
yaml.safe_load() always existed for that purpose, and most of you which are getting a warning now should be able to just replace yaml.load with yaml.safe_load:
yaml.safe_load(...)
yaml.load(..., Loader=yaml.SafeLoader)
use full_load/FullLoader
If you are actually using the serialization features (e.g. !!python/object/... tags) and want to load python objects, you should be fine with using the FullLoader:
yaml.load(..., Loader=yaml.FullLoader)
yaml.full_load(...)
FullLoader is pretty safe, but not completely. For an attack, you would need to
- find a module that does something destructive/exploitable in its constructor
- rely on the fact that this module is installed and already loaded
use unsafe_load/UnsafeLoader
There are two things that the FullLoader won't do, though:
- Automatically import the modules for the objects that you want to load
- Execute arbitrary function/method calls
If you need this then you must:
yaml.load(..., Loader=yaml.UnsafeLoader)
yaml.unsafe_load(...)
# or be backwards compatible
yaml.load(..., Loader=yaml.Loader)
yaml.Loader will behave just like yaml.UnsafeLoader
dump: Make default_flow_style=False
YAML has two styles for collections: block and flow style.
The flow style looks a bit similar to JSON or to Python itself:
block key: { flow: mapping, more: items }
To save some space, the default for dumping was to use flow style on lists and dicts at the lowest level of a data structure, that means, that have no child nodes themselves:
yaml.dump(data, default_flow_style=None)
However, many people were using default_flow_style=False to avoid this, so the default didn't seem to be the best.
We decided that we change the default in a major release:
yaml.dump(data, default_flow_style=False)
You can still get the old behaviour by using None instead of False.
Other important changes
- Allow colon in a plain scalar in a flow context #45 -- Allow colon in a plain scalar in a flow context
- Adding support to Unicode characters over codepoint 0xffff #63 -- Adding support to Unicode characters over codepoint 0xffff
- Allow to turn off sorting keys in Dumper #143 -- Allow to turn off sorting keys in Dumper
- Support escaped slash in double quotes "\/" #158 -- Support escaped slash in double quotes "/"
Breakages to be fixed in 5.2
When the default loader yaml.Loader for yaml.load() was changed to FullLoader, it was forgotten that there are two other places where we need to change the default:
- in
yaml.add_constructor - in the class
YAMLObjectwhich you can inherit from so thatfrom_yamlandto_yamlare called
Also the logic for detecting special characters was broken, which results in strings with tabs (for example) to be dumped in plain style on python 2.7 on systems with sys.maxunicode <= 0xffff.