Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Make binary data output possible #51387
If for example a pillar contains binary data it's possible that it won't
Replace non-printable data during output with a message instead.
What does this PR do?
We noticed that when we had binary data in our pillars, certain functions could not print the output. Also it wouldn't look very good printing big binary files that you have in your pillars. We haven't found any other way of doing this differently so here is a PR that solves this problem. Instead of trying to print the binary data, it will print a message instead.
When looking at the code you would think that in this passage it would be a string, because we are checking against "six.string_types". But this variable is set from type "Str" to a Tuple of the form: "(<class 'str'>, <class 'bytes'>)". I was not able to follow the runtime code fully, but it looks like the six.string_types is changed somewhere in the salt cli code. I'm very new to Saltstack so maybe all this is known things and intended behaviour.
What issues does this PR fix or reference?
None that I've seen.
Pillars containing binary data would make the pillar.items or other Salt output call to fail.
Output with binary data would be replaced with a message.
Commits signed with GPG?
If for example a pillar contains binary data it's possible that it won't be possible to print it when doing e.g pillar.items: File "/usr/lib/python3/dist-packages/salt/output/__init__.py", line 232, in strip_esc_sequence return txt.replace('\033', '?') TypeError: a bytes-like object is required, not 'str' Replace non-printable data during output with a message instead.
From my research it looks like Saltstack is not supposed to support printing binary data, correct? And if you need to distribute binary files via pillars, e.g file_tree or custom ext_pillar, you need to base64 encode the data before putting it in the pillar. Then you can use file.decode, set encoding_type: base64, and your pillar will be decoded and saved as a binary file again.
This commit to modules/hashutil.py is needed for this to work though: ed25416
This is totally fine in our case. We have a custom Vault ext_pillar and when I set it to encode the file in base64 encoding, file.decode could correctly save the file as binary again.