Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

yaml_dquote adds whitespace when string > 80 chars long #34181

Closed
pykler opened this issue Jun 21, 2016 · 3 comments
Closed

yaml_dquote adds whitespace when string > 80 chars long #34181

pykler opened this issue Jun 21, 2016 · 3 comments
Labels
Bug broken, incorrect, or confusing behavior Core relates to code central or existential to Salt P4 Priority 4 severity-medium 3rd level, incorrect or bad functionality, confusing and lacks a work around
Milestone

Comments

@pykler
Copy link
Contributor

pykler commented Jun 21, 2016

Description of Issue/Question

yaml_dquote filter uses the basic yaml Emitter which adds a Emitter.best_line_break when Emitter.column > Emitter.best_width (+ some additional yaml criteria). This works for yaml but doesnt work when using these strings inside sls files.

Setup

{% set v='"'*42 %}
/tmp/t.sh:
    file.managed:
        - contents: |
            # Environment variables
            export VAR={{ v | yaml_dquote }}            

Steps to Reproduce Issue

Execute the above sls, you will notice a newline is inserted before the last double quote.

Versions Report

Salt Version:
           Salt: 2016.3.1

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.7
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
         Python: 2.7.6 (default, Jun 22 2015, 17:58:13)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.3
            ZMQ: 4.1.2

System Versions:
           dist: Ubuntu 14.04 trusty
        machine: x86_64
        release: 3.13.0-48-generic
         system: Linux
        version: Ubuntu 14.04 trusty
@Ch3LL
Copy link
Contributor

Ch3LL commented Jun 22, 2016

@pykler looks like i'm able to replicate this behavior:

➜  salt sudo salt-call --local state.sls issues.34181
[CRITICAL] Rendering SLS 'base:issues.34181' failed: could not found expected ':'; line 8

---
[...]
    file.managed:
        - contents: |
            # Environment variables
            export VAR="\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\
\"";
    <======================
---
local:
    Data failed to compile:
----------
    Rendering SLS 'base:issues.34181' failed: could not found expected ':'; line 8

---
[...]
    file.managed:
        - contents: |
            # Environment variables
            export VAR="\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\
\"";
    <======================
---

@Ch3LL Ch3LL added Bug broken, incorrect, or confusing behavior P4 Priority 4 severity-medium 3rd level, incorrect or bad functionality, confusing and lacks a work around Core relates to code central or existential to Salt labels Jun 22, 2016
@Ch3LL Ch3LL added this to the Approved milestone Jun 22, 2016
@pykler
Copy link
Contributor Author

pykler commented Jun 22, 2016

One solution is to initialize the Emitter with maxint as the line width

yemitter = yaml.emitter.Emitter(ostream)

http://pyyaml.org/browser/pyyaml/trunk/lib/yaml/emitter.py#L38

yemitter = yaml.emitter.Emitter(ostream, width=six.MAXSIZE)

@pykler
Copy link
Contributor Author

pykler commented Jun 22, 2016

For anyone interested in a quick and dirty solution, in the jinja files I replaced newlines as follows. Note that this only works if your original string has no newlines, if you actually need the original newlines only your probably SOL until this is resolved. Though, I do not think there is a usecase like that where you would want some of the newlines.

{{ '"' * 42 | yaml_dquote | replace('\\\n','') }}

Notice the extra escaped backslash.

cachedout pushed a commit that referenced this issue Jun 23, 2016
Fixes #34181 no more newlines in long yaml encodes
gitebra pushed a commit to gitebra/salt that referenced this issue Jun 23, 2016
* upstream/develop: (37 commits)
  Use ipaddress to validate ipv4/ipv6 (saltstack#33956)
  Updated winrepo_test (saltstack#34227)
  Whitespace fix for saltstack#34235 (saltstack#34250)
  message_format was not set in the correct function (saltstack#34235)
  Typo in dockerio doc (saltstack#34244)
  Fixup test failure
  Pass through update_holds to pkg.install
  fix regression from saltstack#33681 which causes pulling a list of s3 objects via s3.query to fail (saltstack#34208)
  fix regression in s3.query from saltstack#33599 that causes pulling a file list from s3 to fail (saltstack#34207)
  utils/parsers.py: disable minion multiprocessing logging if only running one process
  Fix a pair of gitfs bugs (saltstack#34218)
  rsync state: Removed source existance check saltstack#25251
  Linted saltstack#34200
  Allow specifying memory, cpu and vcpu as kwargs for OpenNebula (saltstack#34203)
  Fixes saltstack#34181 no more newlines in long yaml encodes
  Clarify pkg.list_repo_pkgs docstring for held packages (saltstack#34188)
  Update saltutil.wheel docs to specify remote vs local minion behavior
  fix regression in s3.query from saltstack#33682
  Change target for dockerng assuming default status to Nitrogen release (saltstack#34206)
  Correct the docstrings formatting in pkgbuild modules and state (saltstack#34194)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug broken, incorrect, or confusing behavior Core relates to code central or existential to Salt P4 Priority 4 severity-medium 3rd level, incorrect or bad functionality, confusing and lacks a work around
Projects
None yet
Development

No branches or pull requests

2 participants