Weird error with shell heredocs in 2.4.1.0

[karlmdavis]

ISSUE TYPE

• Bug Report

COMPONENT NAME

YAML parsing

ANSIBLE VERSION

2.4.1.0

(Apologies, didn't have Travis recording the full ansible --version.)

CONFIGURATION

Here's the config file used in my tests: ansible.cfg

OS / ENVIRONMENT

N/A

SUMMARY

Started getting a weird test failure in Travis when I tested against 2.4.1.0. Here's the failing build: ansible-role-jenkins2: Job #145.1. I have a separate job in that build that runs the exact same tests against 2.4.0.0, which works fine.

The problem is this task (apologies for the length):

    - name: Configure Security
      shell:
        cmd: |
          cat <<EOF |
          // These are the basic imports that Jenkin's interactive script console
          // automatically includes.
          import jenkins.*;
          import jenkins.model.*;
          import hudson.*;
          import hudson.model.*;
          def securityRealm = new hudson.security.HudsonPrivateSecurityRealm(false)
          if(!securityRealm.equals(Jenkins.instance.getSecurityRealm())) {
            Jenkins.instance.setSecurityRealm(securityRealm)
            def testUser = securityRealm.createAccount("test", "supersecret")
            testUser.addProperty(new hudson.tasks.Mailer.UserProperty("foo@example.com"));
            testUser.addProperty(new org.jenkinsci.main.modules.cli.auth.ssh.UserPropertyImpl("{{ jenkins_user_ssh_public_key }}"))
            testUser.save()
            Jenkins.instance.save()
            println "Changed authentication."
          }
          def authorizationStrategy = new hudson.security.FullControlOnceLoggedInAuthorizationStrategy()
          if(!authorizationStrategy.equals(Jenkins.instance.getAuthorizationStrategy())) {
            authorizationStrategy.setAllowAnonymousRead(false)
            Jenkins.instance.setAuthorizationStrategy(authorizationStrategy)
            Jenkins.instance.save()
            println "Changed authorization."
          }
          EOF
          {{ jenkins_cli_command }} groovy =
      become: true
      become_user: jenkins
      register: shell_jenkins_security
      changed_when: "(shell_jenkins_security | success) and 'Changed' not in shell_jenkins_security.stdout"

Now, that's not the prettiest thing ever, I confess -- I'll eventually get around to writing a module to clean it up. But the screwy thing here is that if I delete the apostophe in "Jenkin's" on the first comment line, it works just fine.

STEPS TO REPRODUCE

(see above)

EXPECTED RESULTS

I expected this to work.

ACTUAL RESULTS

TASK [Configure Security] ******************************************************

Monday 30 October 2017  01:32:49 +0000 (0:00:00.029)       0:02:14.880 ******** 

fatal: [docker_container]: FAILED! => {"changed": false, "failed": true, "module_stderr": "bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)\nTraceback (most recent call last):\n  File \"/tmp/ansible_uJ8mMn/ansible_module_command.py\", line 213, in <module>\n    main()\n  File \"/tmp/ansible_uJ8mMn/ansible_module_command.py\", line 185, in main\n    rc, out, err = module.run_command(args, executable=executable, use_unsafe_shell=shell, encoding=None, data=stdin)\n  File \"/tmp/ansible_uJ8mMn/ansible_modlib.zip/ansible/module_utils/basic.py\", line 2684, in run_command\n  File \"/usr/lib/python2.7/shlex.py\", line 279, in split\n    return list(lex)\n  File \"/usr/lib/python2.7/shlex.py\", line 269, in next\n    token = self.get_token()\n  File \"/usr/lib/python2.7/shlex.py\", line 96, in get_token\n    raw = self.read_token()\n  File \"/usr/lib/python2.7/shlex.py\", line 172, in read_token\n    raise ValueError, \"No closing quotation\"\nValueError: No closing quotation\n", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 1}

	to retry, use: --limit @/home/travis/build/karlmdavis/ansible-role-jenkins2/test/test_basic.retry

[bcoca]

the error seems unrelated to the heredoc itself, but caused bye locale settings.

can you run with -vvv to get a better view of what is happening?

[karlmdavis]

Here you go: https://travis-ci.org/karlmdavis/ansible-role-jenkins2#L5476

Blowing up on the apostrophe here: https://github.com/karlmdavis/ansible-role-jenkins2/blob/053a21821770e88699143bc53fc30dfe93918a8e/test/test_basic.yml#L75

[karlmdavis]

@bcoca Apologies: the Travis link I included there was no good. Here's a corrected one that points to verbose Ansible log output with the failure: https://travis-ci.org/karlmdavis/ansible-role-jenkins2/builds/297648635#L5481

When you say that this seems to be caused by locale settings, do you mean I could workaround this by adjusting my config somewhere/somehow?

[sivel]

We specifically recommend against using heredocs and multiline commands via the shell module.

See 5e7ee9df0aa

Instead, you should use the script module or template+command to execute a script instead.

If you have further questions please stop by IRC or the mailing list:

• IRC: #ansible on irc.freenode.net
• mailing list: https://groups.google.com/forum/#!forum/ansible-project

[karlmdavis]

@sivel That's totally fine, of course -- it's your project! However, in terms of making Ansible plays more maintainable, I've found that including short scripts inline like this can really improve readability. Less jumping between files just to figure out what's going on.

I'd suggest reconsidering this recommendation in light of that. Ansible already has issues with things getting scattered across too many files, so anything that helps reverse that trend -- even marginally -- can be a big win.