Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ansible 2.0 - synchronize change in functionality - broke my deploys #13825

Closed
undeadops opened this issue Jan 13, 2016 · 31 comments
Closed

Ansible 2.0 - synchronize change in functionality - broke my deploys #13825

undeadops opened this issue Jan 13, 2016 · 31 comments
Labels
P2
Milestone

Comments

@undeadops
Copy link

@undeadops undeadops commented Jan 13, 2016

I had been using a play for a few months now on the 1.9 releases. It looks like the following:

{code}
- name: Ensure All SSL Certs Are Copied to Server
synchronize: src=ssl/ dest=/opt/backup/ssl/ archive=no recursive=yes
{code}

With Ansible 2.0, its now trying to sudo on my local host, in order to be able to sudo copy them on the remote host.

From the Docs:
Note
The user and permissions for the synchronize src are those of the user running the Ansible task on the local host, or the become_user if become: yes is active. synchronize will attempt to escalate privileges to the become_user on the local host.

This is counter to the previous functionality. The user running the play has access to the remote server, the Local root account isn't available to use.

I'm not understanding the purpose of the local sudo when "become:" is enabled. When the module is showing a "set_remote_user" option as well. the purpose of 'become' is for the remote system. not the local one. Make the 'set_remote_user' to be 'set_local_user'? and keep the same functionality as was present in 1.9?

@jacobweber

This comment has been minimized.

Copy link
Contributor

@jacobweber jacobweber commented Jan 13, 2016

This sounds like it's the cause of #13034 . I'm guessing that using "become" on the remote host isn't possible, since it's just running the rsync command locally (thus Ansible has no control over the remote side). So there's no harm in setting "become: false" on your synchronize tasks, since it will only affect the local side.

@undeadops

This comment has been minimized.

Copy link
Author

@undeadops undeadops commented Jan 13, 2016

Yes, looks like the same issue.

On Wed, Jan 13, 2016 at 2:03 PM, Jacob Weber notifications@github.com
wrote:

This sounds like it's the cause of #13034
#13034, which is also causing
problems for me.


Reply to this email directly or view it on GitHub
#13825 (comment).

@nocive

This comment has been minimized.

Copy link

@nocive nocive commented Jan 14, 2016

Facing the same issue, the change in behaviour on the synchronize module broke my deployments as well 😞

Currently working around the problem with: rsync_opts: --rsh "ssh -l {{ lookup('env', 'USER') }}", forcing ansible to use the real user and not root.

@abadger abadger added the P2 label Jan 15, 2016
@abadger abadger added this to the v2 milestone Jan 15, 2016
@abadger

This comment has been minimized.

Copy link
Member

@abadger abadger commented Jan 15, 2016

We're looking at what kind of fix we can apply to this for 2.0.1 or 2.0.2. synchronize has always been somewhat hacky and unfortunately the hacks will interact badly with the become code that is in 2.0. We're hoping that even if we can't make this code perfect for the next release that we can at least get something more like the 1.9.x behaviour. However, there's a lot of ugliness in the code so we'll have to be careful not to break other things even worse.

Am I correct in thinking that what you all want is for no privilege escalation to be performed locally and privileges to be escalated on the remote host via passwordless sudo? I think that's the limitations that were imposed on this usage in 1.9.x... just making sure that I'm not missing any wrinkles and additional features there.

@jacobweber

This comment has been minimized.

Copy link
Contributor

@jacobweber jacobweber commented Jan 15, 2016

Not sure about the original poster, but I didn't really need privilege escalation on the remote host, so setting become=false worked for me. Having "become" affect the local host was what confused me.

@nocive

This comment has been minimized.

Copy link

@nocive nocive commented Jan 15, 2016

@abadger In my case I'm using privilege escalation already for both sides with become: true and rsync_path: "sudo rsync".
What i want is for ansible to respect what was formerly the ansible_ssh_user fact and pass it along to ssh (with the -l flag) when rsycing, in order to take advantage of the user's ssh key i have setup (and not the root key). This was the observed behaviour in ansible 1.9.x.

@abadger

This comment has been minimized.

Copy link
Member

@abadger abadger commented Jan 15, 2016

@nocive From my brief testing, 1.9.x does not do privilege escalation on the controller. Can you show how you have that working?

@nocive

This comment has been minimized.

Copy link

@nocive nocive commented Jan 15, 2016

@abadger Apologies, maybe I didn't explain myself properly.
I'm using it in pull mode together with delegate_to, so none of it actually runs on the controller host.

synchronize:
    src:        "{{ some_src }}/"
    dest:       "{{ some_dest }}"
    mode:       pull
    archive:    yes
    compress:   yes
    rsync_path: "sudo rsync"
    #rsync_opts: --rsh "ssh -l {{ lookup('env', 'USER') }}" # for ansible 2.x
  sudo: true
  delegate_to: "{{ item }}"
  when: "play_hosts|length > 1 and item not in groups['foo'] and item not in groups['bar']"
  with_items: play_hosts
@abadger

This comment has been minimized.

Copy link
Member

@abadger abadger commented Jan 15, 2016

@nocive That's not working for me either with 1.9.4 (although I think the issue is bugs in 1.9's delegate_to handling rather than in it's sudo handling... but I'll have to figure out how to check that to be sure). Could you check for me that 1.9 really is escalating privileges on the delegate_to host? If you make some_src readable only by root does it succeed?

@abadger

This comment has been minimized.

Copy link
Member

@abadger abadger commented Jan 15, 2016

Warning: This commit isn't tested with anything except the basic problem listed in this ticket so with further testing it may not prove suitable for inclusion in the next release. I'm linking it here so you all can take a look at it but like I say, even though this change appears to fix this problem, there's no promise yet that it will go into the next release:

https://github.com/ansible/ansible/compare/synchronize-become-is-reversed?expand=1

That said, anyone who can test is encouraged to do so. I finding important cornercases is mostly about more people using it in their real-world code.

@undeadops

This comment has been minimized.

Copy link
Author

@undeadops undeadops commented Jan 15, 2016

On Thu, Jan 14, 2016 at 6:07 PM, Toshio Kuratomi notifications@github.com
wrote:

Am I correct in thinking that what you all want is for no privilege
escalation to be performed locally and privileges to be escalated on the
remote host via passwordless sudo? I think that's the limitations that were
imposed on this usage in 1.9.x... just making sure that I'm not missing any
wrinkles and additional features there

​Yes, this is exactly how I used it.​

@abadger

This comment has been minimized.

Copy link
Member

@abadger abadger commented Jan 19, 2016

For those following along at home, a couple new commits to this branch: https://github.com/ansible/ansible/compare/synchronize-become-is-reversed?expand=1

which fix some problems with delegate_to and sudo.

Also, proposed update to the module documentation: https://gist.github.com/abadger/b719c89497a7018f51ae

@jpcope

This comment has been minimized.

Copy link

@jpcope jpcope commented Jan 20, 2016

If it helps anyone playing along at home:
1.9.2 does not have the issue; 1.9.4 and later appear to have the issue.

@pwaring

This comment has been minimized.

Copy link

@pwaring pwaring commented Jan 23, 2016

Setting become_user: False at the top of the playbook fixed the problem for me in 2.0.0.2. The task which was having the problem was:

- name: synchronize static sites
      with_items: static_sites
      synchronize:
        src: "{{ static_sites_local_dir }}/{{ item }}/output/"
        dest: "{{ vhosts_top_dir }}/{{ item }}/public_html/"
        archive: no
        recursive: yes
@abadger abadger closed this in 3cf59d3 Jan 26, 2016
abadger added a commit that referenced this issue Jan 26, 2016
* In 2.0.0.x become was reversed for synchronize. It was happening on
  the local machine instead of the remote machine. This restores the
  ansible-1.9.x behaviour of doing become on the remote machine.
  However, there's aspects of this that are hacky (no hackier than
  ansible-1.9 but not using 2.0 features).  The big problem is that it
  does not understand any become method except sudo.  I'm willing to use
  a partial fix now because we don't want people to get used to the
  reversed semantics in their playbooks.
* synchronize copying to the wrong host when inventory_hostname is
  localhost
* Fix problem with unicode arguments (first seen as a bug on synchronize)

Fixes #14041
Fixes #13825
@abadger

This comment has been minimized.

Copy link
Member

@abadger abadger commented Jan 26, 2016

Okay, the synchronize branch I've been working on to fix this has been merged to devel and stable-2.0. This should now be fixed if you checkout and run either of those from a checkout. It is schedued to be in the 2.0.1 release.

@makmanalp

This comment has been minimized.

Copy link
Contributor

@makmanalp makmanalp commented Feb 4, 2016

🍰 This solved my issue - the tricky part was realizing that the Password: prompt was a local one versus a remote one and for sudo vs ssh login.

@javiermatos

This comment has been minimized.

Copy link

@javiermatos javiermatos commented Feb 8, 2016

Hi, I'm still having the issue using the latest ansible version (2.0.0.2). The problem appears when I use the rsync_opts to change ownership.

I have a main.yml file with the following:

- include: backend.yml
  become_user: '{{ project_user }}'

and then I have my backend.yml as follows:

- name: update backend
  synchronize:
    src: '{{ backend_path }}/'
    dest: '{{ project_path }}/backend/'
    delete: yes
    rsync_opts: '--chown={{ project_user }}:{{ project_user }}'

It works on ansible 1.9.4 but on 2.0.0.2 it has an error:
fatal: [192.168.220.104]: FAILED! => {"changed": false, "cmd": "rsync --delay-updates -F --compress --delete-after --archive --rsh 'ssh -S none -o StrictHostKeyChecking=no' --rsync-path=\"sudo rsync\" --out-format='<<CHANGED>>%i %n%L' \"/home/ubuntu/Workspace/go/src/backend/\" \"192.168.220.104:/srv/go/backend/\"", "failed": true, "msg": "Could not create directory '/var/www/.ssh'.\r\nFailed to add the host to the list of known hosts (/var/www/.ssh/known_hosts).\r\nPermission denied (publickey).\r\nrsync: connection unexpectedly closed (0 bytes received so far) [sender]\nrsync error: unexplained error (code 255) at io.c(226) [sender=3.1.0]\n", "rc": 255}

My project_user is www-data, that's why it tries to add host to /var/www/.ssh. As I told, it works when moving back to ansible 1.9.4. I think mine is a strange corner case because of the ownership thing.

@magnetik

This comment has been minimized.

Copy link
Contributor

@magnetik magnetik commented Feb 15, 2016

I had a playbook that was working on 1.9.(not sure) but not in 2.0.0.2

- name: Copy config includes
  synchronize: src=includes dest=/etc/apache2

with the global config become = True

But had it working by changing it to:

- name: Copy config includes
  synchronize: src=includes dest=/etc/apache2 rsync_path="sudo rsync"
  become: false
@lfalcao

This comment has been minimized.

Copy link

@lfalcao lfalcao commented Feb 18, 2016

@magnetik 👍 tks

@ramayer

This comment has been minimized.

Copy link

@ramayer ramayer commented Apr 27, 2016

The workaround to explicitly call rsync as mentioned here works well for me:

https://groups.google.com/d/msg/ansible-project/aC3WStjfoh4/Hd1Pjvvsb-wJ

I.e. use this:

rsync task:

    - name: my copy in bin dir
      sudo: no
      local_action: >
        command
        rsync
        --delay-updates -FF --compress --timeout=10 --delete-after
        --archive --no-owner --no-group
        --rsh 'ssh -i /home/mumble/.ansible/keys/mumble-id_rsa -o stricthostkeychecking=no'   
        --rsync-path 'sudo rsync'
        --out-format='<<CHANGED>>%i %n%L'
        {{ inventory_dir }}/working/mumble-bin/
        admin@{{ inventory_hostname }}:/mumble/bin

instead of the synchronize module.

@jeff1985

This comment has been minimized.

Copy link

@jeff1985 jeff1985 commented May 3, 2016

My vagrant deployment also has been broken with the switch from ansible 1.9.2 to 2.0.2
In my case ansible has been trying to sync the given local path to the global path on the same machine. I had to explicitly set
dest="{{ ansible_ssh_user }}@{{ ansible_ssh_host }}:/etc/somedir/" to fix it.

@flyte

This comment has been minimized.

Copy link
Contributor

@flyte flyte commented May 10, 2016

I have the same issue as @jeff1985 in Ansible 2.0.2. The docs say:

# Synchronization of src on the control machine to dest on the remote hosts
synchronize: src=some/relative/path dest=/some/absolute/path

but in fact the src and dest are both relative to my own machine and not the remote host.

@jeff1985

This comment has been minimized.

Copy link

@jeff1985 jeff1985 commented May 10, 2016

@flyte
i must admit, my issue must have to do with using vagrant in combination with ansible as provisioner. i had to use the usual dest="/etc/somedir/" when i tried to run the playbook on a remote host without vagrant. so i believe there might me some bug related to handling local virtual machines.

@jeffwidman

This comment has been minimized.

Copy link
Contributor

@jeffwidman jeffwidman commented May 10, 2016

I am still hitting this on a playbook with a globally defined become: true:

- name: Deploy code using rsync
  synchronize:
    mode: push
    src: ../data_pusher/
    dest: "{{ code_path }}/test"
    archive: no
    recursive: yes
  become: false

When I don't set become: false at the task level, I get an error about three password failures, which I assume is my macbook because other sudo tasks on the remote host are succeeding.

@mydagobah

This comment has been minimized.

Copy link

@mydagobah mydagobah commented May 25, 2016

@jeffwidman I had the same issue with Ansible 2.0.2.0. My walk around is using rsync_path

synchronize:
 ...
 rsync_path: "sudo -u <remote user> rsync"
@jesusch

This comment has been minimized.

Copy link

@jesusch jesusch commented Jun 1, 2016

+1

@nocive

This comment has been minimized.

Copy link

@nocive nocive commented Jun 8, 2016

bumped into this again after another try at migrating my codebase to ansible 2.x

@tima

This comment has been minimized.

Copy link
Contributor

@tima tima commented Jun 8, 2016

@nocive and others: This issue was closed some time ago and isn't going to get the attention of the core developers.

@nocive

This comment has been minimized.

Copy link

@nocive nocive commented Jun 8, 2016

@tima fair enough, im still gonna rant about it anyway.

@ritzk

This comment has been minimized.

Copy link
Contributor

@ritzk ritzk commented Jun 16, 2016

@nocive could you try - #16138 .

@abadger

This comment has been minimized.

Copy link
Member

@abadger abadger commented Jun 16, 2016

@nocive what tima means is that once an issue has been "closed, fixed", people who can do something about any problems that exist after the fix has been applied are probably not going to see the comments on the closed ticket. Opening the new ticket is the way to get attention on the issue.

For people suffering issues with synchronize and vagrant pushing files to the wrong machine, we merged this PR in the last few days which probably fixes that bug: #15993

I'm going to lock this issue now. if you are having issues still, (1) search for open issues that have the same symptoms to chime in on rather than closed issues. (2) if no open issue exists, open a fresh issue so that we see that there's still problems and can sort them out.

@ansible ansible locked and limited conversation to collaborators Jun 16, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
You can’t perform that action at this time.