A few questions...

What version of Ansible are you using?

If you run this without async (maybe on a small directory), do you still
see problems (such as the command failing?)

On Sun, May 12, 2013 at 5:00 PM, Matthew Johnson
notifications@github.comwrote:

Using this task definition:

33 - command: s3cmd put --recursive /mnt/dir1/ s3://bucket1/obj1/obj2-${item}/${ansible_hostname}/
34 with_pipe: date +%Y%m%d-%H%M
35 only_if: "'$action' == 'backup'"
36 async: 14400
37 poll: 120

Returns this error:

ERROR: unexpected error: unable to determine jid

Date is being fetched appropriately with the pipe lookup; the s3cmd job
launches and continues to run in the background on the target hosts.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/2897
.

Ansible 1.1

I re-tested this using the binary distribution of Tomcat 7.0.33; it is 14MB and 673 files.

Without async + poll, and keeping the with_pipe lookup, the job completes as expected.

Adding async + poll back to the job (using 600/30 for the smaller file sizes) causes the error.

Conversely, keeping the async + poll and stripping the with_pipe lookup allows the job polling to proceed as expected.

Ok, might be worth testing under the 1.2 development version, though
nothing around async has really changed in a while.

Does your operation complete (i.e. the async error is spurious) or not?

Also, when does it fail? (immediately, later?)

On Sun, May 12, 2013 at 6:31 PM, Matthew Johnson
notifications@github.comwrote:

Ansible 1.1

I re-tested this using the binary distribution of Tomcat 7.0.33; it is
14MB and 673 files.

Without async + poll, the job completes as expected.

Adding async + poll back to the job (using 600/30 for the smaller file
sizes) causes the error.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/2897#issuecomment-17786628
.

I see this when the jid error is thrown:

TASK: [command s3cmd put --recursive /mnt/apache-tomcat-7.0.33/ s3://bucket/dir1/tomcat-${item}/${ansible_hostname}/] ********************* 
ok: [host+host-03] => (item=20130512-1854)
ok: [host+host-01] => (item=20130512-1854)
ok: [host+host-05] => (item=20130512-1854)
ok: [host+host-04] => (item=20130512-1854)
ok: [host+host-06] => (item=20130512-1854)
ok: [host+host-02] => (item=20130512-1854)
ERROR: unexpected error: unable to determine jid

The s3cmd is launched successfully on the target hosts and it runs to completion; I can verify this by logging into the target hosts and checking the processes. The data also shows up in s3.

The playbook aborts processing and the steps after the s3cmd are not run.

Compared to this, when the lookup plugin is stripped:

TASK: [command s3cmd put --recursive /mnt/apache-tomcat-7.0.33/ s3://bucket/dir1/tomcat-nopipe/${ansible_hostname}/] ********************* 
<job 40433669996> polling, 570s remaining
changed: [host+host-02]
changed: [host+host-01]
changed: [host+host-06]
changed: [host+host-05]
changed: [host+host-04]
changed: [host+host-03]
<job 40433669996> finished on host+host-06
<job 40433669996> finished on host+host-05
<job 40433669996> finished on host+host-04
<job 40433669996> finished on host+host-03
<job 40433669996> finished on host+host-02
<job 40433669996> finished on host+host-01

I see the same behavior with devel branch.

I'm also not sure why the with_pipe part should matter here.

Assuming it's actually not related.

We've decided to flat out disallow this, on the basis that tasks that use something like with_items or with_sequence means all of the tasks are started up at once, which can cause problems. We may revisit this in the future, but for now to accomplish the same thing you're doing above you can use the shell command and register the output into a variable for use in the task.