Unexpected failure during module execution: 'ansible_job_id' when using multiple facts modules on Windows on ansible-core 2.16.0 and later

[samueln-w]

Summary

Hello!

When I configure a Windows machine to use multiple ansible_facts_modules, I get Unexpected failure during module execution: 'ansible_job_id' during the gathering facts stage of any playbook. This error didn't previously occur (in Ansible 2.14) with the same configuration, but occurs now in 2.16.

Issue Type

Bug Report

Component Name

gather_facts

Ansible Version

# See also Steps to Reproduce - this is the version I happened to have installed while writing this report.
$ ansible --version
ansible [core 2.16.0]
  config file = /home/samueln/ansible-error-test/ansible.cfg
  configured module search path = ['/home/samueln/ansible-error-test/modules', '/home/samueln/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/samueln/ansible-error-test/venv/lib64/python3.11/site-packages/ansible
  ansible collection location = /home/samueln/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/samueln/ansible-error-test/venv/bin/ansible
  python version = 3.11.5 (main, Oct 25 2023, 16:19:59) [GCC 8.5.0 20210514 (Red Hat 8.5.0-20)] (/home/samueln/ansible-error-test/venv/bin/python)
  jinja version = 3.1.4
  libyaml = True

Configuration

# if using a version older than ansible-core 2.12 you should omit the '-t all'
$ ansible-config dump --only-changed -t all
ANSIBLE_COW_ACCEPTLIST(/home/samueln/ansible-error-test/ansible.cfg) = ["['bunny'", 'default', 'hellokitty', 'koala', 'moofasa', 'moose', 'sheep', 'small', 'three-eyes', "'tux']"]
ANSIBLE_COW_SELECTION(/home/samueln/ansible-error-test/ansible.cfg) = random
ANSIBLE_NOCOWS(/home/samueln/ansible-error-test/ansible.cfg) = True
CONFIG_FILE() = /home/samueln/ansible-error-test/ansible.cfg
DEFAULT_ASK_PASS(/home/samueln/ansible-error-test/ansible.cfg) = True
DEFAULT_BECOME_ASK_PASS(/home/samueln/ansible-error-test/ansible.cfg) = True
DEFAULT_FORKS(/home/samueln/ansible-error-test/ansible.cfg) = 20
DEFAULT_HOST_LIST(/home/samueln/ansible-error-test/ansible.cfg) = ['/home/samueln/ansible-error-test/inventory.yml']
DEFAULT_LOG_PATH(/home/samueln/ansible-error-test/ansible.cfg) = /home/samueln/ansible-error-test/logs/latest.log
DEFAULT_MODULE_PATH(/home/samueln/ansible-error-test/ansible.cfg) = ['/home/samueln/ansible-error-test/modules', '/home/samueln/.ansible/plugins/modules', '/usr/share/ansible/plugin>
EDITOR(env: EDITOR) = nano
HOST_KEY_CHECKING(/home/samueln/ansible-error-test/ansible.cfg) = False

CONNECTION:
==========

paramiko_ssh:
____________
host_key_checking(/home/samueln/ansible-error-test/ansible.cfg) = False

ssh:
___
host_key_checking(/home/samueln/ansible-error-test/ansible.cfg) = False
(END)

OS / Environment

The control node has Rocky 8. Failing hosts have Windows 10.

Steps to Reproduce

The error seems to have been introduced in ansible-core 2.16.0. I have tested the following versions of ansible-core for the error:

• 2.13.4: does not occur
• 2.15.11: does not occur
• 2.16.0: occurs
• 2.16.6: occurs

The following ansible_ variables are set via inventory on the affected Windows machines. I don't expect them to be relevant, but I'm including them in case they are:

ansible_connection: ssh
ansible_shell_type: cmd
ansible_become_method: runas
ansible_become_user: System

Beyond that, perhaps the simplest way to reproduce the error is to create a simple custom facts module:

win_helloworld_facts.ps1

#!powershell

#AnsibleRequires -CSharpUtil Ansible.Basic

$spec = @{
    supports_check_mode = $true
}

$module = [Ansible.Basic.AnsibleModule]::Create($args, $spec)

$module.Result.ansible_facts = @{
    helloworld = "Hello World!"
}

$module.ExitJson()

...then configure a Windows machine to use both that module and smart (although you'd get the same result with 2 custom facts modules instead):

group_vars/windows.yml, or however you see fit to introduce this variable:

ansible_facts_modules:
- smart
- win_helloworld_facts

...then run a basic do-nothing playbook with fact gathering enabled:

test.yml

- name: test
  hosts: all
  tasks:

  - name: debug
    ansible.builtin.debug:
      msg: "Hello World!"

ansible-playbook --limit some-windows-machine test.yml

Expected Results

I expect the playbook to run through and print "Hello World!"

Actual Results

(venv) [samueln@[redacted] ansible-error-test]$ ansible-playbook --limit [redacted] test.yml -vvvv
ansible-playbook [core 2.16.0]
  config file = /home/samueln/ansible-error-test/ansible.cfg
  configured module search path = ['/home/samueln/ansible-error-test/modules', '/home/samueln/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/samueln/ansible-error-test/venv/lib64/python3.11/site-packages/ansible
  ansible collection location = /home/samueln/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/samueln/ansible-error-test/venv/bin/ansible-playbook
  python version = 3.11.5 (main, Oct 25 2023, 16:19:59) [GCC 8.5.0 20210514 (Red Hat 8.5.0-20)] (/home/samueln/ansible-error-test/venv/bin/python)
  jinja version = 3.1.4
  libyaml = True
Using /home/samueln/ansible-error-test/ansible.cfg as config file
SSH password: 
BECOME password[defaults to SSH password]: 
setting up inventory plugins
Loading collection ansible.builtin from 
host_list declined parsing /home/samueln/ansible-error-test/inventory.yml as it did not pass its verify_file() method
script declined parsing /home/samueln/ansible-error-test/inventory.yml as it did not pass its verify_file() method
Skipping empty key (hosts) in group (windows_11)
Parsed /home/samueln/ansible-error-test/inventory.yml inventory source with yaml plugin
Loading callback plugin default of type stdout, v2.0 from /home/samueln/ansible-error-test/venv/lib64/python3.11/site-packages/ansible/plugins/callback/default.py
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.

PLAYBOOK: test.yml ***************************************************************************************************************************************************************************
Positional arguments: test.yml
verbosity: 4
connection: ssh
ask_pass: True
become_method: sudo
become_ask_pass: True
tags: ('all',)
inventory: ('/home/samueln/ansible-error-test/inventory.yml',)
subset: [redacted]
forks: 20
1 plays in test.yml

PLAY [test] **********************************************************************************************************************************************************************************

TASK [Gathering Facts] ***********************************************************************************************************************************************************************
task path: /home/samueln/ansible-error-test/test.yml:1
Running win_helloworld_facts
Using module file /home/samueln/ansible-error-test/modules/win_helloworld_facts.ps1
Pipelining is enabled.
<[redacted]> ESTABLISH SSH CONNECTION FOR USER: None
<[redacted]> SSH: EXEC sshpass -d13 ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o ConnectTimeout=10 -o 'ControlPath="/home/samueln/.ansible/cp/0eaef8ca5a"' [redacted] 'chcp.com 65001 >nul 2>&1 && PowerShell -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -EncodedCommand UABvAHcAZQByAFMAaABlAGwAbAAgAC0ATgBvAFAAcgBvAGYAaQBsAGUAIAAtAE4AbwBuAEkAbgB0AGUAcgBhAGMAdABpAHYAZQAgAC0ARQB4AGUAYwB1AHQAaQBvAG4AUABvAGwAaQBjAHkAIABVAG4AcgBlAHMAdAByAGkAYwB0AGUAZAAgAC0ARQBuAGMAbwBkAGUAZABDAG8AbQBtAGEAbgBkACAASgBnAEIAagBBAEcAZwBBAFkAdwBCAHcAQQBDADQAQQBZAHcAQgB2AEEARwAwAEEASQBBAEEAMgBBAEQAVQBBAE0AQQBBAHcAQQBEAEUAQQBJAEEAQQArAEEAQwBBAEEASgBBAEIAdQBBAEgAVQBBAGIAQQBCAHMAQQBBAG8AQQBhAFEAQgBtAEEAQwBBAEEASwBBAEEAawBBAEYAQQBBAFUAdwBCAFcAQQBHAFUAQQBjAGcAQgB6AEEARwBrAEEAYgB3AEIAdQBBAEYAUQBBAFkAUQBCAGkAQQBHAHcAQQBaAFEAQQB1AEEARgBBAEEAVQB3AEIAVwBBAEcAVQBBAGMAZwBCAHoAQQBHAGsAQQBiAHcAQgB1AEEAQwBBAEEATABRAEIAcwBBAEgAUQBBAEkAQQBCAGIAQQBGAFkAQQBaAFEAQgB5AEEASABNAEEAYQBRAEIAdgBBAEcANABBAFgAUQBBAGkAQQBEAE0AQQBMAGcAQQB3AEEAQwBJAEEASwBRAEEAZwBBAEgAcwBBAEMAZwBBAG4AQQBIAHMAQQBJAGcAQgBtAEEARwBFAEEAYQBRAEIAcwBBAEcAVQBBAFoAQQBBAGkAQQBEAG8AQQBkAEEAQgB5AEEASABVAEEAWgBRAEEAcwBBAEMASQBBAGIAUQBCAHoAQQBHAGMAQQBJAGcAQQA2AEEAQwBJAEEAUQBRAEIAdQBBAEgATQBBAGEAUQBCAGkAQQBHAHcAQQBaAFEAQQBnAEEASABJAEEAWgBRAEIAeABBAEgAVQBBAGEAUQBCAHkAQQBHAFUAQQBjAHcAQQBnAEEARgBBAEEAYgB3AEIAMwBBAEcAVQBBAGMAZwBCAFQAQQBHAGcAQQBaAFEAQgBzAEEARwB3AEEASQBBAEIAMgBBAEQATQBBAEwAZwBBAHcAQQBDAEEAQQBiAHcAQgB5AEEAQwBBAEEAYgBnAEIAbABBAEgAYwBBAFoAUQBCAHkAQQBDAEkAQQBmAFEAQQBuAEEAQQBvAEEAWgBRAEIANABBAEcAawBBAGQAQQBBAGcAQQBEAEUAQQBDAGcAQgA5AEEAQQBvAEEASgBBAEIAbABBAEgAZwBBAFoAUQBCAGoAQQBGADgAQQBkAHcAQgB5AEEARwBFAEEAYwBBAEIAdwBBAEcAVQBBAGMAZwBCAGYAQQBIAE0AQQBkAEEAQgB5AEEAQwBBAEEAUABRAEEAZwBBAEMAUQBBAGEAUQBCAHUAQQBIAEEAQQBkAFEAQgAwAEEAQwBBAEEAZgBBAEEAZwBBAEUAOABBAGQAUQBCADAAQQBDADAAQQBVAHcAQgAwAEEASABJAEEAYQBRAEIAdQBBAEcAYwBBAEMAZwBBAGsAQQBIAE0AQQBjAEEAQgBzAEEARwBrAEEAZABBAEIAZgBBAEgAQQBBAFkAUQBCAHkAQQBIAFEAQQBjAHcAQQBnAEEARAAwAEEASQBBAEEAawBBAEcAVQBBAGUAQQBCAGwAQQBHAE0AQQBYAHcAQgAzAEEASABJAEEAWQBRAEIAdwBBAEgAQQBBAFoAUQBCAHkAQQBGADgAQQBjAHcAQgAwAEEASABJAEEATABnAEIAVABBAEgAQQBBAGIAQQBCAHAAQQBIAFEAQQBLAEEAQgBBAEEAQwBnAEEASQBnAEIAZwBBAEQAQQBBAFkAQQBBAHcAQQBHAEEAQQBNAEEAQgBnAEEARABBAEEASQBnAEEAcABBAEMAdwBBAEkAQQBBAHkAQQBDAHcAQQBJAEEAQgBiAEEARgBNAEEAZABBAEIAeQBBAEcAawBBAGIAZwBCAG4AQQBGAE0AQQBjAEEAQgBzAEEARwBrAEEAZABBAEIAUABBAEgAQQBBAGQAQQBCAHAAQQBHADgAQQBiAGcAQgB6AEEARgAwAEEATwBnAEEANgBBAEYASQBBAFoAUQBCAHQAQQBHADgAQQBkAGcAQgBsAEEARQBVAEEAYgBRAEIAdwBBAEgAUQBBAGUAUQBCAEYAQQBHADQAQQBkAEEAQgB5AEEARwBrAEEAWgBRAEIAegBBAEMAawBBAEMAZwBCAEoAQQBHAFkAQQBJAEEAQQBvAEEAQwAwAEEAYgBnAEIAdgBBAEgAUQBBAEkAQQBBAGsAQQBIAE0AQQBjAEEAQgBzAEEARwBrAEEAZABBAEIAZgBBAEgAQQBBAFkAUQBCAHkAQQBIAFEAQQBjAHcAQQB1AEEARQB3AEEAWgBRAEIAdQBBAEcAYwBBAGQAQQBCAG8AQQBDAEEAQQBMAFEAQgBsAEEASABFAEEASQBBAEEAeQBBAEMAawBBAEkAQQBCADcAQQBDAEEAQQBkAEEAQgBvAEEASABJAEEAYgB3AEIAMwBBAEMAQQBBAEkAZwBCAHAAQQBHADQAQQBkAGcAQgBoAEEARwB3AEEAYQBRAEIAawBBAEMAQQBBAGMAQQBCAGgAQQBIAGsAQQBiAEEAQgB2AEEARwBFAEEAWgBBAEEAaQBBAEMAQQBBAGYAUQBBAEsAQQBGAE0AQQBaAFEAQgAwAEEAQwAwAEEAVgBnAEIAaABBAEgASQBBAGEAUQBCAGgAQQBHAEkAQQBiAEEAQgBsAEEAQwBBAEEATABRAEIATwBBAEcARQBBAGIAUQBCAGwAQQBDAEEAQQBhAGcAQgB6AEEARwA4AEEAYgBnAEIAZgBBAEgASQBBAFkAUQBCADMAQQBDAEEAQQBMAFEAQgBXAEEARwBFAEEAYgBBAEIAMQBBAEcAVQBBAEkAQQBBAGsAQQBIAE0AQQBjAEEAQgBzAEEARwBrAEEAZABBAEIAZgBBAEgAQQBBAFkAUQBCAHkAQQBIAFEAQQBjAHcAQgBiAEEARABFAEEAWABRAEEASwBBAEMAUQBBAFoAUQBCADQAQQBHAFUAQQBZAHcAQgBmAEEASABjAEEAYwBnAEIAaABBAEgAQQBBAGMAQQBCAGwAQQBIAEkAQQBJAEEAQQA5AEEAQwBBAEEAVwB3AEIAVABBAEcATQBBAGMAZwBCAHAAQQBIAEEAQQBkAEEAQgBDAEEARwB3AEEAYgB3AEIAagBBAEcAcwBBAFgAUQBBADYAQQBEAG8AQQBRAHcAQgB5AEEARwBVAEEAWQBRAEIAMABBAEcAVQBBAEsAQQBBAGsAQQBIAE0AQQBjAEEAQgBzAEEARwBrAEEAZABBAEIAZgBBAEgAQQBBAFkAUQBCAHkAQQBIAFEAQQBjAHcAQgBiAEEARABBAEEAWABRAEEAcABBAEEAbwBBAEoAZwBBAGsAQQBHAFUAQQBlAEEAQgBsAEEARwBNAEEAWAB3AEIAMwBBAEgASQBBAFkAUQBCAHcAQQBIAEEAQQBaAFEAQgB5AEEAQQA9AD0A'
<[redacted]> (0, b'{"changed":false,"invocation":{"module_args":{}},"ansible_facts":{"helloworld":"Hello World!"}}\r\n\r\n', b'OpenSSH_8.0p1, OpenSSL 1.1.1k  FIPS 25 Mar 2021\r\ndebug1: Reading configuration data /home/samueln/.ssh/config\r\ndebug2: checking match for \'user [redacted]\' host [redacted] originally [redacted]\r\ndebug3: /home/samueln/.ssh/config line 1: not matched \'user "samueln"\' \r\ndebug2: match not found\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 52: Including file /etc/ssh/ssh_config.d/05-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf\r\ndebug2: checking match for \'final all\' host [redacted] originally [redacted]\r\ndebug3: /etc/ssh/ssh_config.d/05-redhat.conf line 3: not matched \'final\'\r\ndebug2: match not found\r\ndebug3: /etc/ssh/ssh_config.d/05-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-,gss-gex-sha1-,gss-group14-sha1-]\r\ndebug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1]\r\ndebug1: configuration requests final Match pass\r\ndebug1: re-parsing configuration\r\ndebug1: Reading configuration data /home/samueln/.ssh/config\r\ndebug2: checking match for \'user [redacted]\' host [redacted] originally [redacted]\r\ndebug3: /home/samueln/.ssh/config line 1: not matched \'user "samueln"\' \r\ndebug2: match not found\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 52: Including file /etc/ssh/ssh_config.d/05-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf\r\ndebug2: checking match for \'final all\' host [redacted] originally [redacted]\r\ndebug3: /etc/ssh/ssh_config.d/05-redhat.conf line 3: matched \'final\'\r\ndebug2: match found\r\ndebug3: /etc/ssh/ssh_config.d/05-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-,gss-gex-sha1-,gss-group14-sha1-]\r\ndebug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1]\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 2615185\r\ndebug3: mux_client_request_session: session request sent\r\n#< CLIXML\r\n<Objs Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04"><Obj S="progress" RefId="0"><TN RefId="0"><T>System.Management.Automation.PSCustomObject</T><T>System.Object</T></TN><MS><I64 N="SourceId">1</I64><PR N="Record"><AV>Preparing modules for first use.</AV><AI>0</AI><Nil /><PI>-1</PI><PC>-1</PC><T>Completed</T><SR>-1</SR><SD> </SD></PR></MS></Obj><Obj S="progress" RefId="1"><TNRef RefId="0" /><MS><I64 N="SourceId">2</I64><PR N="Record"><AV>Preparing modules for first use.</AV><AI>0</AI><Nil /><PI>-1</PI><PC>-1</PC><T>Completed</T><SR>-1</SR><SD> </SD></PR></MS></Obj></Objs>debug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n')
Running ansible.legacy.setup
redirecting (type: modules) ansible.builtin.setup to ansible.windows.setup
Loading collection ansible.windows from /home/samueln/ansible-error-test/venv/lib64/python3.11/site-packages/ansible_collections/ansible/windows
Using module file /home/samueln/ansible-error-test/venv/lib64/python3.11/site-packages/ansible_collections/ansible/windows/plugins/modules/setup.ps1
Pipelining is enabled.
<[redacted]> ESTABLISH SSH CONNECTION FOR USER: None
<[redacted]> SSH: EXEC sshpass -d13 ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o ConnectTimeout=10 -o 'ControlPath="/home/samueln/.ansible/cp/0eaef8ca5a"' [redacted] 'chcp.com 65001 >nul 2>&1 && PowerShell -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -EncodedCommand UABvAHcAZQByAFMAaABlAGwAbAAgAC0ATgBvAFAAcgBvAGYAaQBsAGUAIAAtAE4AbwBuAEkAbgB0AGUAcgBhAGMAdABpAHYAZQAgAC0ARQB4AGUAYwB1AHQAaQBvAG4AUABvAGwAaQBjAHkAIABVAG4AcgBlAHMAdAByAGkAYwB0AGUAZAAgAC0ARQBuAGMAbwBkAGUAZABDAG8AbQBtAGEAbgBkACAASgBnAEIAagBBAEcAZwBBAFkAdwBCAHcAQQBDADQAQQBZAHcAQgB2AEEARwAwAEEASQBBAEEAMgBBAEQAVQBBAE0AQQBBAHcAQQBEAEUAQQBJAEEAQQArAEEAQwBBAEEASgBBAEIAdQBBAEgAVQBBAGIAQQBCAHMAQQBBAG8AQQBhAFEAQgBtAEEAQwBBAEEASwBBAEEAawBBAEYAQQBBAFUAdwBCAFcAQQBHAFUAQQBjAGcAQgB6AEEARwBrAEEAYgB3AEIAdQBBAEYAUQBBAFkAUQBCAGkAQQBHAHcAQQBaAFEAQQB1AEEARgBBAEEAVQB3AEIAVwBBAEcAVQBBAGMAZwBCAHoAQQBHAGsAQQBiAHcAQgB1AEEAQwBBAEEATABRAEIAcwBBAEgAUQBBAEkAQQBCAGIAQQBGAFkAQQBaAFEAQgB5AEEASABNAEEAYQBRAEIAdgBBAEcANABBAFgAUQBBAGkAQQBEAE0AQQBMAGcAQQB3AEEAQwBJAEEASwBRAEEAZwBBAEgAcwBBAEMAZwBBAG4AQQBIAHMAQQBJAGcAQgBtAEEARwBFAEEAYQBRAEIAcwBBAEcAVQBBAFoAQQBBAGkAQQBEAG8AQQBkAEEAQgB5AEEASABVAEEAWgBRAEEAcwBBAEMASQBBAGIAUQBCAHoAQQBHAGMAQQBJAGcAQQA2AEEAQwBJAEEAUQBRAEIAdQBBAEgATQBBAGEAUQBCAGkAQQBHAHcAQQBaAFEAQQBnAEEASABJAEEAWgBRAEIAeABBAEgAVQBBAGEAUQBCAHkAQQBHAFUAQQBjAHcAQQBnAEEARgBBAEEAYgB3AEIAMwBBAEcAVQBBAGMAZwBCAFQAQQBHAGcAQQBaAFEAQgBzAEEARwB3AEEASQBBAEIAMgBBAEQATQBBAEwAZwBBAHcAQQBDAEEAQQBiAHcAQgB5AEEAQwBBAEEAYgBnAEIAbABBAEgAYwBBAFoAUQBCAHkAQQBDAEkAQQBmAFEAQQBuAEEAQQBvAEEAWgBRAEIANABBAEcAawBBAGQAQQBBAGcAQQBEAEUAQQBDAGcAQgA5AEEAQQBvAEEASgBBAEIAbABBAEgAZwBBAFoAUQBCAGoAQQBGADgAQQBkAHcAQgB5AEEARwBFAEEAYwBBAEIAdwBBAEcAVQBBAGMAZwBCAGYAQQBIAE0AQQBkAEEAQgB5AEEAQwBBAEEAUABRAEEAZwBBAEMAUQBBAGEAUQBCAHUAQQBIAEEAQQBkAFEAQgAwAEEAQwBBAEEAZgBBAEEAZwBBAEUAOABBAGQAUQBCADAAQQBDADAAQQBVAHcAQgAwAEEASABJAEEAYQBRAEIAdQBBAEcAYwBBAEMAZwBBAGsAQQBIAE0AQQBjAEEAQgBzAEEARwBrAEEAZABBAEIAZgBBAEgAQQBBAFkAUQBCAHkAQQBIAFEAQQBjAHcAQQBnAEEARAAwAEEASQBBAEEAawBBAEcAVQBBAGUAQQBCAGwAQQBHAE0AQQBYAHcAQgAzAEEASABJAEEAWQBRAEIAdwBBAEgAQQBBAFoAUQBCAHkAQQBGADgAQQBjAHcAQgAwAEEASABJAEEATABnAEIAVABBAEgAQQBBAGIAQQBCAHAAQQBIAFEAQQBLAEEAQgBBAEEAQwBnAEEASQBnAEIAZwBBAEQAQQBBAFkAQQBBAHcAQQBHAEEAQQBNAEEAQgBnAEEARABBAEEASQBnAEEAcABBAEMAdwBBAEkAQQBBAHkAQQBDAHcAQQBJAEEAQgBiAEEARgBNAEEAZABBAEIAeQBBAEcAawBBAGIAZwBCAG4AQQBGAE0AQQBjAEEAQgBzAEEARwBrAEEAZABBAEIAUABBAEgAQQBBAGQAQQBCAHAAQQBHADgAQQBiAGcAQgB6AEEARgAwAEEATwBnAEEANgBBAEYASQBBAFoAUQBCAHQAQQBHADgAQQBkAGcAQgBsAEEARQBVAEEAYgBRAEIAdwBBAEgAUQBBAGUAUQBCAEYAQQBHADQAQQBkAEEAQgB5AEEARwBrAEEAWgBRAEIAegBBAEMAawBBAEMAZwBCAEoAQQBHAFkAQQBJAEEAQQBvAEEAQwAwAEEAYgBnAEIAdgBBAEgAUQBBAEkAQQBBAGsAQQBIAE0AQQBjAEEAQgBzAEEARwBrAEEAZABBAEIAZgBBAEgAQQBBAFkAUQBCAHkAQQBIAFEAQQBjAHcAQQB1AEEARQB3AEEAWgBRAEIAdQBBAEcAYwBBAGQAQQBCAG8AQQBDAEEAQQBMAFEAQgBsAEEASABFAEEASQBBAEEAeQBBAEMAawBBAEkAQQBCADcAQQBDAEEAQQBkAEEAQgBvAEEASABJAEEAYgB3AEIAMwBBAEMAQQBBAEkAZwBCAHAAQQBHADQAQQBkAGcAQgBoAEEARwB3AEEAYQBRAEIAawBBAEMAQQBBAGMAQQBCAGgAQQBIAGsAQQBiAEEAQgB2AEEARwBFAEEAWgBBAEEAaQBBAEMAQQBBAGYAUQBBAEsAQQBGAE0AQQBaAFEAQgAwAEEAQwAwAEEAVgBnAEIAaABBAEgASQBBAGEAUQBCAGgAQQBHAEkAQQBiAEEAQgBsAEEAQwBBAEEATABRAEIATwBBAEcARQBBAGIAUQBCAGwAQQBDAEEAQQBhAGcAQgB6AEEARwA4AEEAYgBnAEIAZgBBAEgASQBBAFkAUQBCADMAQQBDAEEAQQBMAFEAQgBXAEEARwBFAEEAYgBBAEIAMQBBAEcAVQBBAEkAQQBBAGsAQQBIAE0AQQBjAEEAQgBzAEEARwBrAEEAZABBAEIAZgBBAEgAQQBBAFkAUQBCAHkAQQBIAFEAQQBjAHcAQgBiAEEARABFAEEAWABRAEEASwBBAEMAUQBBAFoAUQBCADQAQQBHAFUAQQBZAHcAQgBmAEEASABjAEEAYwBnAEIAaABBAEgAQQBBAGMAQQBCAGwAQQBIAEkAQQBJAEEAQQA5AEEAQwBBAEEAVwB3AEIAVABBAEcATQBBAGMAZwBCAHAAQQBIAEEAQQBkAEEAQgBDAEEARwB3AEEAYgB3AEIAagBBAEcAcwBBAFgAUQBBADYAQQBEAG8AQQBRAHcAQgB5AEEARwBVAEEAWQBRAEIAMABBAEcAVQBBAEsAQQBBAGsAQQBIAE0AQQBjAEEAQgBzAEEARwBrAEEAZABBAEIAZgBBAEgAQQBBAFkAUQBCAHkAQQBIAFEAQQBjAHcAQgBiAEEARABBAEEAWABRAEEAcABBAEEAbwBBAEoAZwBBAGsAQQBHAFUAQQBlAEEAQgBsAEEARwBNAEEAWAB3AEIAMwBBAEgASQBBAFkAUQBCAHcAQQBIAEEAQQBaAFEAQgB5AEEAQQA9AD0A'
<[redacted]> (0, b'{"changed":false,"invocation":{"module_args":{"_measure_subset":false,"gather_timeout":10,"gather_subset":["all"],"fact_path":null}},"ansible_facts":{"ansible_ip_addresses":[redacted],"ansible_windows_domain_role":"Member workstation","ansible_architecture2":"x86_64","ansible_user_gecos":"","ansible_distribution_major_version":"10","ansible_pagefiletotal_mb":4864,"ansible_os_name":"Microsoft Windows 10 Pro","ansible_system_description":"","ansible_machine_id":"[redacted]","ansible_product_serial":"[redacted]","ansible_system_vendor":"oVirt","gather_subset":["all"],"ansible_bios_version":"1.11.0-2.el7","ansible_user_id":"samueln","ansible_date_time":{"epoch_local":"1715183500.47678","epoch_int":1715201500,"date":"2024-05-08","second":"40","tz":"Central Standard Time","iso8601_micro":"2024-05-08T20:51:40.476780Z","iso8601_basic_short":"20240508T155140","minute":"51","day":"08","weekday":"Wednesday","iso8601":"2024-05-08T20:51:40Z","tz_offset":"-05:00","iso8601_basic":"20240508T155140476780","epoch":"1715201500.47678","weekday_number":"3","hour":"15","year":"2024","month":"05","time":"15:51:40","weeknumber":"18"},"ansible_user_dir":"C:\\\\Users\\\\samueln","ansible_processor_vcpus":2,"ansible_user_sid":"[redacted]","ansible_owner_contact":"","ansible_distribution":"Microsoft Windows 10 Pro","ansible_processor":["0","GenuineIntel","Intel Core i7 9xx (Nehalem Class Core i7)","1","GenuineIntel","Intel Core i7 9xx (Nehalem Class Core i7)"],"ansible_powershell_version":5,"ansible_uptime_seconds":432705,"module_setup":true,"ansible_interfaces":[{"dns_domain":"[redacted]","connection_name":"Ethernet","default_gateway":"[redacted]","mtu":1500,"interface_name":"Red Hat VirtIO Ethernet Adapter","ipv6":{},"macaddress":"[redacted]","speed":100000,"interface_index":5,"ipv4":{"address":"[redacted]","prefix":"16"}},{"dns_domain":null,"connection_name":"vEthernet (Default Switch)","default_gateway":null,"mtu":1500,"interface_name":"Hyper-V Virtual Ethernet Adapter","ipv6":{},"macaddress":"[redacted]","speed":10000,"interface_index":9,"ipv4":{"address":"[redacted]","prefix":"20"}}],"ansible_distribution_version":"10.0.19045.0","ansible_windows_domain_member":true,"ansible_swaptotal_mb":0,"ansible_hostname":"[redacted]","ansible_virtualization_role":"guest","ansible_memfree_mb":27610,"ansible_processor_count":1,"ansible_os_product_type":"workstation","ansible_netbios_name":"[redacted]","ansible_product_name":"oVirt Node","ansible_virtualization_type":"kvm","ansible_os_installation_type":"Client","ansible_lastboot":"2024-05-03 15:39:57Z","ansible_architecture":"64-bit","ansible_bios_date":"04/01/2014","ansible_processor_cores":2,"ansible_fqdn":"[redacted]","ansible_domain":"[redacted]","ansible_memtotal_mb":32768,"ansible_kernel":"10.0.19045.0","ansible_pagefilefree_mb":4763,"ansible_nodename":"[redacted]","ansible_os_family":"Windows","ansible_processor_threads_per_core":1,"ansible_owner_name":"[redacted]","ansible_env":{"HOME":"C:\\\\Users\\\\samueln","CVSIGNORE":"*.aps *.ncb *.opt *.plg *.pyc Debug Release","ComSpec":"C:\\\\Windows\\\\system32\\\\cmd.exe","DriverData":"C:\\\\Windows\\\\System32\\\\Drivers\\\\DriverData","ANSIBLE_ASYNC_DIR":"%USERPROFILE%\\\\.ansible_async","SystemRoot":"C:\\\\Windows","Path":"C:\\\\Program Files\\\\openjdk_11_0_14\\\\bin;C:\\\\Program Files\\\\Python310\\\\Scripts\\\\;C:\\\\Program Files\\\\Python310\\\\;C:\\\\Windows\\\\system32;C:\\\\Windows;C:\\\\Windows\\\\System32\\\\Wbem;C:\\\\Windows\\\\System32\\\\WindowsPowerShell\\\\v1.0\\\\;C:\\\\Windows\\\\System32\\\\OpenSSH\\\\;C:\\\\bin;C:\\\\Program Files\\\\Git\\\\cmd;C:\\\\Program Files\\\\nodejs\\\\;C:\\\\Program Files\\\\dotnet\\\\;C:\\\\Program Files (x86)\\\\WiX Toolset v3.11\\\\bin;C:\\\\Windows\\\\system32\\\\config\\\\systemprofile\\\\AppData\\\\Local\\\\Microsoft\\\\WindowsApps;C:\\\\Users\\\\samueln\\\\AppData\\\\Local\\\\Microsoft\\\\WindowsApps;C:\\\\Users\\\\samueln\\\\AppData\\\\Roaming\\\\npm;C:\\\\Users\\\\samueln\\\\.dotnet\\\\tools","OneDrive":"C:\\\\Users\\\\Administrator\\\\OneDrive","CommonProgramFiles(x86)":"C:\\\\Program Files (x86)\\\\Common Files","SSH_CONNECTION":"[redacted] 36558 [redacted] 22","ProgramW6432":"C:\\\\Program Files","HOMEPATH":"\\\\Users\\\\samueln","USERPROFILE":"C:\\\\Users\\\\samueln","OS":"Windows_NT","PSModulePath":"C:\\\\Users\\\\samueln\\\\Documents\\\\WindowsPowerShell\\\\Modules;C:\\\\Program Files\\\\WindowsPowerShell\\\\Modules;C:\\\\Windows\\\\system32\\\\WindowsPowerShell\\\\v1.0\\\\Modules","ProgramFiles(x86)":"C:\\\\Program Files (x86)","PSExecutionPolicyPreference":"Unrestricted","SHELL":"c:\\\\windows\\\\system32\\\\cmd.exe","SystemDrive":"C:","CVSROOT":"[redacted]","CommonProgramFiles":"C:\\\\Program Files\\\\Common Files","LOGNAME":"[redacted]","ProgramFiles":"C:\\\\Program Files","USERDOMAIN":"[redacted]","PATHEXT":".COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC;.PY;.PYW;.CPL","PROCESSOR_LEVEL":"6","WIX":"C:\\\\Program Files (x86)\\\\WiX Toolset v3.11","SSH_CLIENT":"[redacted] 36558 22","COMPUTERNAME":"[redacted]","USER":"[redacted]","NUMBER_OF_PROCESSORS":"2","TEMP":"C:\\\\Users\\\\samueln\\\\AppData\\\\Local\\\\Temp","LOCALAPPDATA":"C:\\\\Users\\\\samueln\\\\AppData\\\\Local","PUBLIC":"C:\\\\Users\\\\Public","PROCESSOR_REVISION":"1a03","APPDATA":"C:\\\\Users\\\\samueln\\\\AppData\\\\Roaming","ProgramData":"C:\\\\ProgramData","PROCESSOR_IDENTIFIER":"Intel64 Family 6 Model 26 Stepping 3, GenuineIntel","HOMEDRIVE":"C:","PROCESSOR_ARCHITECTURE":"AMD64","JAVA_HOME":"C:\\\\Program Files\\\\openjdk_11_0_14","CommonProgramW6432":"C:\\\\Program Files\\\\Common Files","PROMPT":"[redacted] $P$G","windir":"C:\\\\Windows","ALLUSERSPROFILE":"C:\\\\ProgramData","USERNAME":"samueln","c28fc6f98a2c44abbbd89d6a3037d0d9_POSIX_FD_STATE":"AAAAAAICAgA=","CVS_RSH":"C:\\\\Windows\\\\Sysnative\\\\OpenSSH\\\\ssh.exe","TMP":"C:\\\\Users\\\\samueln\\\\AppData\\\\Local\\\\Temp"},"ansible_windows_domain":"[redacted]","ansible_reboot_pending":true,"ansible_system":"Win32NT"}}\r\n\r\n', b'OpenSSH_8.0p1, OpenSSL 1.1.1k  FIPS 25 Mar 2021\r\ndebug1: Reading configuration data /home/samueln/.ssh/config\r\ndebug2: checking match for \'user [redacted]\' host [redacted] originally [redacted]\r\ndebug3: /home/samueln/.ssh/config line 1: not matched \'user "samueln"\' \r\ndebug2: match not found\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 52: Including file /etc/ssh/ssh_config.d/05-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf\r\ndebug2: checking match for \'final all\' host [redacted] originally [redacted]\r\ndebug3: /etc/ssh/ssh_config.d/05-redhat.conf line 3: not matched \'final\'\r\ndebug2: match not found\r\ndebug3: /etc/ssh/ssh_config.d/05-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-,gss-gex-sha1-,gss-group14-sha1-]\r\ndebug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1]\r\ndebug1: configuration requests final Match pass\r\ndebug1: re-parsing configuration\r\ndebug1: Reading configuration data /home/samueln/.ssh/config\r\ndebug2: checking match for \'user [redacted]\' host [redacted] originally [redacted]\r\ndebug3: /home/samueln/.ssh/config line 1: not matched \'user "samueln"\' \r\ndebug2: match not found\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 52: Including file /etc/ssh/ssh_config.d/05-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf\r\ndebug2: checking match for \'final all\' host [redacted] originally [redacted]\r\ndebug3: /etc/ssh/ssh_config.d/05-redhat.conf line 3: matched \'final\'\r\ndebug2: match found\r\ndebug3: /etc/ssh/ssh_config.d/05-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-,gss-gex-sha1-,gss-group14-sha1-]\r\ndebug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1]\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 2615185\r\ndebug3: mux_client_request_session: session request sent\r\n#< CLIXML\r\n<Objs Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04"><Obj S="progress" RefId="0"><TN RefId="0"><T>System.Management.Automation.PSCustomObject</T><T>System.Object</T></TN><MS><I64 N="SourceId">1</I64><PR N="Record"><AV>Preparing modules for first use.</AV><AI>0</AI><Nil /><PI>-1</PI><PC>-1</PC><T>Completed</T><SR>-1</SR><SD> </SD></PR></MS></Obj><Obj S="progress" RefId="1"><TNRef RefId="0" /><MS><I64 N="SourceId">2</I64><PR N="Record"><AV>Preparing modules for first use.</AV><AI>0</AI><Nil /><PI>-1</PI><PC>-1</PC><T>Completed</T><SR>-1</SR><SD> </SD></PR></MS></Obj></Objs>debug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n')
The full traceback is:
Traceback (most recent call last):
  File "/home/samueln/ansible-error-test/venv/lib64/python3.11/site-packages/ansible/executor/task_executor.py", line 165, in run
    res = self._execute()
          ^^^^^^^^^^^^^^^
  File "/home/samueln/ansible-error-test/venv/lib64/python3.11/site-packages/ansible/executor/task_executor.py", line 641, in _execute
    result = self._handler.run(task_vars=vars_copy)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/samueln/ansible-error-test/venv/lib64/python3.11/site-packages/ansible/plugins/action/gather_facts.py", line 139, in run
    poll_args = {'jid': jobs[module]['ansible_job_id'], '_async_dir': os.path.dirname(jobs[module]['results_file'])}
                        ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
KeyError: 'ansible_job_id'
fatal: [[redacted]]: FAILED! => {
    "msg": "Unexpected failure during module execution: 'ansible_job_id'",
    "stdout": ""
}

PLAY RECAP ***********************************************************************************************************************************************************************************
[redacted]              : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

Code of Conduct

• I agree to follow the Ansible Code of Conduct

[ansibot]

Files identified in the description:

• lib/ansible/plugins/action/gather_facts.py
• lib/ansible/modules/gather_facts.py

If these files are incorrect, please update the component name section of the description or use the component bot command.

[mkrizek]

cc @jborean93

[kirkzhangtech]

may i know if windows11 also hit the same error ?

[jborean93]

Thanks for the bug report and detailed reproducer.

The problem here stems from the change made in #80597 that made parallel actually run in parallel (parallel is true when multiple modules are specified). The gather_facts action plugin will run each fact module using async but currently it does this by setting the task's async value to 0 and telling the module execution wrapper to wrap it in Async. For PowerShell modules it will only do things in async when the async value is greater than 0 which means the modules aren't run in async and the async return values the action plugin expects to be there are not.

A workaround this issue is to explicitly disable parallel gathering to go back to the old behaviour by setting ansible_facts_parallel: false as a var for the affected hosts. I'm looking into how we can fix this internally for future versions so this workaround is no longer needed.

[jborean93]

I've opened #83275 which should solve this problem. Will need to talk internally around the best way to backport this change as currently it relies on new features being added which we don't backport.

[samueln-w]

may i know if windows11 also hit the same error ?

@kirkzhangtech Given jborean's explanation, it seems to be related specifically to PowerShell rather than to Windows itself, so I expect the results would be the same with a Windows 11 machine.

I haven't yet found a way to use non-PowerShell modules on Windows (the ansiballz framework seems to fall over when a Windows machine is the target). But I suppose such a thing would circumvent this bug just as well if it could be made to work on Windows 10 or 11.

[samueln-w]

Thanks for the bug report and detailed reproducer.

You're welcome!

The problem here stems from the change made in #80597 that made parallel actually run in parallel (parallel is true when multiple modules are specified). The gather_facts action plugin will run each fact module using async but currently it does this by setting the task's async value to 0 and telling the module execution wrapper to wrap it in Async. For PowerShell modules it will only do things in async when the async value is greater than 0 which means the modules aren't run in async and the async return values the action plugin expects to be there are not.

A workaround this issue is to explicitly disable parallel gathering to go back to the old behaviour by setting ansible_facts_parallel: false as a var for the affected hosts. I'm looking into how we can fix this internally for future versions so this workaround is no longer needed.

Ahh, I see. Thanks for the explanation and workaround! It does appear to work.

And, just in case anyone else stumbles across this report and has the same question I did, explicit calls to the ansible.builtin.gather_facts module appear to respect ansible_facts_parallel, as well. No need to specify parallel: False as a module parameter if ansible_facts_parallel is already False. However, this appears to contradict the module documentation, which only states:

By default it will be true if more than one fact module is used.

But, given that ansible_facts_parallel doesn't seem to be documented anywhere, I'm not sure what the intended behavior is, or whether ansible_facts_parallel can be relied upon to affect explicit calls to gather_facts for the foreseeable future.

[bcoca]

it is intended, i'll make sure to update docs

[samueln-w]

Great, thank you!