Ansible Provisioner Missing Pywinrm Dependency in venv ("winrm or requests is not installed: No module named 'winrm'")

[zkhcohen]

Overview of the Issue

When attempting to run an Ansible playbook defined in the Ansible Provisioner block, I receive the error:
"winrm or requests is not installed: No module named 'winrm'"

The specific issue appears to be that the 'ansible python module path' is located in the pipx env that Ansible is automatically installed to, but pywinrm wasn't installed along with it:
azure-arm.image: ansible python module location = /opt/pipx/venvs/ansible-core/lib/python3.8/site-packages/ansible

Reproduction Steps

This deployment is taking place in an Azure pipeline with an Ubuntu container, deploying Windows 10 hosts to Azure. The playbook is intended to target those hosts.

Run a Packer build (note: packer init was run first) with the following configuration and Ansible provisioner using WinRM.

Packer version

Packer Version: 1.7.4 x64

Simplified Packer Buildfile

image_build.pkr.hcl:

packer {
  required_version = ">= 1.7.4"
  required_plugins {
    ansible = {
      source  = "github.com/hashicorp/ansible"
      version = ">=1.0.0"
    }
  }
}

source "azure-arm" "image" {
  async_resourcegroup_delete        = "${var.async_resourcegroup_delete}"
  client_id                         = "${var.client_id}"
  client_secret                     = "${var.client_secret}"
  communicator                      = "${var.communicator}"
  image_offer                       = "${var.image_offer}"
  image_publisher                   = "${var.image_publisher}"
  image_sku                         = "${var.image_sku}"
  location                          = "${var.location}"
  managed_image_name                = "${var.managed_image_name}"
  managed_image_resource_group_name = "${var.managed_image_resource_group_name}"
  os_type                           = "${var.os_type}"
  shared_image_gallery_timeout      = "${var.shared_image_gallery_timeout}"
  subscription_id                   = "${var.subscription_id}"
  tenant_id                         = "${var.tenant_id}"
  vm_size                           = "${var.vm_size}"
  winrm_insecure                    = "${var.winrm_insecure}"
  winrm_timeout                     = "${var.winrm_timeout}"
  winrm_use_ssl                     = "${var.winrm_use_ssl}"
  winrm_username                    = "${var.winrm_username}"

  shared_image_gallery_destination {
    subscription         = "${var.subscription_id}"
    resource_group       = "${var.managed_image_resource_group_name}"
    gallery_name         = "${var.gallery_name}"
    image_name           = "${var.managed_image_name}"
    image_version        = "${var.image_version}"
    replication_regions  = ["${var.location}"]
    storage_account_type = "${var.storage_account_type}"
  }
}

build {
    sources = [
      "source.azure-arm.image",
      "source.azure-arm.otherimage"
    ]

  provisioner "powershell" {
      scripts = var.powershell_scripts
  }

  provisioner "ansible" {
    only = [
      "azure-arm.image"
    ]
    playbook_file   = "./packer/ansible/playbooks/playbook-windows.yml"
    user            = "${var.winrm_username}"
    use_proxy       = false
    extra_arguments = [
      "-vvvvvv",
      "-e", "ansible_winrm_server_cert_validation=ignore",
    ]
  }
}

image_build.pkrvars.hcl:

powershell_scripts = [
    "./packer/scripts/ConfigureRemotingForAnsible.ps1"
]

Ansible Playbook

playbook-windows.yml:

---
- name: This is an example
  hosts: all

  tasks:
    - name: Ping Google
      ansible.windows.win_shell: |
        ping google.com

Operating system and Environment details

OS: ubuntu-20.04 LTS x64
Python Version: 3.8.10
Ansible Version: ansible-core 2.11.3 (installed via packer init)

Log Fragments and crash.log files

2021/08/22 18:12:28 [INFO] (telemetry) Starting provisioner ansible
2021/08/22 18:12:29 packer-plugin-ansible_v1.0.0_x5.0_linux_amd64 plugin: 2021/08/22 18:12:29 ansible-playbook version: 2.11.3
==> azure-arm.image: Provisioning with Ansible...
    azure-arm.image: 	Using WinRM Password from Packer communicator...
    azure-arm.image: Not using Proxy adapter for Ansible run:
    azure-arm.image: 	Using WinRM Password from Packer communicator...
2021/08/22 18:12:29 packer-plugin-ansible_v1.0.0_x5.0_linux_amd64 plugin: 2021/08/22 18:12:29 Creating inventory file for Ansible run...
==> azure-arm.image: Executing Ansible: ansible-playbook -e packer_build_name="image" -e packer_builder_type=azure-arm -vvvvvv -e ansible_winrm_server_cert_validation=ignore -e ansible_password=***** -i /tmp/packer-provisioner-ansible828740952 /home/vsts/work/1/s/packer/ansible/playbooks/playbook-windows.yml
    azure-arm.image: ansible-playbook [core 2.11.3]
    azure-arm.image:   config file = None
    azure-arm.image:   configured module search path = ['/home/vsts/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
    azure-arm.image:   ansible python module location = /opt/pipx/venvs/ansible-core/lib/python3.8/site-packages/ansible
    azure-arm.image:   ansible collection location = /home/vsts/.ansible/collections:/usr/share/ansible/collections
    azure-arm.image:   executable location = /opt/pipx_bin/ansible-playbook
    azure-arm.image:   python version = 3.8.10 (default, Jun  2 2021, 10:49:15) [GCC 9.4.0]
    azure-arm.image:   jinja version = 3.0.1
    azure-arm.image:   libyaml = True
    azure-arm.image: No config file found; using defaults
    azure-arm.image: setting up inventory plugins
    azure-arm.image: host_list declined parsing /tmp/packer-provisioner-ansible828740952 as it did not pass its verify_file() method
    azure-arm.image: script declined parsing /tmp/packer-provisioner-ansible828740952 as it did not pass its verify_file() method
    azure-arm.image: auto declined parsing /tmp/packer-provisioner-ansible828740952 as it did not pass its verify_file() method
    azure-arm.image: Parsed /tmp/packer-provisioner-ansible828740952 inventory source with ini plugin
    azure-arm.image: Loading collection ansible.windows from /opt/pipx/venvs/ansible-core/lib/python3.8/site-packages/ansible_collections/ansible/windows
    azure-arm.image: Loading callback plugin default of type stdout, v2.0 from /opt/pipx/venvs/ansible-core/lib/python3.8/site-packages/ansible/plugins/callback/default.py
    azure-arm.image: Attempting to use 'default' callback.
    azure-arm.image: Skipping callback 'default', as we already have a stdout callback.
    azure-arm.image: Attempting to use 'junit' callback.
    azure-arm.image: Attempting to use 'minimal' callback.
    azure-arm.image: Skipping callback 'minimal', as we already have a stdout callback.
    azure-arm.image: Attempting to use 'oneline' callback.
    azure-arm.image: Skipping callback 'oneline', as we already have a stdout callback.
    azure-arm.image: Attempting to use 'tree' callback.
    azure-arm.image:
    azure-arm.image: PLAYBOOK: playbook-windows.yml *************************************************
    azure-arm.image: Positional arguments: /home/vsts/work/1/s/packer/ansible/playbooks/playbook-windows.yml
    azure-arm.image: verbosity: 6
    azure-arm.image: connection: smart
    azure-arm.image: timeout: 10
    azure-arm.image: become_method: sudo
    azure-arm.image: tags: ('all',)
    azure-arm.image: inventory: ('/tmp/packer-provisioner-ansible828740952',)
    azure-arm.image: extra_vars: ('packer_build_name="image"', 'packer_builder_type=azure-arm', 'ansible_winrm_server_cert_validation=ignore', 'ansible_password=REMOVED')
    azure-arm.image: forks: 5
    azure-arm.image: 1 plays in /home/vsts/work/1/s/packer/ansible/playbooks/playbook-windows.yml
    azure-arm.image:
    azure-arm.image: PLAY [This is an example] ******************************************************
    azure-arm.image:
    azure-arm.image: TASK [Gathering Facts] *********************************************************
    azure-arm.image: task path: /home/vsts/work/1/s/packer/ansible/playbooks/playbook-windows.yml:2
    azure-arm.image: redirecting (type: modules) ansible.builtin.setup to ansible.windows.setup
    azure-arm.image: Using module file /opt/pipx/venvs/ansible-core/lib/python3.8/site-packages/ansible_collections/ansible/windows/plugins/modules/setup.ps1
    azure-arm.image: Pipelining is enabled.
    azure-arm.image: fatal: [default]: FAILED! => {
    azure-arm.image:     "msg": "winrm or requests is not installed: No module named 'winrm'"
    azure-arm.image: }
    azure-arm.image:
    azure-arm.image: PLAY RECAP *********************************************************************
    azure-arm.image: default                    : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0
    azure-arm.image:
==> azure-arm.image: Deleting individual resources ...

[zkhcohen]

I can think of a couple of complicated workarounds for this, but dealing with wrappers and custom virtual environments doesn't seem like a good solution when the fundamental issue appears to be a missing dependency in the ansible provisioner package itself. The official docs for this provisioner certainly make it seem like this is possible out of the box, but maybe they're expecting a Windows to Windows deployment rather than Unix to Windows...

Additionally, I've been unable to find any reference to the /opt/pipx/venv path in any of the related repos or issues. I'm not sure if this is a new feature or potentially an issue with Azure Pipelines. If the module was using the standard system Python module path, I could simply install pywinrm to the Ubuntu container, but in its current state the virtual environment precludes that.

[zkhcohen]

Okay, technically the following workaround fixes the issues, but it would still be nice to get a 'yes' or 'no' as to whether this is usually necessary, and if so, please add a clearer explanation / use-case to the docs:

Added to Azure DevOps pipeline:

  - script: |
     chmod +x ./packer/ansible.sh && \
     python3 -m venv /tmp/venv && \
     source /tmp/venv/bin/activate && \
     python3 -m pip install wheel pywinrm ansible && \
     deactivate
    displayName: 'Add exec bit to ansible.sh and install Ansible and pywinrm to venv'

ansible.sh:

#!/bin/bash
source /tmp/venv/bin/activate && ANSIBLE_FORCE_COLOR=1 PYTHONUNBUFFERED=1 /tmp/venv/bin/ansible-playbook "$@"

Modified image_build.pkr.hcl:

provisioner "ansible" {
    only = [
      "azure-arm.image"
    ]
    command         = "./packer/ansible.sh"
    playbook_file   = "./packer/ansible/playbooks/playbook-windows.yml"
    user            = "${var.winrm_username}"
    use_proxy       = false
    extra_arguments = [
      "-vvvvvv",
      "-e", "ansible_winrm_server_cert_validation=ignore"
    ]
  }

#7569 was helpful in confirming that using your own virtualenv was the appropriate workaround.