Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to import the required Python library (ansible[azure] (azure >= 2.0.0) #1463

Open
ericcames opened this issue Feb 23, 2024 · 33 comments
Labels
dependencies Pull requests that update a dependency file medium_priority Medium priority question Further information is requested work in In trying to solve, or in working with contributors

Comments

@ericcames
Copy link

SUMMARY

Trying to use this module azure.azcollection.azure_rm_account_info with ansible automation platform 2.3 and we are getting the failure list in the title

ISSUE TYPE
  • Bug Report
COMPONENT NAME

azure.azcollection.azure_rm_account_info

ANSIBLE VERSION
[awx@aap ~]$ ansible --version
ansible [core 2.14.11]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/var/lib/awx/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.9/site-packages/ansible
  ansible collection location = /var/lib/awx/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.9.16 (main, Sep 22 2023, 17:57:55) [GCC 8.5.0 20210514 (Red Hat 8.5.0-18)] (/usr/bin/python3.9)
  jinja version = 3.1.2
  libyaml = True

COLLECTION VERSION
azure.azcollection
version: 1.18.1
and
version: latest
CONFIGURATION
[defaults]
action_warnings = false
callbacks_enabled = profile_tasks
show_custom_stats = true
# inject_facts_as_vars = no
deprecation_warnings = false
forks = 10
gathering = explicit
roles_path = ./roles/
collections_path = ./collections/

[privilege_escalation]
become = false
become_method = sudo
become_user = root

[persistent_connection]
command_timeout = 30
OS / ENVIRONMENT

azure subscription account
AAP 2.3 on RHEL 8

STEPS TO REPRODUCE

Set up a Microsoft Azure Resource Manager credential in AAP
Create a template to run the playbook listed below
Use this execution environment: quay.io/seanlee/ee-azure:1.12.0

---
- name: Get account info from Azure
  hosts: localhost
  connection: local

  tasks:

    - name: Get facts for current logged in user
      azure.azcollection.azure_rm_account_info:
...
EXPECTED RESULTS

I expect the playbook to pull back the account info.

ACTUAL RESULTS
{
  "exception": "Traceback (most recent call last):\n  File \"/tmp/ansible_azure.azcollection.azure_rm_account_info_payload_lev0wrek/ansible_azure.azcollection.azure_rm_account_info_payload.zip/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py\", line 256, in <module>\n    from azure.storage.blob import BlobServiceClient\nImportError: cannot import name 'BlobServiceClient' from 'azure.storage.blob' (/usr/local/lib/python3.8/site-packages/azure/storage/blob/__init__.py)\n",
  "msg": "Failed to import the required Python library (ansible[azure] (azure >= 2.0.0)) on aee9dfd4ba17's Python /usr/bin/python3. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter",
  "invocation": {
    "module_args": {
      "auth_source": "auto",
      "cloud_environment": "AzureCloud",
      "api_profile": "latest",
      "profile": null,
      "subscription_id": null,
      "client_id": null,
      "secret": null,
      "tenant": null,
      "ad_user": null,
      "password": null,
      "cert_validation_mode": null,
      "adfs_authority_url": null,
      "log_mode": null,
      "log_path": null,
      "x509_certificate_path": null,
      "thumbprint": null
    }
  },
  "_ansible_no_log": false,
  "changed": false
}
@ericcames
Copy link
Author

@thisguyshouldworkforus I created an issue for what we are seeing.

@Fred-sun Fred-sun added medium_priority Medium priority work in In trying to solve, or in working with contributors labels Feb 26, 2024
@Fred-sun
Copy link
Collaborator

@ericcames After you installed aze.azcollection, did you install the related dependency files? Reinstall all dependent files and try again. Thank you!

@ericcames
Copy link
Author

@ericcames After you installed aze.azcollection, did you install the related dependency files? Reinstall all dependent files and try again. Thank you!

@Fred-sun thank you for the feedback. When I read the dependencies section of the documentation is says there are none. I am installing the collections via the collections directory and a requirements file.
image

Not sure if you are redhat employee or not if you are you can find me via eric.ames@redhat.com. I am working with a customer to prove out that this collection works in their environment. We are looking for ideas on how to get this collection working. Thanks for your feedback!

@A-Beck
Copy link

A-Beck commented Feb 26, 2024

I think the console is not loading dependencies correctly. You can see the python dependencies in the repo for the collection: https://github.com/ansible-collections/azure/blob/dev/requirements-azure.txt

Since you're using AAP, you should have access to the supported EE that includes these dependencies, IIRC

@skertz
Copy link

skertz commented Feb 26, 2024

Tried the requirements-azure.txt with python3.6 and 3.7 both report "msgraph-sdk==1.0.0 not found".
Using python3.8 and python 3.11 gets through this. However, running a playbook reports errors, ModuleNotFoundError: No module named 'azure.storage.cloudstorageaccount'. Tried on fresh install of rh7 and rh8 machines.

@ericcames
Copy link
Author

@alexander-aps tagging

@Fred-sun
Copy link
Collaborator

Tried the requirements-azure.txt with python3.6 and 3.7 both report "msgraph-sdk==1.0.0 not found". Using python3.8 and python 3.11 gets through this. However, running a playbook reports errors, ModuleNotFoundError: No module named 'azure.storage.cloudstorageaccount'. Tried on fresh install of rh7 and rh8 machines.

@skertz This error is strange because we did not ask to install 'azure-storage'. Could you please provide your detailed execution environment and error information? Thank you!

@Fred-sun
Copy link
Collaborator

@ericcames Can you provide 'pip3 freeze' information and python version information? Thank you!

@Fred-sun Fred-sun added the dependencies Pull requests that update a dependency file label Feb 27, 2024
@ericcames
Copy link
Author

@Fred-sun are you wanting this from the container or the controller?? This is the container we are using registry.redhat.io/ansible-automation-platform-23/ee-29-rhel8:latest

@ericcames
Copy link
Author

@Fred-sun we created quay.io/zigfreed/ee-azure:latest and have validated that it works as expected with three other asure collections. ---

  • name: Get info Azure Resource Group
    hosts: localhost
    connection: local

    tasks:

    • name: Get all resource Groups info
      register: resourcegroupinfo
      azure.azcollection.azure_rm_resourcegroup_info:

    • name: Print resourcegroupinfo
      ansible.builtin.debug:
      var: resourcegroupinfo
      ...


  • name: Get info Azure subscription_info
    hosts: localhost
    connection: local

    tasks:

    • name: Get facts for all subscriptions, including ones that are disabled.
      register: subscription_info
      azure.azcollection.azure_rm_subscription_info:
      all: true

    • name: Print subscription_info
      ansible.builtin.debug:
      var: subscription_info
      ...


  • name: Create Azure Resource Group
    hosts: localhost
    connection: local
    vars:
    az_resource_group_name: 'woopdidoo'
    location: 'westus'
    customer: 'mickey.mouse'

    tasks:

    • name: Create a resource group
      azure.azcollection.azure_rm_resourcegroup:
      name: "{{ az_resource_group_name }}"
      location: "{{ location }}"
      state: present
      tags:
      created: ansbile
      delete: never
      demo: "{{ customer }}"
      ...

@ericcames
Copy link
Author

This one does not work: ---

  • name: Get account info from Azure
    hosts: localhost
    connection: local

    tasks:

    • name: Get facts for current logged in user
      azure.azcollection.azure_rm_account_info:
      ...

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: error: MainError(additional_data={}, code='BadRequest', details=None, inner_error=InnerError(additional_data={'date': DateTime(2024, 2, 27, 12, 7, 25, tzinfo=Timezone('UTC'))}, client_request_id='24c090c3-8b51-4d10-bba2-d93ba2af6560', date=None, odata_type=None, request_id='f5d97fd3-a0cd-4f9e-8e69-02c50dd1c1b5'), message='/me request is only valid with delegated authentication flow.', target=None)
fatal: [localhost]: FAILED! => {"ansible_facts": {"discovered_interpreter_python": "/usr/bin/python3.9"}, "changed": false, "module_stderr": "Traceback (most recent call last):\n File "/home/runner/.ansible/tmp/ansible-tmp-1709035643.134935-26-217118459765709/AnsiballZ_azure_rm_account_info.py", line 107, in \n _ansiballz_main()\n File "/home/runner/.ansible/tmp/ansible-tmp-1709035643.134935-26-217118459765709/AnsiballZ_azure_rm_account_info.py", line 99, in _ansiballz_main\n …

@skertz
Copy link

skertz commented Feb 27, 2024

@Fred-sun

Redhat 8

ansible [core 2.15.3]
config file = /etc/ansible/ansible.cfg
configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python3.11/site-packages/ansible
ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
executable location = /bin/ansible
python version = 3.11.5 (main, Sep 22 2023, 15:34:29) [GCC 8.5.0 20210514 (Red Hat 8.5.0-20)] (/usr/bin/python3.11)
jinja version = 3.1.2
libyaml = True

Install:

  1. ansible-galaxy collection install azure.azcollection
  2. python3 -m pip install -r ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt

(python3 is 3.6)
ERROR: Could not find a version that satisfies the requirement msgraph-sdk==1.0.0 (from versions: 1.0.0a0, 1.0.0a1, 1.0.0a2, 1.0.0a3, 1.0.0a4, 1.0.0a7, 1.0.0a8, 1.0.0a9, 1.0.0a10, 1.0.0a11, 1.0.0a12, 1.0.0a13)
ERROR: No matching distribution found for msgraph-sdk==1.0.0

  1. python3.11 -m pip install -r ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt

SUCCEEDS, but then:

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ModuleNotFoundError: No module named 'azure.storage.cloudstorageaccount'
fatal: [pd-ansible-3.production.iknowmed.com]: FAILED! => changed=false
msg: Failed to import the required Python library (ansible[azure] (azure >= 2.0.0)) on pd-ansible-3's Python /usr/bin/python3.11. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter

Isn't azure-mgmt-storage and azure-storage-blob defined in the requirements?

@Fred-sun
Copy link
Collaborator

@skertz There are definitions, all of our requirements files are in ~/ requirements-aze.txt. Thank you!

packaging
requests[security]
xmltodict
msgraph-sdk==1.0.0
azure-cli-core==2.34.0
azure-common==1.1.11
azure-identity==1.14.0
azure-mgmt-authorization==2.0.0
azure-mgmt-apimanagement==3.0.0
azure-mgmt-batch==16.2.0
azure-mgmt-cdn==11.0.0
azure-mgmt-compute==26.1.0
azure-mgmt-containerinstance==9.0.0
azure-mgmt-core==1.3.0
azure-mgmt-containerregistry==9.1.0
azure-containerregistry==1.1.0
azure-mgmt-containerservice==20.0.0
azure-mgmt-datalake-store==1.0.0
azure-mgmt-datafactory==2.0.0
azure-mgmt-dns==8.0.0
azure-mgmt-marketplaceordering==1.1.0
azure-mgmt-monitor==3.0.0
azure-mgmt-managedservices==6.0.0
azure-mgmt-managementgroups==1.0.0
azure-mgmt-network==19.1.0
azure-mgmt-nspkg==2.0.0
azure-mgmt-privatedns==1.0.0
azure-mgmt-redis==13.0.0
azure-mgmt-resource==21.1.0
azure-mgmt-rdbms==10.0.0
azure-mgmt-search==8.0.0
azure-mgmt-servicebus==7.1.0
azure-mgmt-sql==3.0.1
azure-mgmt-storage==19.0.0
azure-mgmt-trafficmanager==1.0.0b1
azure-mgmt-web==6.1.0
azure-nspkg==2.0.0
azure-storage-blob==12.11.0
azure-core==1.28.0
azure-keyvault==4.2.0
azure-mgmt-keyvault==10.0.0
azure-mgmt-cosmosdb==6.4.0
azure-mgmt-hdinsight==9.0.0
azure-mgmt-devtestlabs==9.0.0
azure-mgmt-loganalytics==12.0.0
azure-mgmt-automation==1.0.0
azure-mgmt-iothub==2.2.0
azure-iot-hub==2.6.1
azure-mgmt-recoveryservices==2.0.0
azure-mgmt-recoveryservicesbackup==3.0.0
azure-mgmt-notificationhubs==7.0.0
azure-mgmt-eventhub==10.1.0

@ericcames
Copy link
Author

ericcames commented Feb 29, 2024

@Fred-sun we built these into the execution environment we are using for testing.

quay.io/zigfreed/ee-azure:latest

@daviewales
Copy link

I'm getting this exact error trying to use azure.azcollection.azure_rm_keyvaultsecret_info.

I have installed using pipx as follows:

pipx install --include-deps ansible
pipx runpip ansible install -r ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt

My ansible configuration:

❯ ansible --version
ansible [core 2.16.4]
  config file = /home/dwales/.ansible.cfg
  configured module search path = ['/home/dwales/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/dwales/.local/pipx/venvs/ansible/lib/python3.10/site-packages/ansible
  ansible collection location = /home/dwales/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/dwales/.local/bin/ansible
  python version = 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] (/home/dwales/.local/pipx/venvs/ansible/bin/python)
  jinja version = 3.1.3
  libyaml = True

Error message:

The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_azure.azcollection.azure_rm_keyvaultsecret_info_payload_x0pvcnz2/ansible_azure.azcollection.azure_rm_keyvaultsecret_info_payload.zip/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 240, in <module>
    from azure.mgmt.core.tools import parse_resource_id, resource_id, is_valid_resource_id
ModuleNotFoundError: No module named 'azure'
fatal: [1.2.3.4]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "ad_user": null,
            "adfs_authority_url": null,
            "api_profile": "latest",
            "auth_source": "auto",
            "cert_validation_mode": null,
            "client_id": null,
            "cloud_environment": "AzureCloud",
            "log_mode": null,
            "log_path": null,
            "name": "secret",
            "password": null,
            "profile": null,
            "secret": null,
            "show_deleted_secret": false,
            "subscription_id": null,
            "tags": null,
            "tenant": null,
            "thumbprint": null,
            "vault_uri": "https://example.vault.azure.net",
            "version": "current",
            "x509_certificate_path": null
        }
    },
    "msg": "Failed to import the required Python library (ansible[azure] (azure >= 2.0.0)) on remote-vm's Python /usr/bin/python3. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter"
}

I think the problem is that it's trying to import from azure.mgmt.core.tools on the remote server, rather than the local machine where I have installed the requirements.txt. If I add delegate_to: 127.0.0.1, it works perfectly.

@Fred-sun
Copy link
Collaborator

Fred-sun commented Mar 7, 2024

@ericcames Sorry, there is an issue that has not been clarified, graphrbac migrated to msgraph, supported python version >=3.8. Please use aze.azcollection >=2.0.0 to run only in an environment higher than python3.8. Thank you!

@Fred-sun Fred-sun added the question Further information is requested label Mar 7, 2024
@ericcames
Copy link
Author

@Fred-sun Thank you!

@Fred-sun
Copy link
Collaborator

@ericcames Are you trying the new version? Has the problem been solved?

@nimbusparis
Copy link

I got the same issue and after many try/error, I succeed after manually install python3, ansible on the target computer. Then, I also installed the azure collection and run pip3 install .../requirements/txt all manipulations had been done with root

@Fred-sun
Copy link
Collaborator

@nimbusparis What does your error message look like? Can you provide a detailed log? Thank you!

@nimbusparis
Copy link

@nimbusparis What does your error message look like? Can you provide a detailed log? Thank you!

Well, now, I don't have the issue anymore, I can try to uninstall the collection and python module, but I hope it wouldn't break anything! 😆
Also, I can try to install the required collection/python module on the fly in my playbook.
In fact, I got the exact same error log as @ericcames at the begining of this thread.

@nimbusparis
Copy link

Here the vvvv log, hops it could help.

<51.255.204.38> ESTABLISH SSH CONNECTION FOR USER: user
<51.255.204.38> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="user"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/9afa1bc5be"' 51.255.204.38 '/bin/sh -c '"'"'echo ~user && sleep 0'"'"''
<51.255.204.38> (0, b'/home/user\n', b"OpenSSH_9.6p1 Ubuntu-3ubuntu13, OpenSSL 3.0.13 30 Jan 2024\r\ndebug1: Reading configuration data /root/.ssh/config\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname 51.255.204.38 is address\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'\r\ndebug1: auto-mux: Trying existing master at '/root/.ansible/cp/9afa1bc5be'\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 69017\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet_timeout: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n")
<51.255.204.38> ESTABLISH SSH CONNECTION FOR USER: user
<51.255.204.38> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="user"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/9afa1bc5be"' 51.255.204.38 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /home/user/.ansible/tmp `"&& mkdir "` echo /home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822 `" && echo ansible-tmp-1715760217.3001661-69163-31083769795822="` echo /home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822 `" ) && sleep 0'"'"''
<51.255.204.38> (0, b'ansible-tmp-1715760217.3001661-69163-31083769795822=/home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822\n', b"OpenSSH_9.6p1 Ubuntu-3ubuntu13, OpenSSL 3.0.13 30 Jan 2024\r\ndebug1: Reading configuration data /root/.ssh/config\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname 51.255.204.38 is address\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'\r\ndebug1: auto-mux: Trying existing master at '/root/.ansible/cp/9afa1bc5be'\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 69017\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet_timeout: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n")
Using module file /usr/lib/python3/dist-packages/ansible_collections/azure/azcollection/plugins/modules/azure_rm_storageblob.py
<51.255.204.38> PUT /root/.ansible/tmp/ansible-local-69127aj5xlmin/tmp1a6bemrd TO /home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822/AnsiballZ_azure_rm_storageblob.py
<51.255.204.38> SSH: EXEC sftp -b - -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="user"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/9afa1bc5be"' '[51.255.204.38]'
<51.255.204.38> (0, b'sftp> put /root/.ansible/tmp/ansible-local-69127aj5xlmin/tmp1a6bemrd /home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822/AnsiballZ_azure_rm_storageblob.py\n', b'OpenSSH_9.6p1 Ubuntu-3ubuntu13, OpenSSL 3.0.13 30 Jan 2024\r\ndebug1: Reading configuration data /root/.ssh/config\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname 51.255.204.38 is address\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts\' -> \'/root/.ssh/known_hosts\'\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts2\' -> \'/root/.ssh/known_hosts2\'\r\ndebug1: auto-mux: Trying existing master at \'/root/.ansible/cp/9afa1bc5be\'\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 69017\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug2: Remote version: 3\r\ndebug2: Server supports extension "posix-rename@openssh.com" revision 1\r\ndebug2: Server supports extension "statvfs@openssh.com" revision 2\r\ndebug2: Server supports extension "fstatvfs@openssh.com" revision 2\r\ndebug2: Server supports extension "hardlink@openssh.com" revision 1\r\ndebug2: Server supports extension "fsync@openssh.com" revision 1\r\ndebug2: Server supports extension "lsetstat@openssh.com" revision 1\r\ndebug2: Server supports extension "limits@openssh.com" revision 1\r\ndebug2: Server supports extension "expand-path@openssh.com" revision 1\r\ndebug3: Sent message limits@openssh.com I:1\r\ndebug3: Received limits reply T:201 I:1\r\ndebug3: server upload/download buffer sizes 261120 / 261120; using 261120 / 261120\r\ndebug3: server handle limit 1019; using 64\r\ndebug2: Sending SSH2_FXP_REALPATH "."\r\ndebug3: Sent message fd 3 T:16 I:2\r\ndebug3: SSH2_FXP_REALPATH . -> /home/user\r\ndebug3: Looking up /root/.ansible/tmp/ansible-local-69127aj5xlmin/tmp1a6bemrd\r\ndebug2: Sending SSH2_FXP_STAT "/home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822/AnsiballZ_azure_rm_storageblob.py"\r\ndebug3: Sent message fd 3 T:17 I:3\r\ndebug1: stat remote: No such file or directory\r\ndebug2: sftp_upload: upload local "/root/.ansible/tmp/ansible-local-69127aj5xlmin/tmp1a6bemrd" to remote "/home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822/AnsiballZ_azure_rm_storageblob.py"\r\ndebug2: Sending SSH2_FXP_OPEN "/home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822/AnsiballZ_azure_rm_storageblob.py"\r\ndebug3: Sent dest message SSH2_FXP_OPEN I:4 P:/home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822/AnsiballZ_azure_rm_storageblob.py M:0x001a\r\ndebug3: Sent message SSH2_FXP_WRITE I:6 O:0 S:154044\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 6 154044 bytes at 0\r\ndebug3: Sent message SSH2_FXP_CLOSE I:5\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: mux_client_read_packet_timeout: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n')
<51.255.204.38> ESTABLISH SSH CONNECTION FOR USER: user
<51.255.204.38> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="user"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/9afa1bc5be"' 51.255.204.38 '/bin/sh -c '"'"'chmod u+x /home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822/ /home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822/AnsiballZ_azure_rm_storageblob.py && sleep 0'"'"''
<51.255.204.38> (0, b'', b"OpenSSH_9.6p1 Ubuntu-3ubuntu13, OpenSSL 3.0.13 30 Jan 2024\r\ndebug1: Reading configuration data /root/.ssh/config\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname 51.255.204.38 is address\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'\r\ndebug1: auto-mux: Trying existing master at '/root/.ansible/cp/9afa1bc5be'\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 69017\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet_timeout: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n")
<51.255.204.38> ESTABLISH SSH CONNECTION FOR USER: user
<51.255.204.38> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="user"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/9afa1bc5be"' -tt 51.255.204.38 '/bin/sh -c '"'"'sudo -H -S -p "[sudo via ansible, key=mnwrxfhewwhoddyzirtxjtldezxfwwhl] password:" -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-mnwrxfhewwhoddyzirtxjtldezxfwwhl ; /usr/bin/python3 /home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822/AnsiballZ_azure_rm_storageblob.py'"'"'"'"'"'"'"'"' && sleep 0'"'"''
Escalation succeeded
<51.255.204.38> (1, b'\r\n\r\n{"exception": "Traceback (most recent call last):\\n  File \\"/tmp/ansible_azure.azcollection.azure_rm_storageblob_payload_1q1m5w5f/ansible_azure.azcollection.azure_rm_storageblob_payload.zip/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py\\", line 223, in <module>\\n    from msrestazure.azure_active_directory import AADTokenCredentials\\n  File \\"/usr/local/lib/python3.10/dist-packages/msrestazure/__init__.py\\", line 28, in <module>\\n    from .azure_configuration import AzureConfiguration\\n  File \\"/usr/local/lib/python3.10/dist-packages/msrestazure/azure_configuration.py\\", line 34, in <module>\\n    from msrest import Configuration\\n  File \\"/usr/local/lib/python3.10/dist-packages/msrest/__init__.py\\", line 28, in <module>\\n    from .configuration import Configuration\\n  File \\"/usr/local/lib/python3.10/dist-packages/msrest/configuration.py\\", line 38, in <module>\\n    from .universal_http.requests import (\\n  File \\"/usr/local/lib/python3.10/dist-packages/msrest/universal_http/__init__.py\\", line 53, in <module>\\n    from ..exceptions import ClientRequestError, raise_with_traceback\\n  File \\"/usr/local/lib/python3.10/dist-packages/msrest/exceptions.py\\", line 31, in <module>\\n    from azure.core.exceptions import SerializationError, DeserializationError\\nModuleNotFoundError: No module named \'azure.core\'\\n", "failed": true, "msg": "Failed to import the required Python library (ansible[azure] (azure >= 2.0.0)) on vps350593\'s Python /usr/bin/python3. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter", "invocation": {"module_args": {"resource_group": "soft-rg", "storage_account_name": "softsa", "container": "artifacts", "blob": "soft-backend/soft-backend.zip", "dest": "/root/tmp/soft/backend.zip", "auth_source": "auto", "cloud_environment": "AzureCloud", "api_profile": "latest", "append_tags": true, "blob_type": "block", "force": false, "state": "present", "profile": null, "subscription_id": null, "client_id": null, "secret": null, "tenant": null, "ad_user": null, "password": null, "cert_validation_mode": null, "adfs_authority_url": null, "log_mode": null, "log_path": null, "x509_certificate_path": null, "thumbprint": null, "tags": null, "src": null, "batch_upload_src": null, "batch_upload_dst": null, "public_access": null, "content_type": null, "content_encoding": null, "content_language": null, "content_disposition": null, "cache_control": null, "content_md5": null}}}\r\n', b"OpenSSH_9.6p1 Ubuntu-3ubuntu13, OpenSSL 3.0.13 30 Jan 2024\r\ndebug1: Reading configuration data /root/.ssh/config\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname 51.255.204.38 is address\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'\r\ndebug1: auto-mux: Trying existing master at '/root/.ansible/cp/9afa1bc5be'\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 69017\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet_timeout: read header failed: Broken pipe\r\ndebug2: Received exit status from master 1\r\nShared connection to 51.255.204.38 closed.\r\n")
<51.255.204.38> Failed to connect to the host via ssh: OpenSSH_9.6p1 Ubuntu-3ubuntu13, OpenSSL 3.0.13 30 Jan 2024
debug1: Reading configuration data /root/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug2: resolve_canonicalize: hostname 51.255.204.38 is address
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'
debug1: auto-mux: Trying existing master at '/root/.ansible/cp/9afa1bc5be'
debug2: fd 3 setting O_NONBLOCK
debug2: mux_client_hello_exchange: master version 4
debug3: mux_client_forwards: request forwardings: 0 local, 0 remote
debug3: mux_client_request_session: entering
debug3: mux_client_request_alive: entering
debug3: mux_client_request_alive: done pid = 69017
debug3: mux_client_request_session: session request sent
debug1: mux_client_request_session: master session id: 2
debug3: mux_client_read_packet_timeout: read header failed: Broken pipe
debug2: Received exit status from master 1
Shared connection to 51.255.204.38 closed.
<51.255.204.38> ESTABLISH SSH CONNECTION FOR USER: user
<51.255.204.38> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="user"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/9afa1bc5be"' 51.255.204.38 '/bin/sh -c '"'"'rm -f -r /home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822/ > /dev/null 2>&1 && sleep 0'"'"''
<51.255.204.38> (0, b'', b"OpenSSH_9.6p1 Ubuntu-3ubuntu13, OpenSSL 3.0.13 30 Jan 2024\r\ndebug1: Reading configuration data /root/.ssh/config\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname 51.255.204.38 is address\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'\r\ndebug1: auto-mux: Trying existing master at '/root/.ansible/cp/9afa1bc5be'\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 69017\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet_timeout: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n")
The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_azure.azcollection.azure_rm_storageblob_payload_1q1m5w5f/ansible_azure.azcollection.azure_rm_storageblob_payload.zip/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 223, in <module>
    from msrestazure.azure_active_directory import AADTokenCredentials
  File "/usr/local/lib/python3.10/dist-packages/msrestazure/__init__.py", line 28, in <module>
    from .azure_configuration import AzureConfiguration
  File "/usr/local/lib/python3.10/dist-packages/msrestazure/azure_configuration.py", line 34, in <module>
    from msrest import Configuration
  File "/usr/local/lib/python3.10/dist-packages/msrest/__init__.py", line 28, in <module>
    from .configuration import Configuration
  File "/usr/local/lib/python3.10/dist-packages/msrest/configuration.py", line 38, in <module>
    from .universal_http.requests import (
  File "/usr/local/lib/python3.10/dist-packages/msrest/universal_http/__init__.py", line 53, in <module>
    from ..exceptions import ClientRequestError, raise_with_traceback
  File "/usr/local/lib/python3.10/dist-packages/msrest/exceptions.py", line 31, in <module>
    from azure.core.exceptions import SerializationError, DeserializationError
ModuleNotFoundError: No module named 'azure.core'
fatal: [51.255.204.38]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "ad_user": null,
            "adfs_authority_url": null,
            "api_profile": "latest",
            "append_tags": true,
            "auth_source": "auto",
            "batch_upload_dst": null,
            "batch_upload_src": null,
            "blob": "soft-backend/soft-backend.zip",
            "blob_type": "block",
            "cache_control": null,
            "cert_validation_mode": null,
            "client_id": null,
            "cloud_environment": "AzureCloud",
            "container": "artifacts",
            "content_disposition": null,
            "content_encoding": null,
            "content_language": null,
            "content_md5": null,
            "content_type": null,
            "dest": "/root/tmp/soft/backend.zip",
            "force": false,
            "log_mode": null,
            "log_path": null,
            "password": null,
            "profile": null,
            "public_access": null,
            "resource_group": "soft-rg",
            "secret": null,
            "src": null,
            "state": "present",
            "storage_account_name": "softsa",
            "subscription_id": null,
            "tags": null,
            "tenant": null,
            "thumbprint": null,
            "x509_certificate_path": null
        }
    },
    "msg": "Failed to import the required Python library (ansible[azure] (azure >= 2.0.0)) on vps350593's Python /usr/bin/python3. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter"
}

@Fred-sun
Copy link
Collaborator

@nimbusparis Can you provide your test script? I'd like to try, thank you very much!

@nimbusparis
Copy link

Inventory:

all:
  children:
    ovh:
      hosts: xxx.xxx.xxx.xxx

group_vars/ovh.yaml

servicePrincipal:
  appId: xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx
  secret: fdsdfdsfsdfsdfsdfsf
  tenant:  xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx

Playbook:

- name: Deployment
  remote_user: user
  become: True
  hosts: ovh

  tasks:
  - name: Install Python3
    ansible.builtin.raw: apt install python3
       
  - name: Login to Azure
    # Use Azure CLI (installed manually) to connect to Azure
    ansible.builtin.command: "az login --service-principal -u {{servicePrincipal.appId}} -p {{servicePrincipal.secret}} -t {{servicePrincipal.tenant}}" 

  - name: Download Pixhint packages
    azure.azcollection.azure_rm_storageblob:
      resource_group: soft-rg
      storage_account_name: softsa
      container: artifacts
      blob: "soft/soft-backend.zip"
      dest: "~/tmp/soft/soft-backend.zip"
     

@mlanner
Copy link

mlanner commented Jan 2, 2025

@nimbusparis,

I've run into similar issues to what you're reporting above. Were you ever able to solve this? I did find out that:

      Starting with v5.0.0, the 'azure' meta-package is deprecated and cannot be installed anymore.
      Please install the service specific packages prefixed by `azure` needed for your application.

      The complete list of available packages can be found at:
      https://aka.ms/azsdk/python/all

      Here's a non-exhaustive list of common packages:

      -  azure-mgmt-compute (https://pypi.python.org/pypi/azure-mgmt-compute) : Management of Virtual Machines, etc.
      -  azure-mgmt-storage (https://pypi.python.org/pypi/azure-mgmt-storage) : Management of storage accounts.
      -  azure-mgmt-resource (https://pypi.python.org/pypi/azure-mgmt-resource) : Generic package about Azure Resource Management (ARM)
      -  azure-keyvault-secrets (https://pypi.python.org/pypi/azure-keyvault-secrets) : Access to secrets in Key Vault
      -  azure-storage-blob (https://pypi.python.org/pypi/azure-storage-blob) : Access to blobs in storage accounts

I've tried to install the packages (above) that I need, but it doesn't solve the problem.

Any tips or recollections you have would be greatly appreciated.

@thisguyshouldworkforus
Copy link

thisguyshouldworkforus commented Jan 2, 2025 via email

@nimbusparis
Copy link

@mlanner

I got the same issue and after many try/error, I succeed after manually install python3, ansible on the target computer. Then, I also installed the azure collection and run pip3 install .../requirements/txt all manipulations had been done with root

not tried again since my last message

@mlanner
Copy link

mlanner commented Jan 2, 2025

@nimbusparis Thx.

@thisguyshouldworkforus I'm new to the idea of "execution environment" (EE). It was just brought up as an option to me a few days ago. In the past I've always just built my control server using the Ubuntu packages (from PPA). More recently (for reasons) I built it using Rocky Linux 9, until I realized it was hopelessly behind in versions.

Over the holidays I switched my Ansible installation back to Ubuntu (again), thinking it would just get me a working system. Ultimately I ended up installing using pip and/or pip through UV because the Ansible from PPA had what appeared as a bunch of of issues with the Azure collection. Still, even with the pivot to pip, the issues remain. However, most basic things seem to work, but I do need to get the Azure thing solved for.

Perhaps I really just need to build an EE based on my own container and control and test it all that way. That said, this has turned into a much bigger project than I expected. My experience with Ansible over the years has generally been that it just worked. Now, merely installing it, getting it working and using various collections successfully seems a lot more complex and a bit hit or miss. :(

@thisguyshouldworkforus
Copy link

thisguyshouldworkforus commented Jan 2, 2025 via email

@mlanner
Copy link

mlanner commented Jan 3, 2025

Ah, gotcha @thisguyshouldworkforus. Another rabbit hole for me to go down ... 🤣 But first, I think I'll try to make it all work in a container. Thx!

@soupyframe
Copy link

I am stuck with the same issue as above here.
This was working flawlessly not too long ago. I ran through some package updates and updated Ansible, and this issue has been the outcome of that.
Currently trying to create a DNS entry with the below task:

 - name: Add an "A" record to Azure
   azure.azcollection.azure_rm_dnsrecordset:
     resource_group: "{{ resource_group }}"
     relative_name: "{{ inventory_hostname }}"
     zone_name: "{{ domain_name }}"
     record_type: A
     records:
       - entry: "{{ ip_address }}"
   delegate_to: localhost
   environment:
     AZURE_SECRET: "{{ azure_secret }}"
     AZURE_CLIENT_ID: "{{ azure_client_id }}"
     AZURE_SUBSCRIPTION_ID: "{{ subscription_id }}"
     AZURE_TENANT: "{{ tenant }}"

Error:

TASK [common/azure_dns : Add an "A" record to Azure] ***************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ModuleNotFoundError: No module named 'azure'
fatal: [myhost-01 -> localhost(mylocalip)]: FAILED! => {"changed": false, "msg": "Failed to import the required Python library (ansible[azure] (azure >= 2.0.0)) on localhost's Python /usr/bin/python3.10. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter"}
pip3 freeze
adal==1.2.7
aiohappyeyeballs==2.4.6
aiohttp==3.9.3
aiosignal==1.3.1
amqtt==0.10.1
ansible==10.1.0
ansible-base==2.10.8
ansible-builder==3.0.1
ansible-compat==24.10.0
ansible-core==2.17.1
ansible-creator==24.12.0
ansible-dev-environment==24.1.0
ansible-dev-tools==24.2.0
ansible-lint==24.2.1
ansible-navigator==24.2.0
ansible-pylibssh==1.2.2
ansible-runner==2.3.6
ansible-sign==0.1.1
anyio==4.3.0
apache-libcloud==3.2.0
applicationinsights==0.11.10
appnope==0.1.4
ara==1.7.2
argcomplete==3.3.0
astroid==3.1.0
asttokens==2.4.1
async-timeout==4.0.3
attrs==23.2.0
autopage==0.5.2
azure-cli-core==2.64.0
azure-cli-telemetry==1.1.0
azure-common==1.1.28
azure-containerregistry==1.2.0
azure-core==1.31.0
azure-graphrbac==0.61.1
azure-identity==1.19.0
azure-iot-hub==2.6.1
azure-keyvault==4.2.0
azure-keyvault-certificates==4.7.0
azure-keyvault-keys==4.8.0
azure-keyvault-secrets==4.7.0
azure-mgmt-apimanagement==4.0.1
azure-mgmt-authorization==4.0.0
azure-mgmt-automation==1.1.0b4
azure-mgmt-batch==17.3.0
azure-mgmt-cdn==13.1.1
azure-mgmt-compute==33.0.0
azure-mgmt-containerinstance==10.1.0
azure-mgmt-containerregistry==10.3.0
azure-mgmt-containerservice==32.1.0
azure-mgmt-core==1.4.0
azure-mgmt-cosmosdb==10.0.0b3
azure-mgmt-datafactory==9.0.0
azure-mgmt-datalake-nspkg==3.0.1
azure-mgmt-datalake-store==1.0.0
azure-mgmt-devtestlabs==10.0.0b2
azure-mgmt-dns==8.1.0
azure-mgmt-eventhub==11.1.0
azure-mgmt-hdinsight==9.1.0b1
azure-mgmt-iothub==3.0.0
azure-mgmt-keyvault==10.3.1
azure-mgmt-loganalytics==13.0.0b7
azure-mgmt-managedservices==7.0.0b2
azure-mgmt-managementgroups==1.1.0b2
azure-mgmt-marketplaceordering==1.2.0b2
azure-mgmt-monitor==6.0.2
azure-mgmt-network==28.0.0
azure-mgmt-notificationhubs==8.1.0b1
azure-mgmt-nspkg==3.0.2
azure-mgmt-privatedns==1.1.0
azure-mgmt-rdbms==10.2.0b17
azure-mgmt-recoveryservices==3.0.0
azure-mgmt-recoveryservicesbackup==9.1.0
azure-mgmt-redis==14.4.0
azure-mgmt-resource==23.2.0
azure-mgmt-search==9.2.0b2
azure-mgmt-servicebus==8.2.1
azure-mgmt-sql==4.0.0b19
azure-mgmt-storage==21.2.1
azure-mgmt-trafficmanager==1.1.0
azure-mgmt-web==7.3.1
azure-nspkg==3.0.2
azure-storage-blob==12.23.0b1
azure-storage-file-datalake==12.14.0
azure-storage-file-share==12.15.0
Babel==2.8.0
backcall==0.2.0
bcrypt==4.1.2
bindep==2.11.0
black==22.12.0
blinker==1.4
bracex==2.4
cachetools==5.3.3
certifi==2024.2.2
cffi==1.16.0
chardet==5.2.0
charset-normalizer==3.3.2
click==8.1.7
click-help-colors==0.9.4
cliff==4.8.0
cmd2==2.5.11
colorama==0.4.6
comm==0.2.2
command-not-found==0.3
cryptography==42.0.5
dbus-python==1.2.18
debugpy==1.8.1
decorator==5.1.1
Deprecated==1.2.14
dill==0.3.8
distlib==0.3.8
distro==1.7.0
distro-info==1.1+ubuntu0.1
dnspython==2.1.0
docker==7.1.0
docopt==0.6.2
docutils==0.20.1
enrich==1.2.7
exceptiongroup==1.2.0
execnet==2.0.2
executing==2.0.1
filelock==3.13.3
frozenlist==1.4.1
geographiclib==2.0
geopy==2.4.1
h11==0.14.0
h2==4.1.0
hpack==4.0.0
httpcore==1.0.5
httplib2==0.20.2
httpx==0.27.0
humanfriendly==10.0
hyperframe==6.0.1
idna==3.7
importlib-metadata==7.0.0
inflection==0.5.1
iniconfig==2.0.0
ipykernel==6.29.3
ipython==8.22.2
ipython-genutils==0.2.0
isodate==0.6.1
isort==5.13.2
jedi==0.19.1
jeepney==0.7.1
Jinja2==3.1.4
jmespath==0.10.0
jsonschema==4.21.1
jsonschema-specifications==2023.12.1
jupyter_client==8.6.1
jupyter_core==5.7.2
keyring==23.5.0
knack==0.11.0
launchpadlib==1.10.16
lazr.restfulclient==0.14.4
lazr.uri==1.0.6
lazy-object-proxy==1.10.0
lockfile==0.12.2
markdown-it-py==3.0.0
MarkupSafe==3.0.2
matplotlib-inline==0.1.6
mccabe==0.7.0
mdurl==0.1.2
microsoft-kiota-abstractions==1.9.1
microsoft-kiota-authentication-azure==1.0.0
microsoft-kiota-http==1.3.1
microsoft-kiota-serialization-form==1.9.1
microsoft-kiota-serialization-json==1.9.1
microsoft-kiota-serialization-multipart==1.9.1
microsoft-kiota-serialization-text==1.0.0
mitogen==0.3.7
molecule==24.12.0
molecule-docker==2.1.0
molecule-plugins==23.5.3
more-itertools==8.10.0
msal==1.30.0
msal-extensions==1.2.0
msgraph-core==1.0.0
msgraph-sdk==1.6.0
msrest==0.7.1
msrestazure==0.6.4
multidict==6.0.5
mypy-extensions==1.0.0
nest-asyncio==1.6.0
netaddr==0.8.0
netifaces==0.11.0
ntlm-auth==1.4.0
numpy==1.26.4
oauthlib==3.2.0
onigurumacffi==1.3.0
opentelemetry-api==1.30.0
opentelemetry-sdk==1.30.0
opentelemetry-semantic-conventions==0.51b0
oras==0.2.25
packaging==24.2
pandas==2.2.1
paramiko==2.12.0
Parsley==1.3
parso==0.8.3
passlib==1.7.4
pathspec==0.12.1
pbr==6.0.0
pendulum==3.0.0
pexpect==4.9.0
phpypam==1.0.2
pickleshare==0.7.5
pkginfo==1.10.0
platformdirs==4.2.0
pluggy==1.4.0
portalocker==1.7.1
prettytable==3.14.0
prompt-toolkit==3.0.43
propcache==0.3.0
proxmoxer==2.0.1
psutil==5.9.8
ptyprocess==0.7.0
pure-eval==0.2.2
pyclickup==0.1.4
pycparser==2.22
pycryptodomex==3.11.0
Pygments==2.17.2
PyGObject==3.42.1
PyJWT==2.3.0
pykerberos==1.1.14
pylint==3.1.0
PyNaCl==1.5.0
pyOpenSSL==22.0.0
pyparsing==2.4.7
pyperclip==1.9.0
pyproject-api==1.6.1
pypsexec==0.3.0
pypsrp==0.8.1
pyrsistent==0.20.0
PySocks==1.7.1
pyspnego==0.10.2
pytest==8.1.1
pytest-ansible==24.1.2
pytest-xdist==3.5.0
python-apt==2.4.0+ubuntu2
python-daemon==3.0.1
python-dateutil==2.9.0.post0
python-gnupg==0.5.2
pytz==2022.1
pywinrm==0.4.3
PyYAML==6.0.2
pyzmq==25.1.2
referencing==0.34.0
reflection==0.1.1
requests==2.31.0
requests-credssp==2.0.0
requests-kerberos==0.12.0
requests-ntlm==1.2.0
requests-oauthlib==2.0.0
requests-toolbelt==0.9.1
requirements-parser==0.5.0
resolvelib==1.0.1
rich==13.7.1
rpds-py==0.18.0
ruamel.yaml==0.18.6
ruamel.yaml.clib==0.2.8
SecretStorage==3.3.1
selinux==3.3
simplejson==3.17.6
six==1.16.0
smbprotocol==1.14.0
sniffio==1.3.1
speedtest-cli==2.1.3
stack-data==0.6.3
std-uritemplate==2.0.1
stevedore==5.4.0
streamdeck==0.9.5
subprocess-tee==0.4.1
systemd-python==234
tabulate==0.9.0
time-machine==2.14.1
TogglPy==0.1.2
toml==0.10.2
tomli==2.0.1
tomlkit==0.12.4
tornado==6.4
tox==4.14.2
tox-ansible==24.2.0
traitlets==5.14.2
transitions==0.8.11
types-setuptools==69.2.0.20240317
typing_extensions==4.10.0
tzdata==2024.1
uamqp==1.6.9
ubuntu-advantage-tools==8001
ufw==0.36.1
unattended-upgrades==0.1
urllib3==2.2.1
virtualenv==20.25.1
wadllib==1.3.6
wcmatch==8.5.1
wcwidth==0.2.13
websocket-client==1.2.3
websockets==12.0
wrapt==1.16.0
xmltodict==0.13.0
yamllint==1.35.1
yarl==1.9.4
zabbix-api==0.5.6
zipp==3.21.0

@mlanner
Copy link

mlanner commented Feb 24, 2025

@soupyframe I did do a few things to work around this problem. Ultimately I don't recall what actually fixed it, but I know for sure that I created a few entries in an ansible.cfg file, either at the top level (globally) or at the project level under the [defaults] section:

interpreter_python = /path/to/python3
collections_path = /path/to/ansible/lib/python3.12/site-packages/ansible_collections

... where the /path/to is the path to the specific locations for your Python installation and Ansible collections.

That said, in my case I had a bunch of other dependency issues, which were generally caused by my original Ansible installation running on a RHEL 9 derivative, which had old Ansible, combined with issues around target machines that were running RHEL 8 derivatives. I then switched my Ansible installation to Ubuntu, which fixed a bunch of legacy and EOL issues, including things around the Azure collection, but then caused even more issues due to incompatible/EOL Python versions. Yeah, it's a mess. I started down the path of containerizing Ansible, generally working towards the Execution Environment (EE) path mentioned above. I think that's probably the right direction to go, but it also adds several layers of complexity and additional knowledge, understanding and infra you need to create to make it work. I'm not 100% there yet myself. Surely, doing it the EE way allows you to be very explicit and have controllable environments. But, it's also a lot of work to a) wrap your head around all of that new stuff, b) get that all stood up and working, and c) maintain whatever environments you might need to create for your use-case. Well, anyways ...

@soupyframe
Copy link

@soupyframe I did do a few things to work around this problem. Ultimately I don't recall what actually fixed it, but I know for sure that I created a few entries in an ansible.cfg file, either at the top level (globally) or at the project level under the [defaults] section:

interpreter_python = /path/to/python3
collections_path = /path/to/ansible/lib/python3.12/site-packages/ansible_collections

... where the /path/to is the path to the specific locations for your Python installation and Ansible collections.

That said, in my case I had a bunch of other dependency issues, which were generally caused by my original Ansible installation running on a RHEL 9 derivative, which had old Ansible, combined with issues around target machines that were running RHEL 8 derivatives. I then switched my Ansible installation to Ubuntu, which fixed a bunch of legacy and EOL issues, including things around the Azure collection, but then caused even more issues due to incompatible/EOL Python versions. Yeah, it's a mess. I started down the path of containerizing Ansible, generally working towards the Execution Environment (EE) path mentioned above. I think that's probably the right direction to go, but it also adds several layers of complexity and additional knowledge, understanding and infra you need to create to make it work. I'm not 100% there yet myself. Surely, doing it the EE way allows you to be very explicit and have controllable environments. But, it's also a lot of work to a) wrap your head around all of that new stuff, b) get that all stood up and working, and c) maintain whatever environments you might need to create for your use-case. Well, anyways ...

Thank you for providing some insight.
In my case, Ansible is correctly picking up the right package and python locations


ansible --version
ansible [core 2.17.8]
  config file = /home/user/vss/Automation/ansible.cfg
  configured module search path = ['/home/user/vss/Automation/library']
  ansible python module location = /home/user/.local/lib/python3.10/site-packages/ansible
  ansible collection location = /home/user/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/user/.local/bin/ansible
  python version = 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] (/usr/bin/python3)
  jinja version = 3.1.4
  libyaml = True

Trying a new install to see if i can get around the issue. I used to use an EE, but since stopped as it was adding unnecessary overhead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file medium_priority Medium priority question Further information is requested work in In trying to solve, or in working with contributors
Projects
None yet
Development

No branches or pull requests

9 participants