Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bug in pip_installed when trying to install multiple packages with specific version for package #55288

Open
Mapel88 opened this issue Nov 13, 2019 · 5 comments
Open

Bug in pip_installed when trying to install multiple packages with specific version for package #55288

Mapel88 opened this issue Nov 13, 2019 · 5 comments
Labels
Bug broken, incorrect, or confusing behavior Pending-Discussion The issue or pull request needs more discussion before it can be closed or merged
Milestone
Blocked

Comments

@Mapel88
Copy link
Contributor

Mapel88 commented Nov 13, 2019

Description of Issue

pip install throws exception when trying to install multiple packages with specific version for package

Setup

Tested with pillar and requirements files. Same error received.

Using pillar:

my pillar:

pip_pkg_list:
  - httplib2
  - sqlalchemy
  - argparse
  - boto >= 2.30, <= 2.30
  - python-dateutil
  - psycopg2
  - logstash_formatter
  - psutil
  - boxsdk

my state;

{% set pip_pkg_list = salt['pillar.get']( 'pip_pkg_list', none) %}
install_pip_packages:
  pip.installed:
    - names: {{ pip_pkg_list }}

using requirements file:

my requirements file:

httplib2
boto==2.30

install_pip_packages:
  pip.installed:
    - requirements: C:\Python27\scripts\requirements.txt

Steps to Reproduce Issue

C:\Users\maayana>salt-call state.apply windows.install_pkgs.test --master=1.1.1.1 -l debug
[DEBUG   ] Reading configuration from c:\salt\conf\minion
[DEBUG   ] Including configuration from 'c:\salt\conf\minion.d\_schedule.conf'
[DEBUG   ] Reading configuration from c:\salt\conf\minion.d\_schedule.conf
[DEBUG   ] Configuration file path: c:\salt\conf\minion
[WARNING ] Insecure logging configuration detected! Sensitive data may be logged.
[DEBUG   ] Grains refresh requested. Refreshing grains.
[DEBUG   ] Reading configuration from c:\salt\conf\minion
[DEBUG   ] Including configuration from 'c:\salt\conf\minion.d\_schedule.conf'
[DEBUG   ] Reading configuration from c:\salt\conf\minion.d\_schedule.conf
[DEBUG   ] Motherboard info not available on this system
[DEBUG   ] Loading static grains from c:\salt\conf\grains
[DEBUG   ] Connecting to master. Attempt 1 (infinite attempts)
[DEBUG   ] Master URI: tcp://1.1.1.1:4506
[DEBUG   ] Initializing new AsyncAuth for ('c:\\salt\\conf\\pki\\minion', 'my-minion', 'tcp://1.1.1.1:4506')
[DEBUG   ] Generated random reconnect delay between '1000ms' and '11000ms' (6955)
[DEBUG   ] Setting zmq_reconnect_ivl to '6955ms'
[DEBUG   ] Setting zmq_reconnect_ivl_max to '11000ms'
[DEBUG   ] Initializing new AsyncZeroMQReqChannel for ('c:\\salt\\conf\\pki\\minion', 'my-minion', 'tcp://1.1.1.1:4506', 'clear')
[DEBUG   ] Connecting the Minion to the Master URI (for the return server): tcp://1.1.1.1:4506
[DEBUG   ] Trying to connect to: tcp://1.1.1.1:4506
[DEBUG   ] salt.crypt.get_rsa_pub_key: Loading public key
[DEBUG   ] Decrypting the current master AES key
[DEBUG   ] salt.crypt.get_rsa_key: Loading private key
[DEBUG   ] salt.crypt._get_key_with_evict: Loading private key
[DEBUG   ] Loaded minion key: c:\salt\conf\pki\minion\minion.pem
[DEBUG   ] salt.crypt.get_rsa_pub_key: Loading public key
[DEBUG   ] Connecting the Minion to the Master publish port, using the URI: tcp://1.1.1.1:4505
[DEBUG   ] salt.crypt.get_rsa_key: Loading private key
[DEBUG   ] Loaded minion key: c:\salt\conf\pki\minion\minion.pem
[DEBUG   ] Determining pillar cache
[DEBUG   ] Initializing new AsyncZeroMQReqChannel for ('c:\\salt\\conf\\pki\\minion', 'my-minion', 'tcp://1.1.1.1:4506', 'aes')
[DEBUG   ] Initializing new AsyncAuth for ('c:\\salt\\conf\\pki\\minion', 'my-minion', 'tcp://1.1.1.1:4506')
[DEBUG   ] Connecting the Minion to the Master URI (for the return server): tcp://1.1.1.1:4506
[DEBUG   ] Trying to connect to: tcp://1.1.1.1:4506
[DEBUG   ] salt.crypt.get_rsa_key: Loading private key
[DEBUG   ] Loaded minion key: c:\salt\conf\pki\minion\minion.pem
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded state.apply
[DEBUG   ] LazyLoaded saltutil.is_running
[DEBUG   ] LazyLoaded grains.get
[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Initializing new AsyncZeroMQReqChannel for ('c:\\salt\\conf\\pki\\minion', 'my-minion', 'tcp://1.1.1.1:4506', 'aes')
[DEBUG   ] Initializing new AsyncAuth for ('c:\\salt\\conf\\pki\\minion', 'my-minion', 'tcp://1.1.1.1:4506')
[DEBUG   ] Connecting the Minion to the Master URI (for the return server): tcp://1.1.1.1:4506
[DEBUG   ] Trying to connect to: tcp://1.1.1.1:4506
[DEBUG   ] Gathering pillar data for state run
[DEBUG   ] Finished gathering pillar data for state run
[INFO    ] Loading fresh modules for state activity
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] In saltenv 'base', looking at rel_path 'windows/install_pkgs/test.sls' to resolve 'salt://windows/install_pkgs/test.sls'
[DEBUG   ] In saltenv 'base', ** considering ** path 'c:\salt\var\cache\salt\minion\files\base\windows\install_pkgs\test.sls' to resolve 'salt://windows/install_pkgs/test.sls'
[DEBUG   ] Fetching file from saltenv 'base', ** attempting ** 'salt://windows/install_pkgs/test.sls'
[DEBUG   ] No dest file found
[INFO    ] Fetching file from saltenv 'base', ** done ** 'windows/install_pkgs/test.sls'
[DEBUG   ] compile template: c:\salt\var\cache\salt\minion\files\base\windows\install_pkgs\test.sls
[DEBUG   ] Jinja search path: ['c:\\salt\\var\\cache\\salt\\minion\\files\\base']
[DEBUG   ] Initializing new AsyncZeroMQReqChannel for ('c:\\salt\\conf\\pki\\minion', 'my-minion', 'tcp://1.1.1.1:4506', 'aes')
[DEBUG   ] Initializing new AsyncAuth for ('c:\\salt\\conf\\pki\\minion', 'my-minion', 'tcp://1.1.1.1:4506')
[DEBUG   ] Connecting the Minion to the Master URI (for the return server): tcp://1.1.1.1:4506
[DEBUG   ] Trying to connect to: tcp://1.1.1.1:4506
[PROFILE ] Time (in seconds) to render 'c:\salt\var\cache\salt\minion\files\base\windows\install_pkgs\test.sls' using 'jinja' renderer: 0.04297304153442383
[DEBUG   ] Rendered data from file: c:\salt\var\cache\salt\minion\files\base\windows\install_pkgs\test.sls:
copy_pip_requirements_file:
  file.managed:
    - name: 'C:\Python27\scripts\requirements.txt'
    - source: salt://windows/install_pkgs/files/requirements.txt

install_pip_packages:
  pip.installed:
    - requirements: C:\Python27\scripts\requirements.txt
[DEBUG   ] Results of YAML rendering:
OrderedDict([('copy_pip_requirements_file', OrderedDict([('file.managed', [OrderedDict([('name', 'C:\\Python27\\scripts\\requirements.txt')]), OrderedDict([('source', 'salt://windows/install_pkgs/files/requirements.txt')])])])), ('install_pip_packages', OrderedDict([('pip.installed', [OrderedDict([('requirements', 'C:\\Python27\\scripts\\requirements.txt')])])]))])
[PROFILE ] Time (in seconds) to render 'c:\salt\var\cache\salt\minion\files\base\windows\install_pkgs\test.sls' using 'yaml' renderer: 0.004998922348022461
[DEBUG   ] LazyLoaded config.option
[DEBUG   ] LazyLoaded file.managed
[INFO    ] Running state [C:\Python27\scripts\requirements.txt] at time 08:01:15.283484
[INFO    ] Executing state file.managed for [C:\Python27\scripts\requirements.txt]
[DEBUG   ] LazyLoaded file.source_list
[DEBUG   ] LazyLoaded cp.hash_file
[DEBUG   ] Initializing new AsyncZeroMQReqChannel for ('c:\\salt\\conf\\pki\\minion', 'my-minion', 'tcp://1.1.1.1:4506', 'aes')
[DEBUG   ] Initializing new AsyncAuth for ('c:\\salt\\conf\\pki\\minion', 'my-minion', 'tcp://1.1.1.1:4506')
[DEBUG   ] Connecting the Minion to the Master URI (for the return server): tcp://1.1.1.1:4506
[DEBUG   ] Trying to connect to: tcp://1.1.1.1:4506
[DEBUG   ] In saltenv 'base', looking at rel_path 'windows/install_pkgs/files/requirements.txt' to resolve 'salt://windows/install_pkgs/files/requirements.txt'
[DEBUG   ] In saltenv 'base', ** considering ** path 'c:\salt\var\cache\salt\minion\files\base\windows\install_pkgs\files\requirements.txt' to resolve 'salt://windows/install_pkgs/files/requirements.txt'
[INFO    ] File C:\Python27\scripts\requirements.txt is in the correct state
[INFO    ] Completed state [C:\Python27\scripts\requirements.txt] at time 08:01:16.458744 (duration_in_ms=1175.26)
[DEBUG   ] LazyLoaded event.fire_master
[DEBUG   ] Initializing new SAuth for ('c:\\salt\\conf\\pki\\minion', 'my-minion', 'tcp://1.1.1.1:4506')
[DEBUG   ] salt.crypt.get_rsa_key: Loading private key
[DEBUG   ] Loaded minion key: c:\salt\conf\pki\minion\minion.pem
[DEBUG   ] Initializing new AsyncZeroMQReqChannel for ('c:\\salt\\conf\\pki\\minion', 'my-minion', 'tcp://1.1.1.1:4506', 'aes')
[DEBUG   ] Initializing new AsyncAuth for ('c:\\salt\\conf\\pki\\minion', 'my-minion', 'tcp://1.1.1.1:4506')
[DEBUG   ] Connecting the Minion to the Master URI (for the return server): tcp://1.1.1.1:4506
[DEBUG   ] Trying to connect to: tcp://1.1.1.1:4506
[DEBUG   ] LazyLoaded reg.read_value
[DEBUG   ] LazyLoaded platform.is_windows
[DEBUG   ] LazyLoaded path.which
[WARNING ] C:\salt\bin\lib\site-packages\salt\modules\win_update.py:94: DeprecationWarning: The 'win_update' module is being deprecated and will be removed in Salt Fluorine (Unreleased). Please use the 'win_wua' module instead.

[DEBUG   ] Could not LazyLoad boto_datapipeline.create_pipeline: 'boto_datapipeline' __virtual__ returned False: A minimum version of boto3 1.2.6 is required.
[DEBUG   ] Registered VCS backend: git
[DEBUG   ] Registered VCS backend: hg
[DEBUG   ] Config variable 'Py_DEBUG' is unset, Python ABI tag may be incorrect
[DEBUG   ] Config variable 'WITH_PYMALLOC' is unset, Python ABI tag may be incorrect
[DEBUG   ] Config variable 'Py_DEBUG' is unset, Python ABI tag may be incorrect
[DEBUG   ] Config variable 'WITH_PYMALLOC' is unset, Python ABI tag may be incorrect
[DEBUG   ] Registered VCS backend: svn
[DEBUG   ] Registered VCS backend: bzr
[DEBUG   ] LazyLoaded pip.installed
[INFO    ] Running state [install_pip_packages] at time 08:01:24.582744
[INFO    ] Executing state pip.installed for [install_pip_packages]
[DEBUG   ] pip: Using pip from currently-running Python
[INFO    ] Executing command ['C:\\salt\\bin\\python.exe', '-m', 'pip', '--version'] in directory 'C:\Users\maayana'
[DEBUG   ] stdout: pip 9.0.1 from C:\salt\bin\lib\site-packages (python 3.5)
[DEBUG   ] pip: Using pip from currently-running Python
[DEBUG   ] TREQ IS: None
[DEBUG   ] CLEANUP_REQUIREMENTS: [None]
[DEBUG   ] TRY BLOCK: end of pip.install -- cmd: ['C:\\salt\\bin\\python.exe', '-m', 'pip', 'install', '--requirement', 'C:\\Python27\\scripts\\requirements.txt'], cmd_kwargs: {'saltenv': 'base', 'use_vt': False, 'runas': None}
[INFO    ] Executing command ['C:\\salt\\bin\\python.exe', '-m', 'pip', 'install', '--requirement', 'C:\\Python27\\scripts\\requirements.txt'] in directory 'C:\Users\maayana'
[ERROR   ] Command '['C:\\salt\\bin\\python.exe', '-m', 'pip', 'install', '--requirement', 'C:\\Python27\\scripts\\requirements.txt']' failed with return code: 1
[ERROR   ] stdout: Requirement already satisfied: httplib2 in c:\salt\bin\lib\site-packages (from -r C:\Python27\scripts\requirements.txt (line 1))
Collecting boto==2.30 (from -r C:\Python27\scripts\requirements.txt (line 2))
  Using cached https://files.pythonhosted.org/packages/5b/b0/7ab9e9be2459783cccba8d92f2e56b28ac2193e182a0ef89ba5b7f71d7cd/boto-2.30.0.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\maayana\AppData\Local\Temp\2\pip-build-jjgd_tg7\boto\setup.py", line 37, in <module>
        from boto import __version__
      File "C:\Users\maayana\AppData\Local\Temp\2\pip-build-jjgd_tg7\boto\boto\__init__.py", line 27, in <module>
        from boto.pyami.config import Config, BotoConfigLocations
      File "C:\Users\maayana\AppData\Local\Temp\2\pip-build-jjgd_tg7\boto\boto\pyami\config.py", line 182
        print s.getvalue()
              ^
    SyntaxError: invalid syntax

    ----------------------------------------
[ERROR   ] stderr: Command "python setup.py egg_info" failed with error code 1 in C:\Users\maayana\AppData\Local\Temp\2\pip-build-jjgd_tg7\boto\
You are using pip version 9.0.1, however version 19.3.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
[ERROR   ] retcode: 1
[ERROR   ] Unable to process requirements file "C:\Python27\scripts\requirements.txt". Error: Requirement already satisfied: httplib2 in c:\salt\bin\lib\site-packages (from -r C:\Python27\scripts\requirements.txt (line 1))
Collecting boto==2.30 (from -r C:\Python27\scripts\requirements.txt (line 2))
  Using cached https://files.pythonhosted.org/packages/5b/b0/7ab9e9be2459783cccba8d92f2e56b28ac2193e182a0ef89ba5b7f71d7cd/boto-2.30.0.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\maayana\AppData\Local\Temp\2\pip-build-jjgd_tg7\boto\setup.py", line 37, in <module>
        from boto import __version__
      File "C:\Users\maayana\AppData\Local\Temp\2\pip-build-jjgd_tg7\boto\boto\__init__.py", line 27, in <module>
        from boto.pyami.config import Config, BotoConfigLocations
      File "C:\Users\maayana\AppData\Local\Temp\2\pip-build-jjgd_tg7\boto\boto\pyami\config.py", line 182
        print s.getvalue()
              ^
    SyntaxError: invalid syntax

    ---------------------------------------- Command "python setup.py egg_info" failed with error code 1 in C:\Users\maayana\AppData\Local\Temp\2\pip-build-jjgd_tg7\boto\
You are using pip version 9.0.1, however version 19.3.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
[INFO    ] Completed state [install_pip_packages] at time 08:01:29.132024 (duration_in_ms=4549.28)
[DEBUG   ] Initializing new SAuth for ('c:\\salt\\conf\\pki\\minion', 'my-minion', 'tcp://1.1.1.1:4506')
[DEBUG   ] salt.crypt.get_rsa_key: Loading private key
[DEBUG   ] Loaded minion key: c:\salt\conf\pki\minion\minion.pem
[DEBUG   ] Initializing new AsyncZeroMQReqChannel for ('c:\\salt\\conf\\pki\\minion', 'my-minion', 'tcp://1.1.1.1:4506', 'aes')
[DEBUG   ] Initializing new AsyncAuth for ('c:\\salt\\conf\\pki\\minion', 'my-minion', 'tcp://1.1.1.1:4506')
[DEBUG   ] Connecting the Minion to the Master URI (for the return server): tcp://1.1.1.1:4506
[DEBUG   ] Trying to connect to: tcp://1.1.1.1:4506
[DEBUG   ] File c:\salt\var\cache\salt\minion\accumulator\2118932491400 does not exist, no need to cleanup
[INFO    ] Executing command ['attrib', '-R', 'c:\\salt\\var\\cache\\salt\\minion\\sls.p'] in directory 'C:\Users\maayana'
[DEBUG   ] output:
[DEBUG   ] LazyLoaded state.check_result
[DEBUG   ] Initializing new AsyncZeroMQReqChannel for ('c:\\salt\\conf\\pki\\minion', 'my-minion', 'tcp://1.1.1.1:4506', 'aes')
[DEBUG   ] Initializing new AsyncAuth for ('c:\\salt\\conf\\pki\\minion', 'my-minion', 'tcp://1.1.1.1:4506')
[DEBUG   ] Connecting the Minion to the Master URI (for the return server): tcp://1.1.1.1:4506
[DEBUG   ] Trying to connect to: tcp://1.1.1.1:4506
[DEBUG   ] LazyLoaded highstate.output
local:
----------
          ID: copy_pip_requirements_file
    Function: file.managed
        Name: C:\Python27\scripts\requirements.txt
      Result: True
     Comment: File C:\Python27\scripts\requirements.txt is in the correct state
     Started: 08:01:15.283484
    Duration: 1175.26 ms
     Changes:
----------
          ID: install_pip_packages
    Function: pip.installed
      Result: False
     Comment: Unable to process requirements file "C:\Python27\scripts\requirements.txt". Error: Requirement already satisfied: httplib2 in c:\salt\bin\lib\site-packages (from -r C:\Python27\scripts\requirements.txt (line 1))
              Collecting boto==2.30 (from -r C:\Python27\scripts\requirements.txt (line 2))
                Using cached https://files.pythonhosted.org/packages/5b/b0/7ab9e9be2459783cccba8d92f2e56b28ac2193e182a0ef89ba5b7f71d7cd/boto-2.30.0.tar.gz
                  Complete output from command python setup.py egg_info:
                  Traceback (most recent call last):
                    File "<string>", line 1, in <module>
                    File "C:\Users\maayana\AppData\Local\Temp\2\pip-build-jjgd_tg7\boto\setup.py", line 37, in <module>
                      from boto import __version__
                    File "C:\Users\maayana\AppData\Local\Temp\2\pip-build-jjgd_tg7\boto\boto\__init__.py", line 27, in <module>
                      from boto.pyami.config import Config, BotoConfigLocations
                    File "C:\Users\maayana\AppData\Local\Temp\2\pip-build-jjgd_tg7\boto\boto\pyami\config.py", line 182
                      print s.getvalue()
                            ^
                  SyntaxError: invalid syntax

                  ---------------------------------------- Command "python setup.py egg_info" failed with error code 1 in C:\Users\maayana\AppData\Local\Temp\2\pip-build-jjgd_tg7\boto\
              You are using pip version 9.0.1, however version 19.3.1 is available.
              You should consider upgrading via the 'python -m pip install --upgrade pip' command.
     Started: 08:01:24.582744
    Duration: 4549.28 ms
     Changes:

Summary for local
------------
Succeeded: 1
Failed:    1
------------
Total states run:     2
Total run time:   5.725 s

Versions Report

minion version: 2018.3.4

master version:

Salt Version:
           Salt: 2019.2.1

Dependency Versions:
           cffi: 1.13.2
       cherrypy: unknown
       dateutil: 1.5
      docker-py: Not Installed
          gitdb: 0.6.4
      gitpython: 1.0.1
          ioflo: Not Installed
         Jinja2: 2.7.2
        libgit2: 0.24.6
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.8
   mysql-python: Not Installed
      pycparser: 2.14
       pycrypto: 2.6.1
   pycryptodome: 3.6.1
         pygit2: 0.24.2
         Python: 2.7.5 (default, Aug  7 2019, 00:51:29)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.3.0
           RAET: Not Installed
          smmap: 0.9.0
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4

System Versions:
           dist: centos 7.6.1810 Core
         locale: UTF-8
        machine: x86_64
        release: 3.10.0-957.1.3.el7.x86_64
         system: Linux
        version: CentOS Linux 7.6.1810 Core
@xeacott
Copy link
Contributor

xeacott commented Nov 14, 2019

Thanks for submitting. Not sure on this one exactly as to why its failing, as the traceback is showing issues with boto. Are you able to install a single package using something like...
salt-call windows state.apply list

where list.sls looks like...

install_pip_packages:
  pip.installed:
    - pkgs:
      - httplib2
      - sqlalchemy
      - argparse

@xeacott xeacott added Bug broken, incorrect, or confusing behavior P3 Priority 3 labels Nov 14, 2019
@xeacott xeacott added this to the Blocked milestone Nov 14, 2019
@xeacott xeacott added the Pending-Discussion The issue or pull request needs more discussion before it can be closed or merged label Nov 14, 2019
@Mapel88
Copy link
Contributor Author

Mapel88 commented Nov 17, 2019

@xeacott it's working properly with multiple packages if I'm not using a specific version.

@stale
Copy link

stale bot commented Jan 7, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

@stale stale bot added the stale label Jan 7, 2020
@Mapel88
Copy link
Contributor Author

Mapel88 commented Jan 8, 2020

not stale...

@stale
Copy link

stale bot commented Jan 8, 2020

Thank you for updating this issue. It is no longer marked as stale.

@stale stale bot removed the stale label Jan 8, 2020
@sagetherage sagetherage removed the P3 Priority 3 label Jun 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug broken, incorrect, or confusing behavior Pending-Discussion The issue or pull request needs more discussion before it can be closed or merged
Projects
None yet
Milestone
Blocked
Development

No branches or pull requests
3 participants
@Mapel88 @xeacott @sagetherage