awslog agent fails to install

[epackorigan]

When creating the stack, the automate instance is failing relatively quickly. it looks like something in python has changed and it is causing the instance to fail the setup.

Version 1.4.6 Install Starting
Collecting virtualenv
  Downloading https://files.pythonhosted.org/packages/c1/61/7506ddd79ef6f09beeefb81c4c55bf395a8ad96b33ff1c6b06e40f8aa101/virtualenv-20.0.7-py2.py3-none-any.whl (8.0MB)
Requirement already satisfied (use --upgrade to upgrade): six<2,>=1.9.0 in /usr/lib/python2.7/site-packages (from virtualenv)
Collecting appdirs<2,>=1.4.3 (from virtualenv)
  Downloading https://files.pythonhosted.org/packages/56/eb/810e700ed1349edde4cbdc1b2a21e28cdf115f9faf263f6bbf8447c1abf3/appdirs-1.4.3-py2.py3-none-any.whl
Collecting importlib-metadata<2,>=0.12; python_version < "3.8" (from virtualenv)
  Downloading https://files.pythonhosted.org/packages/8b/03/a00d504808808912751e64ccf414be53c29cad620e3de2421135fcae3025/importlib_metadata-1.5.0-py2.py3-none-any.whl
Collecting distlib<1,>=0.3.0 (from virtualenv)
  Downloading https://files.pythonhosted.org/packages/7d/29/694a3a4d7c0e1aef76092e9167fbe372e0f7da055f5dcf4e1313ec21d96a/distlib-0.3.0.zip (571kB)
Collecting filelock<4,>=3.0.0 (from virtualenv)
  Downloading https://files.pythonhosted.org/packages/14/ec/6ee2168387ce0154632f856d5cc5592328e9cf93127c5c9aeca92c8c16cb/filelock-3.0.12.tar.gz
Collecting importlib-resources<2,>=1.0; python_version < "3.7" (from virtualenv)
  Downloading https://files.pythonhosted.org/packages/45/51/5baae3dde223ff6b64aecaf4c191d2a2679f60abf1270b337823af668bf5/importlib_resources-1.2.0-py2.py3-none-any.whl
Collecting contextlib2<1,>=0.6.0; python_version < "3.3" (from virtualenv)
  Downloading https://files.pythonhosted.org/packages/85/60/370352f7ef6aa96c52fb001831622f50f923c1d575427d021b8ab3311236/contextlib2-0.6.0.post1-py2.py3-none-any.whl
Collecting pathlib2<3,>=2.3.3; python_version < "3.4" and sys_platform != "win32" (from virtualenv)
  Downloading https://files.pythonhosted.org/packages/e9/45/9c82d3666af4ef9f221cbb954e1d77ddbb513faf552aea6df5f37f1a4859/pathlib2-2.3.5-py2.py3-none-any.whl
Collecting configparser>=3.5; python_version < "3" (from importlib-metadata<2,>=0.12; python_version < "3.8"->virtualenv)
  Downloading https://files.pythonhosted.org/packages/7a/2a/95ed0501cf5d8709490b1d3a3f9b5cf340da6c433f896bbe9ce08dbe6785/configparser-4.0.2-py2.py3-none-any.whl
Collecting zipp>=0.5 (from importlib-metadata<2,>=0.12; python_version < "3.8"->virtualenv)
  Downloading https://files.pythonhosted.org/packages/ce/8c/2c5f7dc1b418f659d36c04dec9446612fc7b45c8095cc7369dd772513055/zipp-3.1.0.tar.gz
  Running setup.py (path:/tmp/pip-build-JHtZ_h/zipp/setup.py) egg_info for package zipp produced metadata for project name unknown. Fix your #egg=zipp fragments.
Collecting typing; python_version < "3.5" (from importlib-resources<2,>=1.0; python_version < "3.7"->virtualenv)
  Downloading https://files.pythonhosted.org/packages/22/30/64ca29543375759dc589ade14a6cd36382abf2bec17d67de8481bc9814d7/typing-3.7.4.1-py2-none-any.whl
Collecting singledispatch; python_version < "3.4" (from importlib-resources<2,>=1.0; python_version < "3.7"->virtualenv)
  Downloading https://files.pythonhosted.org/packages/c5/10/369f50bcd4621b263927b0a1519987a04383d4a98fb10438042ad410cf88/singledispatch-3.4.0.3-py2.py3-none-any.whl
Collecting scandir; python_version < "3.5" (from pathlib2<3,>=2.3.3; python_version < "3.4" and sys_platform != "win32"->virtualenv)
  Downloading https://files.pythonhosted.org/packages/df/f5/9c052db7bd54d0cbf1bc0bb6554362bba1012d03e5888950a4f5c5dadc4e/scandir-1.10.0.tar.gz
Installing collected packages: appdirs, configparser, unknown, contextlib2, scandir, pathlib2, importlib-metadata, distlib, filelock, typing, singledispatch, importlib-resources, virtualenv
  Running setup.py install for unknown: started
    Running setup.py install for unknown: finished with status 'done'
  Running setup.py install for scandir: started
    Running setup.py install for scandir: finished with status 'done'
  Running setup.py install for distlib: started
    Running setup.py install for distlib: finished with status 'done'
  Running setup.py install for filelock: started
    Running setup.py install for filelock: finished with status 'done'
Successfully installed appdirs-1.4.3 configparser-4.0.2 contextlib2-0.6.0.post1 distlib-0.3.0 filelock-3.0.12 importlib-metadata-1.5.0 importlib-resources-1.2.0 pathlib2-2.3.5 scandir-1.10.0 singledispatc
h-3.4.0.3 typing-3.7.4.1 unknown-0.0.0 virtualenv-20.0.7
You are using pip version 8.1.2, however version 20.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
/usr/bin/virtualenv
Traceback (most recent call last):
  File "/usr/bin/virtualenv", line 7, in <module>
    from virtualenv.__main__ import run_with_catch
  File "/usr/lib/python2.7/site-packages/virtualenv/__init__.py", line 3, in <module>
    from .run import cli_run
  File "/usr/lib/python2.7/site-packages/virtualenv/run/__init__.py", line 12, in <module>
    from .plugin.activators import ActivationSelector
  File "/usr/lib/python2.7/site-packages/virtualenv/run/plugin/activators.py", line 6, in <module>
    from .base import ComponentBuilder
  File "/usr/lib/python2.7/site-packages/virtualenv/run/plugin/base.py", line 9, in <module>
    from importlib_metadata import entry_points
  File "/usr/lib/python2.7/site-packages/importlib_metadata/__init__.py", line 9, in <module>
    import zipp
ImportError: No module named zipp```

interestingly enough, if i update pip with `pip install --upgrade pip` then I can run the command successfully in the instance (after it's failed).

[epackorigan]

relevant: pypa/virtualenv#1630

[irvingpop]

hey @epackorigan I suggest starting by respinning a new AMI from here: https://github.com/irvingpop/packer-chef-highperf-centos-ami using the packer-marketplace.json definition. see if that error is still reproducible.

[epackorigan]

It appears that the centos-7-minimal-* base ami is no longer there, as i get the following:

Build 'amazon-ebs' errored: No AMI was found matching filters: {
  Filters: [{
      Name: "virtualization-type",
      Values: ["hvm"]
    },{
      Name: "name",
      Values: ["centos-7-minimal-install-*"]
    },{
      Name: "root-device-type",
      Values: ["ebs"]
    }],
  Owners: ["self","446539779517","amazon"]
}

when attempting to build the image.

I went to my console and try to find a suitable image to check, and searching for 'centos-7-minimal' yields no results.

based on my limited testing in the existing baked ami, if i run pip install --upgrade pip, the setup script appears to work then. the script complains about a 'zipp' module import failure, but installing that module doesn't appear to fix the issue. maybe the solution is to alter the marketplace_dl.sh script, after the python2-pip yum install to include a pip upgrade for things should work out.

[irvingpop]

I'm not sure if someone at Chef cleaned that up or not. You can create your own minimal image from the create_base_ami/packer.json definition here: https://github.com/irvingpop/packer-chef-highperf-centos-ami/tree/master/create_base_ami

[epackorigan]

After rebuilding all the images, it appears that it's functioning correctly now (at least it goes beyond that point).

[irvingpop]

FYI @thomascate @niamhcahill looks like those images are due for a rebuild

[epackorigan]

Also, the push-jobs server rpm v3.0.3 is 404...

…

On Mar 6, 2020, at 11:30 AM, Irving Popovetsky ***@***.***> wrote:  FYI @thomascate @niamhcahill looks like those images are due for a rebuild — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or unsubscribe.