[BUG] Not able to create virtualenv on AL2023 Single Host due to packaging pkg bug

[peterzhuamazon]

We only run python code on raw EC2 host when validating Docker, using Jenkins-Agent-AL2023-X64-C54xlarge-Single-Host.

It is currently on Python version 3.9.16 and failed to create virtualenv:

$ ./validation.sh --version 2.15.0 --distribution docker --arch x64 --projects opensearch opensearch-dashboards --docker-source dockerhub --using-staging-artifact-only
Installing dependencies in . ...
Creating a virtualenv for this project...
Pipfile: /home/ec2-user/opensearch-build/Pipfile
Using /usr/bin/python3 (3.9.16) to create virtualenv...
⠙ Creating virtual environment...
✘ Failed creating virtual environment
[pipenv.exceptions.VirtualenvCreationException]: Traceback (most recent call last):
  File "/usr/lib64/python3.9/runpy.py", line 188, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/usr/lib64/python3.9/runpy.py", line 147, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/usr/lib64/python3.9/runpy.py", line 111, in _get_module_details
    __import__(pkg_name)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/virtualenv/__init__.py", line 3, in <module>
    from .run import cli_run, session_via_cli
  File "/home/ec2-user/.local/lib/python3.9/site-packages/virtualenv/run/__init__.py", line 15, in <module>
    from .plugin.creators import CreatorSelector
  File "/home/ec2-user/.local/lib/python3.9/site-packages/virtualenv/run/plugin/creators.py", line 7, in <module>
    from virtualenv.create.via_global_ref.builtin.builtin_way import VirtualenvBuiltin
  File "/home/ec2-user/.local/lib/python3.9/site-packages/virtualenv/create/via_global_ref/builtin/builtin_way.py", line 5, in <module>
    from virtualenv.create.creator import Creator
  File "/home/ec2-user/.local/lib/python3.9/site-packages/virtualenv/create/creator.py", line 13, in <module>
    from virtualenv.discovery.cached_py_info import LogCmd
  File "/home/ec2-user/.local/lib/python3.9/site-packages/virtualenv/discovery/cached_py_info.py", line 25, in <module>
    _CACHE[Path(sys.executable)] = PythonInfo()
  File "/home/ec2-user/.local/lib/python3.9/site-packages/virtualenv/discovery/py_info.py", line 100, in __init__
    self.distutils_install = self._distutils_install().copy()
  File "/home/ec2-user/.local/lib/python3.9/site-packages/virtualenv/discovery/py_info.py", line 191, in _distutils_install
    i.finalize_options()
  File "/home/ec2-user/.local/lib/python3.9/site-packages/setuptools/command/install.py", line 62, in finalize_options
    super().finalize_options()
  File "/home/ec2-user/.local/lib/python3.9/site-packages/setuptools/_distutils/command/install.py", line 407, in finalize_options
    'dist_fullname': self.distribution.get_fullname(),
  File "/home/ec2-user/.local/lib/python3.9/site-packages/setuptools/_core_metadata.py", line 266, in get_fullname
    return _distribution_fullname(self.get_name(), self.get_version())
  File "/home/ec2-user/.local/lib/python3.9/site-packages/setuptools/_core_metadata.py", line 284, in _distribution_fullname
    canonicalize_version(version, strip_trailing_zero=False),
TypeError: canonicalize_version() got an unexpected keyword argument 'strip_trailing_zero'

Issue is likely the same as this:

• [BUG] TypeError: canonicalize_version() got an unexpected keyword argument 'strip_trailing_zero' pypa/setuptools#4483
• [BUG] TypeError: canonicalize_version() got an unexpected keyword argument 'strip_trailing_zero' with setuptools 71.x pypa/setuptools#4501
• python3 setup.py build_ext -i not working chipsec/chipsec#2230

Would need to update the packaging package to at least v22 to resolve it.

• Fix compatible version specifier incorrectly strip trailing '0' pypa/packaging#493

$ ./validation.sh --version 2.15.0 --distribution docker --arch x64 --projects opensearch opensearch-dashboards --docker-source dockerhub --using-staging-artifact-only
Installing dependencies in . ...
Creating a virtualenv for this project...
Pipfile: /home/ec2-user/opensearch-build/Pipfile
Using /usr/bin/python3 (3.9.16) to create virtualenv...
⠴ Creating virtual environment...created virtual environment CPython3.9.16.final.0-64 in 291ms
  creator CPython3Posix(dest=/home/ec2-user/.local/share/virtualenvs/opensearch-build-6InJtTzb, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/ec2-user/.local/share/virtualenv)
    added seed packages: pip==24.1, setuptools==70.1.0, wheel==0.43.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

✔ Successfully created virtual environment!
Virtualenv location: /home/ec2-user/.local/share/virtualenvs/opensearch-build-6InJtTzb
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
Installing dependencies from Pipfile.lock (6b098c)...

Thanks.