No such file or directory: '/usr/local/lib/python3.8/dist-packages/setuptools-68.0.0.dist-info/METADATA

[seanthegeek]

thug fails to install when attempting to upgrade from REMnux v2023.20.1 to v2023.26.1.

pip_|-remnux-python3-packages-thug-packages_|-thug_|-installed:
    __id__: remnux-python3-packages-thug-packages
    __run_num__: 342
    __sls__: remnux.python3-packages.thug
    changes: {}
    comment: "Failed to install packages: thug. Error: Requirement already satisfied:
      thug in /usr/local/lib/python3.8/dist-packages (5.0.1)\nCollecting thug\n  Using
      cached thug-5.2.tar.gz (185 kB)\n  Installing build dependencies: started\n
      \ Installing build dependencies: finished with status 'done'\n  Getting requirements
      to build wheel: started\n  Getting requirements to build wheel: finished with
      status 'done'\n  Preparing metadata (pyproject.toml): started\n  Preparing metadata
      (pyproject.toml): finished with status 'done'\nRequirement already satisfied:
      networkx==3.1 in /usr/local/lib/python3.8/dist-packages (from thug) (3.1)\nRequirement
      already satisfied: yara-python==4.3.1 in /usr/local/lib/python3.8/dist-packages
      (from thug) (4.3.1)\nRequirement already satisfied: setuptools>=65.5.1 in /usr/local/lib/python3.8/dist-packages
      (from thug) (68.0.0)\nRequirement already satisfied: pymongo==4.3.3 in /usr/local/lib/python3.8/dist-packages
      (from thug) (4.3.3)\nRequirement already satisfied: zope.interface==6.0 in /usr/local/lib/python3.8/dist-packages
      (from thug) (6.0)\nRequirement already satisfied: pefile==2023.2.7 in /usr/local/lib/python3.8/dist-packages
      (from thug) (2023.2.7)\nRequirement already satisfied: html5lib==1.1 in /usr/local/lib/python3.8/dist-packages
      (from thug) (1.1)\nRequirement already satisfied: pysocks==1.7.1 in /usr/local/lib/python3.8/dist-packages
      (from thug) (1.7.1)\nRequirement already satisfied: requests==2.31.0 in /usr/local/lib/python3.8/dist-packages
      (from thug) (2.31.0)\nCollecting cssutils==2.7.1\n  Using cached cssutils-2.7.1-py3-none-any.whl
      (399 kB)\nRequirement already satisfied: ssdeep==3.4 in /usr/local/lib/python3.8/dist-packages
      (from thug) (3.4)\nRequirement already satisfied: lxml==4.9.2 in /usr/local/lib/python3.8/dist-packages
      (from thug) (4.9.2)\nRequirement already satisfied: pylibemu==0.8 in /usr/local/lib/python3.8/dist-packages
      (from thug) (0.8)\nRequirement already satisfied: appdirs==1.4.4 in /usr/local/lib/python3.8/dist-packages
      (from thug) (1.4.4)\nCollecting python-magic==0.4.27\n  Using cached python_magic-0.4.27-py2.py3-none-any.whl
      (13 kB)\nCollecting charset-normalizer==3.1.0\n  Using cached charset_normalizer-3.1.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
      (195 kB)\nRequirement already satisfied: beautifulsoup4==4.12.2 in /usr/local/lib/python3.8/dist-packages
      (from thug) (4.12.2)\nRequirement already satisfied: rarfile==4.0 in /usr/local/lib/python3.8/dist-packages
      (from thug) (4.0)\nCollecting requests-futures==1.0.1\n  Using cached requests_futures-1.0.1-py2.py3-none-any.whl
      (7.6 kB)\nRequirement already satisfied: promise==2.3 in /usr/local/lib/python3.8/dist-packages
      (from thug) (2.3)\nRequirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.8/dist-packages
      (from beautifulsoup4==4.12.2->thug) (2.3.1)\nRequirement already satisfied:
      six>=1.9 in /usr/local/lib/python3.8/dist-packages (from html5lib==1.1->thug)
      (1.16.0)\nRequirement already satisfied: webencodings in /usr/local/lib/python3.8/dist-packages
      (from html5lib==1.1->thug) (0.5.1)\nRequirement already satisfied: dnspython<3.0.0,>=1.16.0
      in /usr/local/lib/python3.8/dist-packages (from pymongo==4.3.3->thug) (2.0.0)\nRequirement
      already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.8/dist-packages
      (from requests==2.31.0->thug) (2.0.3)\nRequirement already satisfied: certifi>=2017.4.17
      in /usr/local/lib/python3.8/dist-packages (from requests==2.31.0->thug) (2023.5.7)\nRequirement
      already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.8/dist-packages (from
      requests==2.31.0->thug) (3.4)\nRequirement already satisfied: cffi>=0.8.6 in
      /usr/local/lib/python3.8/dist-packages (from ssdeep==3.4->thug) (1.15.1)\nRequirement
      already satisfied: pycparser in /usr/local/lib/python3.8/dist-packages (from
      cffi>=0.8.6->ssdeep==3.4->thug) (2.21) ERROR: Could not install packages due
      to an OSError: [Errno 2] No such file or directory: '/usr/local/lib/python3.8/dist-packages/setuptools-68.0.0.dist-info/METADATA'"
    duration: 6357.627
    name: thug
    result: false

[digitalsleuth]

Hi @seanthegeek , can you run the following and provide the output?

sudo python3 -m pip -V

[seanthegeek]

pip 22.0.4 from /usr/local/lib/python3.8/dist-packages/pip (python 3.8)

[seanthegeek]

I noticed that setuptools.sls specifies setuptools<66.0.0, but my system was running 68.0.0. When I tried running sudo pip install setuptools<66.0.0, I got some helpful output.

Collecting setuptools<66.0.0
  Using cached setuptools-65.7.0-py3-none-any.whl (1.2 MB)
WARNING: Error parsing requirements for setuptools: [Errno 2] No such file or directory: '/usr/local/lib/python3.8/dist-packages/setuptools-68.0.0.dist-info/METADATA'
Installing collected packages: setuptools
  Attempting uninstall: setuptools
    WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages
    Found existing installation: setuptools 68.0.0
    WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages
    WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages
    WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages
    WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages
    WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages
ERROR: Cannot uninstall setuptools 68.0.0, RECORD file not found. You might be able to recover from this via: 'pip install --force-reinstall --no-deps setuptools==68.0.0'.

Running sudo pip install --upgrade --force-reinstall --no-deps setuptools<66.0.0 solved the problem. I'll submit a PR to use that fix.

[seanthegeek]

Never mind. All I had to do was upgrade to the latest version of pip.

[digitalsleuth]

Hi @seanthegeek , sorry for the delay in getting back to you on this. I was out of town for the last few days, and have had terrible internet connectivity since I've returned. It seems you've already come to the same conclusion I have, but I haven't had the opportunity to test it properly (due to the aforementioned internet issues). I'll take a closer look at this and see if this needs to be "forcefully" fixed to ensure there are no issues between previous and existing issues.

Thanks for this, and sorry again for the delay!

[seanthegeek]

@digitalsleuth I think the only thing that needs changed is rasing the minimum version of pip. the setuptools package is fine as-is with the latest version of pip.

[digitalsleuth]

Sounds reasonable. To avoid potential issues introduced with future versions of pip, I'll pin it to the current latest version, then test regularly and update over reasonable intervals. Cheers!