Failure to install due to ssdeep build issue

[diegojorquera]

A transitive dependency on https://pypi.org/project/ssdeep/ was recently introduced. Unfortunately, on both macOS and Linux, that package fails to build. Example output log included below.

Collecting ssdeep (from oslili->xmonkey-namonica)
  Downloading ssdeep-3.4.tar.gz (110 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 110.8/110.8 kB 13.0 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [129 lines of output]
      <string>:8: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
      running dist_info
      creating /tmp/pip-modern-metadata-kya58ovu/ssdeep.egg-info
      writing /tmp/pip-modern-metadata-kya58ovu/ssdeep.egg-info/PKG-INFO
      writing dependency_links to /tmp/pip-modern-metadata-kya58ovu/ssdeep.egg-info/dependency_links.txt
      writing requirements to /tmp/pip-modern-metadata-kya58ovu/ssdeep.egg-info/requires.txt
      writing top-level names to /tmp/pip-modern-metadata-kya58ovu/ssdeep.egg-info/top_level.txt
      writing manifest file '/tmp/pip-modern-metadata-kya58ovu/ssdeep.egg-info/SOURCES.txt'
      src/ssdeep/__pycache__/_ssdeep_cffi_a28e5628x27adcb8d.c:266:14: fatal error: fuzzy.h: No such file or directory
           #include "fuzzy.h"
                    ^~~~~~~~~
      compilation terminated.
      Traceback (most recent call last):
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/_distutils/unixccompiler.py", line 188, in _compile
          self.spawn(compiler_so + cc_args + [src, '-o', obj] + extra_postargs)
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/_distutils/ccompiler.py", line 1041, in spawn
          spawn(cmd, dry_run=self.dry_run, **kwargs)
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/_distutils/spawn.py", line 68, in spawn
          raise DistutilsExecError(f"command {cmd!r} failed with exit code {exitcode}")
      distutils.errors.DistutilsExecError: command '/usr/bin/gcc' failed with exit code 1

      During handling of the above exception, another exception occurred:

      Traceback (most recent call last):
        File "/tmp/pip-build-env-1ct6lat3/normal/lib/python3.12/site-packages/cffi/ffiplatform.py", line 48, in _build
          dist.run_command('build_ext')
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 967, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/command/build_ext.py", line 91, in run
          _build_ext.run(self)
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
          self.build_extensions()
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 479, in build_extensions
          self._build_extensions_serial()
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 505, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/command/build_ext.py", line 252, in build_extension
          _build_ext.build_extension(self, ext)
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 560, in build_extension
          objects = self.compiler.compile(
                    ^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/_distutils/ccompiler.py", line 600, in compile
          self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/_distutils/unixccompiler.py", line 190, in _compile
          raise CompileError(msg)
      distutils.errors.CompileError: command '/usr/bin/gcc' failed with exit code 1

      During handling of the above exception, another exception occurred:

      Traceback (most recent call last):
        File "/local/home/tapdiego/xmonkey-namonica/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/local/home/tapdiego/xmonkey-namonica/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/local/home/tapdiego/xmonkey-namonica/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
          return hook(metadata_directory, config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 366, in prepare_metadata_for_build_wheel
          self.run_setup()
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 487, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 108, in <module>
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 104, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 184, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
          dist.run_commands()
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 967, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/command/dist_info.py", line 92, in run
          self.egg_info.run()
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 321, in run
          self.find_sources()
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 329, in find_sources
          mm.run()
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 550, in run
          self.add_defaults()
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 588, in add_defaults
          sdist.add_defaults(self)
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/command/sdist.py", line 102, in add_defaults
          super().add_defaults()
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/sdist.py", line 248, in add_defaults
          self._add_defaults_python()
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/command/sdist.py", line 113, in _add_defaults_python
          build_py = self.get_finalized_command('build_py')
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 303, in get_finalized_command
          cmd_obj.ensure_finalized()
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 111, in ensure_finalized
          self.finalize_options()
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/command/build_py.py", line 39, in finalize_options
          orig.build_py.finalize_options(self)
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/build_py.py", line 46, in finalize_options
          self.set_undefined_options(
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 291, in set_undefined_options
          src_cmd_obj.ensure_finalized()
        File "/tmp/pip-build-env-1ct6lat3/overlay/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 111, in ensure_finalized
          self.finalize_options()
        File "<string>", line 24, in finalize_options
        File "<string>", line 79, in get_ext_modules
        File "/tmp/pip-install-1nraccee/ssdeep_a105c459a38b4596ac5fec154c15b24a/src/ssdeep/binding.py", line 126, in verify
          self._lib = self.ffi.verify(
                      ^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-1ct6lat3/normal/lib/python3.12/site-packages/cffi/api.py", line 468, in verify
          lib = self.verifier.load_library()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-1ct6lat3/normal/lib/python3.12/site-packages/cffi/verifier.py", line 105, in load_library
          self._compile_module()
        File "/tmp/pip-build-env-1ct6lat3/normal/lib/python3.12/site-packages/cffi/verifier.py", line 201, in _compile_module
          outputfilename = ffiplatform.compile(tmpdir, self.get_extension())
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-1ct6lat3/normal/lib/python3.12/site-packages/cffi/ffiplatform.py", line 20, in compile
          outputfilename = _build(tmpdir, ext, compiler_verbose, debug)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-1ct6lat3/normal/lib/python3.12/site-packages/cffi/ffiplatform.py", line 54, in _build
          raise VerificationError('%s: %s' % (e.__class__.__name__, e))
      cffi.VerificationError: CompileError: command '/usr/bin/gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[oscarvalenzuelab]

This is a well known issue, here is the solution: Xpertians/xmonkey-curator#6

[oscarvalenzuelab]

brew install libtool automake
ln -s /opt/homebrew/bin/glibtoolize /opt/homebrew/bin/libtoolize
BUILD_LIB=1 pip3 install ssdeep

[oscarvalenzuelab]

Incorporated solution into the readme as part of the troubleshooting guide.