Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pip install external/dv-toolkit/. #3

Closed
wwj-53 opened this issue Nov 15, 2023 · 19 comments
Closed

pip install external/dv-toolkit/. #3

wwj-53 opened this issue Nov 15, 2023 · 19 comments

Comments

@wwj-53
Copy link

wwj-53 commented Nov 15, 2023

Building wheels for collected packages: dv-toolkit
Building wheel for dv-toolkit (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [182 lines of output]

  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for dv-toolkit
Running setup.py clean for dv-toolkit
Failed to build dv-toolkit
ERROR: Could not build wheels for dv-toolkit, which is required to install pyproject.toml-based projects

@wwj-53
Copy link
Author

wwj-53 commented Nov 16, 2023

(emlb) wwj@wwj-virtual-machine:~/Downloads/cuke_emlb$ pip install external/dv-toolkit/.
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Processing ./external/dv-toolkit
  Preparing metadata (setup.py) ... done
Requirement already satisfied: numpy>=1.24 in /home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages (from dv-toolkit==0.1.0) (1.24.4)
Requirement already satisfied: dv_processing>=1.7 in /home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages (from dv-toolkit==0.1.0) (1.7.9)
Requirement already satisfied: plotly>=5.17 in /home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages (from dv-toolkit==0.1.0) (5.18.0)
Requirement already satisfied: matplotlib>=3.7 in /home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages (from dv-toolkit==0.1.0) (3.7.3)
Requirement already satisfied: contourpy>=1.0.1 in /home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages (from matplotlib>=3.7->dv-toolkit==0.1.0) (1.1.1)
Requirement already satisfied: cycler>=0.10 in /home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages (from matplotlib>=3.7->dv-toolkit==0.1.0) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages (from matplotlib>=3.7->dv-toolkit==0.1.0) (4.44.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages (from matplotlib>=3.7->dv-toolkit==0.1.0) (1.4.5)
Requirement already satisfied: packaging>=20.0 in /home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages (from matplotlib>=3.7->dv-toolkit==0.1.0) (23.2)
Requirement already satisfied: pillow>=6.2.0 in /home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages (from matplotlib>=3.7->dv-toolkit==0.1.0) (10.1.0)
Requirement already satisfied: pyparsing>=2.3.1 in /home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages (from matplotlib>=3.7->dv-toolkit==0.1.0) (3.1.1)
Requirement already satisfied: python-dateutil>=2.7 in /home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages (from matplotlib>=3.7->dv-toolkit==0.1.0) (2.8.2)
Requirement already satisfied: importlib-resources>=3.2.0 in /home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages (from matplotlib>=3.7->dv-toolkit==0.1.0) (6.1.1)
Requirement already satisfied: tenacity>=6.2.0 in /home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages (from plotly>=5.17->dv-toolkit==0.1.0) (8.2.3)
Requirement already satisfied: zipp>=3.1.0 in /home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages (from importlib-resources>=3.2.0->matplotlib>=3.7->dv-toolkit==0.1.0) (3.17.0)
Requirement already satisfied: six>=1.5 in /home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages (from python-dateutil>=2.7->matplotlib>=3.7->dv-toolkit==0.1.0) (1.16.0)
Building wheels for collected packages: dv-toolkit
  Building wheel for dv-toolkit (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [124 lines of output]
      Detected installation of g++10, it will be used for compilation
      running bdist_wheel
      running build
      running build_py
      running build_ext
      -- TOOLKIT_ENABLE_SAMPLES OFF
      -- TOOLKIT_ENABLE_PYTHON ON
      CMake Warning at /usr/share/cmake-3.22/Modules/FindBoost.cmake:1369 (message):
        New Boost version may have incorrect or missing dependencies and imported
        targets
      Call Stack (most recent call first):
        /usr/share/cmake-3.22/Modules/FindBoost.cmake:1492 (_Boost_COMPONENT_DEPENDENCIES)
        /usr/share/cmake-3.22/Modules/FindBoost.cmake:2102 (_Boost_MISSING_DEPENDENCIES)
        /usr/lib/x86_64-linux-gnu/cmake/dv-processing/dv-processing-config.cmake:29 (FIND_PACKAGE)
        CMakeLists.txt:23 (find_package)
      
      
      CMake Warning (dev) at /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
        The package name passed to `find_package_handle_standard_args` (PkgConfig)
        does not match the name of the calling package (dv-processing).  This can
        lead to problems in calling code that expects `find_package` result
        variables (e.g., `_FOUND`) to follow a certain pattern.
      Call Stack (most recent call first):
        /usr/share/cmake-3.22/Modules/FindPkgConfig.cmake:99 (find_package_handle_standard_args)
        /usr/lib/x86_64-linux-gnu/cmake/dv-processing/dv-processing-config.cmake:54 (INCLUDE)
        CMakeLists.txt:23 (find_package)
      This warning is for project developers.  Use -Wno-dev to suppress it.
      
      -- Linker flag --as-needed enabled
      -- Project name is: dv-toolkit
      -- Project version is: 0.1.0
      -- Build type is: Release
      -- Compiler is Clang: FALSE
      -- Compiler is GCC: TRUE
      -- Compiler is IntelCC: FALSE
      -- Compiler is MS VisualC: FALSE
      -- OS is Unix: TRUE
      -- OS is Linux: TRUE
      -- OS is MacOS: FALSE
      -- OS is Windows: FALSE
      -- System is big-endian: 0
      -- Thread support is PThreads: TRUE
      -- Thread support is Win32 Threads: FALSE
      -- Thread support libraries: -lpthread
      -- Current C flags are:
      -- Current CXX flags are:
      -- Final install bindir is: /usr/local/bin
      -- Final install libdir is: /usr/local/lib
      -- Final install includedir is: /usr/local/include
      -- pybind11 v2.11.1
      -- Configuring done
      CMake Error at /usr/share/cmake-3.22/Modules/FindPython/Support.cmake:3330 (add_library):
        Target "_lib_toolkit" links to target "PkgConfig::libzstd" but the target
        was not found.  Perhaps a find_package() call is missing for an IMPORTED
        target, or an ALIAS target is missing?
      Call Stack (most recent call first):
        /usr/share/cmake-3.22/Modules/FindPython3.cmake:494 (__Python3_add_library)
        python/external/pybind11/tools/pybind11NewTools.cmake:189 (python3_add_library)
        python/src/CMakeLists.txt:1 (pybind11_add_module)
      
      
      CMake Warning at /usr/share/cmake-3.22/Modules/FindPython/Support.cmake:3330 (add_library):
        Cannot generate a safe runtime search path for target _lib_toolkit because
        files in some directories may conflict with libraries in implicit
        directories:
      
          runtime library [liblz4.so.1] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
            /home/wwj/anaconda3/lib
      
        Some of these libraries may not be found correctly.
      Call Stack (most recent call first):
        /usr/share/cmake-3.22/Modules/FindPython3.cmake:494 (__Python3_add_library)
        python/external/pybind11/tools/pybind11NewTools.cmake:189 (python3_add_library)
        python/src/CMakeLists.txt:1 (pybind11_add_module)
      
      
      -- Generating done
      CMake Generate step failed.  Build files cannot be regenerated correctly.
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/home/wwj/Downloads/cuke_emlb/external/dv-toolkit/setup.py", line 188, in <module>
          setup(
        File "/home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
        File "/home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
        File "/home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 369, in run
          self.run_command("build")
        File "/home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 131, in run
          self.run_command(cmd_name)
        File "/home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 88, in run
          _build_ext.run(self)
        File "/home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
          self.build_extensions()
        File "/home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
          self._build_extensions_serial()
        File "/home/wwj/anaconda3/envs/emlb/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
          self.build_extension(ext)
        File "/home/wwj/Downloads/cuke_emlb/external/dv-toolkit/setup.py", line 132, in build_extension
          ret = subprocess.run(["cmake", ext.sourcedir, *cmake_args], cwd=build_temp, check=True)
        File "/home/wwj/anaconda3/envs/emlb/lib/python3.8/subprocess.py", line 516, in run
          raise CalledProcessError(retcode, process.args,
      subprocess.CalledProcessError: Command '['cmake', '/home/wwj/Downloads/cuke_emlb/external/dv-toolkit', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/wwj/Downloads/cuke_emlb/external/dv-toolkit/build/lib.linux-x86_64-cpython-38/dv_toolkit/lib', '-DPYTHON_EXECUTABLE=/home/wwj/anaconda3/envs/emlb/bin/python', '-DCMAKE_BUILD_TYPE=Release', '-DPython3_EXECUTABLE=/home/wwj/anaconda3/envs/emlb/bin/python', '-DTOOLKIT_ENABLE_SAMPLES=OFF', '-DTOOLKIT_ENABLE_PYTHON=ON', '-DCMAKE_C_COMPILER=gcc-10', '-DCMAKE_CXX_COMPILER=g++-10']' returned non-zero exit status 1.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for dv-toolkit
  Running setup.py clean for dv-toolkit
Failed to build dv-toolkit
ERROR: Could not build wheels for dv-toolkit, which is required to install pyproject.toml-based projects

@KugaMaxx
Copy link
Owner

Hi, @wwj-53! I have checked your error message and assume that you are missing zstd.

sudo apt-get install libzstd-dev

PS: If this doesn't work, please refer to the following steps to self-testing and send me the error message:

  • Enter the subfolder ./external/dv-toolkit/:

    cd external/dv-toolkit/
  • Build package with cmake

    sudo rm -r build
    mkdir build && cd build
    cmake .. -DTOOLKIT_ENABLE_PYTHON=ON
    make

If you have missing dependencies, errors will be reported during the compilation process.

@wwj-53
Copy link
Author

wwj-53 commented Nov 16, 2023

Hi, @wwj-53! I have checked your error message and assume that you are missing zstd.

sudo apt-get install libzstd-dev

PS: If this doesn't work, please refer to the following steps to self-testing and send me the error message:

  • Enter the subfolder ./external/dv-toolkit/:
    cd external/dv-toolkit/
  • Build package with cmake
    sudo rm -r build
    mkdir build && cd build
    cmake .. -DTOOLKIT_ENABLE_PYTHON=ON
    make

If you have missing dependencies, errors will be reported during the compilation process.

Thinks!But I confirm that libzstd-dev has been installed, and according to your instructions, I reported the following error on cmake:

CMake Error at /usr/lib/x86_64-linux-gnu/cmake/dv-processing/dv-processing-targets.cmake:66 (set_target_properties):
  The link interface of target "dv::processing" contains:

    PkgConfig::libzstd

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.

Call Stack (most recent call first):
  /usr/lib/x86_64-linux-gnu/cmake/dv-processing/dv-processing-config.cmake:119 (INCLUDE)
  CMakeLists.txt:23 (find_package)


-- Generating done
CMake Generate step failed.  Build files cannot be regenerated correctly.
  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.

Call Stack (most recent call first):
  /usr/lib/x86_64-linux-gnu/cmake/dv-processing/dv-processing-config.cmake:119 (INCLUDE)
  CMakeLists.txt:23 (find_package)


-- Generating done
CMake Generate step failed.  Build files cannot be regenerated correctly.

@wwj-53
Copy link
Author

wwj-53 commented Nov 16, 2023

I guess there may be a problem with the following paragraph, but I'm not sure and I don't know how to modify it

FIND_PACKAGE(zstd QUIET)
IF(NOT zstd_FOUND)
	# Search for external libraries with pkg-config.
	INCLUDE(FindPkgConfig)

	PKG_CHECK_MODULES(libzstd IMPORTED_TARGET libzstd)

	IF(NOT libzstd_FOUND)
		# Fall back to library path.
		FIND_LIBRARY(LIBZSTD zstd)
		IF(${LIBZSTD} STREQUAL "LIBZSTD-NOTFOUND")
			MESSAGE(FATAL_ERROR "zstd not found, required for I/O compression.")
		ENDIF()
	ENDIF()
ENDIF()

@KugaMaxx
Copy link
Owner

Coincidentally, I also encountered this error before, which is an issue in dv:: processing and it seems that there is currently no good solution.

I guess you are using ubuntu 22.04, so using the Ubuntu 20.04 can resolve this compile issue. You can check this links to view related discussions.


However, if it is inconvenient for you to switch to Ubuntu 20.04, let's try another possible solution by installing zstd manually.

  • Clone zstd

    git clone git@github.com:facebook/zstd.git
  • Build zstd to root directory

    cd zstd
    make
    sudo make install

Then try to enter ./external/dv-toolkit/ and build again. If this solution works, please also let me know. ;)

@wwj-53
Copy link
Author

wwj-53 commented Nov 16, 2023

Thank you very much for your timely response. However, my system is ubuntu20.04, and after implementing your suggestion, the problem has not been resolved and the same error is still reported.

@KugaMaxx
Copy link
Owner

KugaMaxx commented Nov 16, 2023

Hi, I think I just found two possible resolutions!

Solution 1: use pip install

  • Make sure you are in conda virtual environment, and use following command. After that, build dv-toolkit again.

    pip install dv-processing

Solution 2: set environment variable

  • Check where the PkgConfig::libzstd is (usually located in the /usr/lib/pkgconfig, /usr/local/lib/pkgconfig or /usr/lib/x86_64-linux-gnu/pkgconfig/ directory).

    locate libzstd.pc
  • Set the PKG_CONFIG_PATH environment variable by using following command. After that, build dv-toolkit again.

    export PKG_CONFIG_PATH=/path/to/pkgconfig:$PKG_CONFIG_PATH

    NOTE: Make sure to replace /path/to/pkgconfig with the actual path.

@wwj-53
Copy link
Author

wwj-53 commented Nov 16, 2023

Still can't do it.So sad

@KugaMaxx
Copy link
Owner

KugaMaxx commented Feb 29, 2024

The good news is, I finally found a solution a few months later. The steps are as follows:

  • Find the path of cmake-config for dv-processing, whose default directory is /usr/lib/x86_64-linux-gnu/cmake/dv-processing/dv-processing-config.cmake

    sudo vim /usr/lib/x86_64-linux-gnu/cmake/dv-processing/dv-processing-config.cmake
  • Insert the following code

    INCLUDE(FindPkgConfig)
    PKG_CHECK_MODULES(liblz4 IMPORTED_TARGET liblz4)
    PKG_CHECK_MODULES(libzstd IMPORTED_TARGET libzstd)

    cmake

  • Save and exit, then retry

    pip install external/dv-toolkit/.

@wwj-53
Copy link
Author

wwj-53 commented Mar 8, 2024

Thank you very much for your patience and help. The advice you provided has indeed solved my problem. You are a very patient person, thank you again

@hjvupnonon
Copy link

@hjvupnonon
Copy link

![图片](https://github.com/KugaMaxx/cuke-emlb/assets/130629331/3395aaa2-9fc5-4ff2- 微信图片_20240611165640

I'm not sure. Obviously I have the setup.py file, so I don't know why I'm reporting this error. If it's convenient, please help me.

@hjvupnonon
Copy link

The good news is, I finally found a solution a few months later. The steps are as follows:

  • Find the path of cmake-config for dv-processing, whose default directory is /usr/lib/x86_64-linux-gnu/cmake/dv-processing/dv-processing-config.cmake

    sudo vim /usr/lib/x86_64-linux-gnu/cmake/dv-processing/dv-processing-config.cmake
  • Insert the following code

    INCLUDE(FindPkgConfig)
    PKG_CHECK_MODULES(liblz4 IMPORTED_TARGET liblz4)
    PKG_CHECK_MODULES(libzstd IMPORTED_TARGET libzstd)

    cmake

  • Save and exit, then retry

    pip install external/dv-toolkit/.

I see your solution, but unfortunately I don't use Ubuntu and can't use the sudo command.

@KugaMaxx
Copy link
Owner

Hello, @hjvupnonon! I noticed that you didn't run pip install . in the external/dv-toolkit/.

@hjvupnonon
Copy link

1 I did as you said, but unfortunately, it still can't be resolved. In the dv-toolkit folder, the pyproject.toml-based project should be included, but I did not find it in your file. Could you help me solve this problem?

@KugaMaxx
Copy link
Owner

Maybe your environment has not installed setuptools yet [1]. Try the following command and then reinstall to see if it works:

pip install setuptools wheel

@hjvupnonon
Copy link

2

I'm very sorry, I tried your method, but still reported this error.

@KugaMaxx
Copy link
Owner

Hi, @hjvupnonon! Please try to pull the latest commit. It should resolve the issue you've been experiencing. Best of luck!

@hjvupnonon
Copy link

image
Thank you very much. You've been very patient. I downloaded your latest uploaded file on the website and ran it, but unfortunately, such an error is still reported at present.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants