Add libtorch #169

PinkySan · 2019-10-02T20:07:37Z

Specify library name and version: lib/1.0

I've read the guidelines for contributing.
I've followed the PEP8 style guides for Python code in the recipes.
I've used the latest Conan client version.
I've tried at least one configuration locally with the
conan-center hook activated.
What should be inside the test_package folder?
conandata.yml? I am creating the url within the recipe

CLAassistant · 2019-10-02T20:08:14Z

All committers have signed the CLA.

conan-center-bot · 2019-10-02T20:08:17Z

Sorry, the build is only launched for Early Access Program users. You can request access writing in this issue.

conan-center-bot · 2019-10-02T23:17:29Z

Sorry, the build is only launched for Early Access Program users. You can request access writing in this issue.

recipes/libtorch/1.2.0/conanfile.py

conan-center-bot · 2019-10-05T11:50:41Z

Sorry, the build is only launched for Early Access Program users. You can request access writing in this issue.

conan-center-bot · 2019-10-10T07:43:11Z

Some configurations of 'libtorch/1.2.0' have failed:

Linux x86_64, Release, gcc 5, libstdc++
- Hooks errors detected:
  - [HOOK - conan-center.py] pre_export(): ERROR: [RECIPE METADATA (KB-H003)] Conanfile doesn't have 'homepage' attribute. (https://github.com/conan-io/conan-center-index/wiki/Error-Knowledge-Base#KB-H003)
  - [HOOK - conan-center.py] pre_export(): ERROR: [TEST PACKAGE FOLDER (KB-H024)] There is no 'test_package' for this recipe (https://github.com/conan-io/conan-center-index/wiki/Error-Knowledge-Base#KB-H024)
  - [HOOK - conan-center.py] pre_export(): ERROR: [CONAN CENTER INDEX URL (KB-H027)] The attribute 'url' should point to: https://github.com/conan-io/conan-center-index (https://github.com/conan-io/conan-center-index/wiki/Error-Knowledge-Base#KB-H027)
- Log download
- Log error Download
- Profile
Access to all the logs

uilianries · 2019-10-21T21:04:39Z

I've checked this recipe and it's complex ... To build it from source, it requires other libraries. I've tried to create a generic version for any version here: uilianries@3b007a8

conan-center-bot · 2019-10-31T08:34:04Z

Some configurations of 'libtorch/1.2.0' have failed:

Linux x86_64, Release, gcc 5, libstdc++
- Hooks errors detected:
  - [HOOK - conan-center.py] pre_export(): ERROR: [RECIPE METADATA (KB-H003)] Conanfile doesn't have 'homepage' attribute. (https://github.com/conan-io/conan-center-index/wiki/Error-Knowledge-Base#KB-H003)
  - [HOOK - conan-center.py] pre_export(): ERROR: [TEST PACKAGE FOLDER (KB-H024)] There is no 'test_package' for this recipe (https://github.com/conan-io/conan-center-index/wiki/Error-Knowledge-Base#KB-H024)
  - [HOOK - conan-center.py] pre_export(): ERROR: [CONAN CENTER INDEX URL (KB-H027)] The attribute 'url' should point to: https://github.com/conan-io/conan-center-index (https://github.com/conan-io/conan-center-index/wiki/Error-Knowledge-Base#KB-H027)
- Log download
- Log error Download
- Profile
Access to all the logs

SSE4 · 2019-11-20T13:10:21Z

@uilianries may you create a PR to author's repo with your changes? (+ hook fixes)

uilianries · 2019-11-20T13:27:37Z

@SSE4 Niet, but I should.

merge conan-io into my fork

conan-center-bot · 2019-11-26T20:46:56Z

Some configurations of 'libtorch/1.2.0' have failed:

Linux x86_64, Release, gcc 5, libstdc++
- Hooks errors detected:
  - [HOOK - conan-center.py] pre_export(): ERROR: [RECIPE METADATA (KB-H003)] Conanfile doesn't have 'homepage' attribute. (https://github.com/conan-io/conan-center-index/wiki/Error-Knowledge-Base#KB-H003)
  - [HOOK - conan-center.py] pre_export(): ERROR: [TEST PACKAGE FOLDER (KB-H024)] There is no 'test_package' for this recipe (https://github.com/conan-io/conan-center-index/wiki/Error-Knowledge-Base#KB-H024)
  - [HOOK - conan-center.py] pre_export(): ERROR: [CONAN CENTER INDEX URL (KB-H027)] The attribute 'url' should point to: https://github.com/conan-io/conan-center-index (https://github.com/conan-io/conan-center-index/wiki/Error-Knowledge-Base#KB-H027)
- Log download
- Log error Download
- Profile
Access to all the logs

conan-center-bot · 2019-11-26T20:59:44Z

Some configurations of 'libtorch/1.2.0' have failed:

Linux x86_64, Release, gcc 5, libstdc++
- Hooks errors detected:
  - [HOOK - conan-center.py] pre_export(): ERROR: [RECIPE METADATA (KB-H003)] Conanfile doesn't have 'url' attribute. (https://github.com/conan-io/conan-center-index/wiki/Error-Knowledge-Base#KB-H003)
  - [HOOK - conan-center.py] pre_export(): ERROR: [TEST PACKAGE FOLDER (KB-H024)] There is no 'test_package' for this recipe (https://github.com/conan-io/conan-center-index/wiki/Error-Knowledge-Base#KB-H024)
- Log download
- Log error Download
- Profile
Access to all the logs

PinkySan · 2019-12-06T10:05:32Z

There is a way of installing pytorch from sources
https://github.com/pytorch/pytorch#from-source

How to install dependent python packages within conanfile.py?
Does this include the libtorch packages?

conan-center-bot · 2019-12-20T08:50:53Z

Some configurations of 'libtorch/1.2.0' have failed:

Linux x86_64, Release, gcc 5, libstdc++
- Hooks errors detected:
  - [HOOK - conan-center.py] pre_export(): ERROR: [RECIPE METADATA (KB-H003)] Conanfile doesn't have 'url' attribute. (https://github.com/conan-io/conan-center-index/wiki/Error-Knowledge-Base#KB-H003)
  - [HOOK - conan-center.py] pre_export(): ERROR: [TEST PACKAGE FOLDER (KB-H024)] There is no 'test_package' for this recipe (https://github.com/conan-io/conan-center-index/wiki/Error-Knowledge-Base#KB-H024)
- Log download
- Log error Download
- Profile
Access to all the logs

danimtb · 2019-12-20T08:56:07Z

@PinkySan there is no way to install additional dependencies from python. All the resources should be in the recipe.

There are some things missing in this PR, like the test_package recipe and the missing attributes

PinkySan · 2019-12-20T10:17:26Z

@PinkySan there is no way to install additional dependencies from python. All the resources should be in the recipe.

There are some things missing in this PR, like the test_package recipe and the missing attributes

Unfortunatly, that is true.
I do not really understand, why they are trying to build the package by their own. Providing the binaries should be enough. (I am having the same difficulties with OpenCV)

forwardmeasure · 2019-12-20T18:55:51Z

@PinkySan, for what it's worth, I have put together a Conan recipe to build Libtorch from scratch. It works, but not consistently, often failing on this error:

( "You found a build system error: AVX512F, AVX512DQ, AVX512VL " "is defined (via e.g. -mavx512f, -mavx512dq, and -mavx512vl) " "but CAFFE2_PERF_WITH_AVX512 is not defined.")

You can find the recipe here: https://github.com/forwardmeasure/conan/tree/master/recipes/conan-libtorch

Regards,
Prashanth

conan-center-bot · 2020-01-15T16:51:40Z

Some configurations of 'libtorch/1.2.0' failed in build 9 (62b282312b6334c4a4799841ea9da8583cb26a90):

Linux x86_64, Release, gcc 5, libstdc++
- Hooks errors detected:
  - [HOOK - conan-center.py] pre_export(): ERROR: [RECIPE METADATA (KB-H003)] Conanfile doesn't have 'url' attribute. (https://github.com/conan-io/conan-center-index/wiki/Error-Knowledge-Base#KB-H003)
  - [HOOK - conan-center.py] pre_export(): ERROR: [TEST PACKAGE FOLDER (KB-H024)] There is no 'test_package' for this recipe (https://github.com/conan-io/conan-center-index/wiki/Error-Knowledge-Base#KB-H024)
- Log download
- Log error Download
- Profile
Access to all the logs

bmanga · 2020-01-24T14:35:58Z

For what it's worth, I also have a WIP implementation of a libtorch recipe that reuses some packages: https://github.com/bmanga/conan-libtorch

forwardmeasure · 2020-01-24T17:16:28Z

For what it's worth, I also have a WIP implementation of a libtorch recipe that reuses some packages: https://github.com/bmanga/conan-libtorch

Thanks, Bruno. I got mine working soon after posting. Cheers!

conan-center-bot · 2020-03-05T09:42:17Z

Some configurations of 'libtorch/1.2.0' failed in build 10 (62b282312b6334c4a4799841ea9da8583cb26a90):

Linux x86_64, Release, gcc 5, libstdc++
- Hooks errors detected:
  - [HOOK - conan-center.py] pre_export(): ERROR: [RECIPE METADATA (KB-H003)] Conanfile doesn't have 'url' attribute. (https://github.com/conan-io/conan-center-index/wiki/Error-Knowledge-Base#KB-H003)
  - [HOOK - conan-center.py] pre_export(): ERROR: [TEST PACKAGE FOLDER (KB-H024)] There is no 'test_package' for this recipe (https://github.com/conan-io/conan-center-index/wiki/Error-Knowledge-Base#KB-H024)
- Log download
- Log error Download
- Profile
Access to all the logs

danimtb · 2020-03-16T09:27:42Z

@PinkySan please address the issues reported above by the bot, otherwise the PR will be closed. Thanks!

PinkySan · 2020-03-16T11:00:51Z

I guess this solution works better. Please close this PR

For what it's worth, I also have a WIP implementation of a libtorch recipe that reuses some packages: https://github.com/bmanga/conan-libtorch

bmanga · 2020-03-16T11:34:45Z

For what it's worth, I'm planning to cleanup my version and propose a barebone version for inclusion soon-ish. I think it would be good to converge and work on a single attempt (there seem to be at least 4 versions based on this PR alone)

remove gcc4.8, gcc9, windows-debug jobs

blackliner · 2020-12-16T13:31:17Z

@bmanga any updates on this? I did not find another open PR for libtorch, and as we migrate to conan, we might need it soon. Willing to help out 👍

madebr · 2020-03-12T09:02:38Z

recipes/libtorch/1.2.0/conanfile.py

+    settings = {"os": ["Windows","Linux"],
+        "build_type": ["Debug", "Release", "RelWithDebInfo"],
+        "compiler": {"gcc": {"version": None,
+                            "libcxx": ["libstdc++", "libstdc++11"]},
+                    "Visual Studio": {"version": None},
+                    }
+        }


See conan-io/conan#6534

madebr · 2020-03-12T09:03:00Z

recipes/libtorch/1.2.0/conanfile.py

+    url_base = "https://download.pytorch.org/libtorch/"
+
+
+    def getGPU(self):


Suggested change

def getGPU(self):

def _getGPU(self):

madebr · 2020-03-12T09:03:48Z

recipes/libtorch/1.2.0/conanfile.py

+    def getOS(self):
+        if(self.settings.os == "Windows"):
+            if(self.settings.build_type == "Debug"):
+                return "libtorch-win-shared-with-deps-debug-"
+            return "libtorch-win-shared-with-deps-"
+        if(self.settings.os == "Linux"):
+            if(self.settings.compiler.libcxx == "libstdc++11"):
+                return "libtorch-cxx11-abi-shared-with-deps-"
+            return "libtorch-shared-with-deps-"


Suggested change

def getOS(self):

if(self.settings.os == "Windows"):

if(self.settings.build_type == "Debug"):

return "libtorch-win-shared-with-deps-debug-"

return "libtorch-win-shared-with-deps-"

if(self.settings.os == "Linux"):

if(self.settings.compiler.libcxx == "libstdc++11"):

return "libtorch-cxx11-abi-shared-with-deps-"

return "libtorch-shared-with-deps-"

def _getOS(self):

if self.settings.os == "Windows":

if self.settings.build_type == "Debug":

return "libtorch-win-shared-with-deps-debug-"

return "libtorch-win-shared-with-deps-"

if self.settings.os == "Linux":

if self.settings.compiler.libcxx == "libstdc++11":

return "libtorch-cxx11-abi-shared-with-deps-"

return "libtorch-shared-with-deps-"

madebr · 2020-03-12T09:04:00Z

recipes/libtorch/1.2.0/conanfile.py

+            return "libtorch-shared-with-deps-"
+
+    def source(self):
+        url = self.url_base + self.getGPU() + self.getOS() + self.version + ".zip"


Suggested change

url = self.url_base + self.getGPU() + self.getOS() + self.version + ".zip"

url = self.url_base + self._getGPU() + self._getOS() + self.version + ".zip"

SpaceIm · 2021-01-30T22:51:56Z

I'm working on a pytorch recipe, but if someone has a good recipe almost ready for a PR, I'll be happy to help.

blackliner · 2021-01-31T12:42:28Z

I am living with distributing binaries for now :-/

from conans import ConanFile, tools


class LibtorchConan(ConanFile):
    name = "libtorch"
    settings = "os", "arch"
    url = "https://github.com/pytorch/pytorch"
    description = "An open source machine learning framework that accelerates the path from research prototyping to production deployment."
    license = "All rights reserved."
    topics = ("computer-vision", "meta-data")

    def build(self):
        tools.get(**self.conan_data["binaries"][self.version][str(self.settings.arch)])

    def package(self):
        self.copy("*", src=self.name)

    def package_info(self):
        self.cpp_info.libs = tools.collect_libs(self)

conandata.yml:

binaries:
  1.4.0:
    x86_64:
      url: https://download.pytorch.org/libtorch/cu101/libtorch-cxx11-abi-shared-with-deps-1.4.0.zip
      sha256: f214bfde532877aa5d4e0803e51a28fa8edd97b6a44b6615f75a70352b6b542e

PinkySan · 2021-02-17T19:35:41Z

@blackliner That is the same idea, i had at the beginning of this PR. I guess that is the best solution

blackliner · 2021-02-17T19:45:57Z

Yes, I guess some things are not meant to be built from scratch :D

I forgot to mention, to use my previous recipe, you need to use the cmake_paths generator in your root recipe and include the created file:

if(EXISTS "${CMAKE_BINARY_DIR}/conan_paths.cmake")
  include("${CMAKE_BINARY_DIR}/conan_paths.cmake")
endif()

First version of libtorch recipe

213bf32

conan-center-bot added the No Beta user label Oct 2, 2019

This was referenced Oct 2, 2019

[!!] Access request #4

Open

LibTorch conan-io/wishlist#187

Closed

Croydon reviewed Oct 5, 2019

View reviewed changes

recipes/libtorch/1.2.0/conanfile.py Outdated Show resolved Hide resolved

Update after review

735baf1

PinkySan changed the title ~~First version of libtorch recipe~~ Add libtorch Oct 5, 2019

lasote removed the No Beta user label Oct 10, 2019

conan-center-bot added the Failed label Oct 10, 2019

Merge pull request #1 from conan-io/master

fca2d07

merge conan-io into my fork

PinkySan added 2 commits November 26, 2019 21:55

adding basic conandata.yml

68970ec

updating conanfile.py

62b2823

PinkySan closed this Mar 16, 2020

artem-kamyshev pushed a commit to artem-kamyshev/conan-center-index that referenced this pull request Sep 28, 2020

Merge pull request conan-io#169 from trassir/rm-excessive

cf59f5b

remove gcc4.8, gcc9, windows-debug jobs

madebr reviewed Dec 16, 2020

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add libtorch #169

Add libtorch #169

PinkySan commented Oct 2, 2019

CLAassistant commented Oct 2, 2019 •

edited

conan-center-bot commented Oct 2, 2019

conan-center-bot commented Oct 2, 2019

conan-center-bot commented Oct 5, 2019

conan-center-bot commented Oct 10, 2019

uilianries commented Oct 21, 2019

conan-center-bot commented Oct 31, 2019

SSE4 commented Nov 20, 2019

uilianries commented Nov 20, 2019

conan-center-bot commented Nov 26, 2019

conan-center-bot commented Nov 26, 2019

PinkySan commented Dec 6, 2019

conan-center-bot commented Dec 20, 2019

danimtb commented Dec 20, 2019

PinkySan commented Dec 20, 2019

forwardmeasure commented Dec 20, 2019

conan-center-bot commented Jan 15, 2020

bmanga commented Jan 24, 2020

forwardmeasure commented Jan 24, 2020

conan-center-bot commented Mar 5, 2020

danimtb commented Mar 16, 2020

PinkySan commented Mar 16, 2020

bmanga commented Mar 16, 2020

blackliner commented Dec 16, 2020

madebr Mar 12, 2020

madebr Mar 12, 2020

madebr Mar 12, 2020

madebr Mar 12, 2020

SpaceIm commented Jan 30, 2021 •

edited

blackliner commented Jan 31, 2021

PinkySan commented Feb 17, 2021

blackliner commented Feb 17, 2021 •

edited

		url_base = "https://download.pytorch.org/libtorch/"


		def getGPU(self):

	url = self.url_base + self.getGPU() + self.getOS() + self.version + ".zip"
	url = self.url_base + self._getGPU() + self._getOS() + self.version + ".zip"

Add libtorch #169

Add libtorch #169

Conversation

PinkySan commented Oct 2, 2019

CLAassistant commented Oct 2, 2019 • edited

conan-center-bot commented Oct 2, 2019

conan-center-bot commented Oct 2, 2019

conan-center-bot commented Oct 5, 2019

conan-center-bot commented Oct 10, 2019

uilianries commented Oct 21, 2019

conan-center-bot commented Oct 31, 2019

SSE4 commented Nov 20, 2019

uilianries commented Nov 20, 2019

conan-center-bot commented Nov 26, 2019

conan-center-bot commented Nov 26, 2019

PinkySan commented Dec 6, 2019

conan-center-bot commented Dec 20, 2019

danimtb commented Dec 20, 2019

PinkySan commented Dec 20, 2019

forwardmeasure commented Dec 20, 2019

conan-center-bot commented Jan 15, 2020

bmanga commented Jan 24, 2020

forwardmeasure commented Jan 24, 2020

conan-center-bot commented Mar 5, 2020

danimtb commented Mar 16, 2020

PinkySan commented Mar 16, 2020

bmanga commented Mar 16, 2020

blackliner commented Dec 16, 2020

madebr Mar 12, 2020

Choose a reason for hiding this comment

madebr Mar 12, 2020

Choose a reason for hiding this comment

madebr Mar 12, 2020

Choose a reason for hiding this comment

madebr Mar 12, 2020

Choose a reason for hiding this comment

SpaceIm commented Jan 30, 2021 • edited

blackliner commented Jan 31, 2021

PinkySan commented Feb 17, 2021

blackliner commented Feb 17, 2021 • edited

CLAassistant commented Oct 2, 2019 •

edited

SpaceIm commented Jan 30, 2021 •

edited

blackliner commented Feb 17, 2021 •

edited