clojure: cross-platform unique library dependency #13971
Comments
Thanks for reporting the issue @leontalbot 😸 @nswamy @lanking520 @andrewfayres @frankfliu I'm wondering if it would be possible to have another "single" artifact for scala that was packaged for both Linux and OSX (and eventually windows). It would help cross platform portability and adoption and enable others to build on top of the mxnet and include it as a dependency for all systems. One problem is, I don't know how the gpu/cpu would work though.... Here is a possible maven plugin https://github.com/maven-nar/nar-maven-plugin. Thoughts? |
@gigasquid There are three issues inside:
|
Thank you for taking a look @gigasquid @lanking520 ! Was wondering, globally, would a single artefact take much more disk space then platform specific ones? Is that an issue? |
Theoretically, a single global artefact would in fact take a lot more disk space as well, which might or might not be an issue given the standards of modern day computing. But, @leontalbot just @lanking520 mentioned, the issue is not about the space for the artefact, rather the platform specific dependencies that we need to bundle into the jars. There are technical limitations to doing so. In Scala, when we are not sure whether our end users will be using Ubuntu CPU/GPU or MacOS, we add these profiles in pom.xml and let Maven pick it up for us. eg snippet taken from here : https://github.com/apache/incubator-mxnet/blob/master/scala-package/mxnet-demo/java-demo/pom.xml
Hope that answers your question :) @leontalbot Please feel free to re-open if closed in error. |
Thanks @piyushghai and @lanking520 - I think we could do something similar with Leiningen https://stackoverflow.com/questions/4688336/what-is-an-elegant-way-to-set-up-a-leiningen-project-that-requires-different-dep |
This seems a good approach to me, as we'll be abstracting away the complexity of choosing which jar/artifact from the user. |
I created a JIRA ticket to track this request in our backlog : |
@piyushghai Thanks for filing this as an item in backlog. Close this issue for now. @leontalbot Please feel free to reopen it if you believe your problems is unsolved. |
Really nice, thanks everyone! |
I am building a clojure library. I'd like to use MxNet in it.
I found these different dependencies I could use:
[org.apache.mxnet.contrib.clojure/clojure-mxnet-osx-cpu "1.3.1"]
[org.apache.mxnet.contrib.clojure/clojure-mxnet-linux-cpu "1.3.1"]
[org.apache.mxnet.contrib.clojure/clojure-mxnet-linux-gpu "1.3.1"]
Say I want to build a Clojure library that needs to use Clojure MXNet
I cannot tell if someone who will be using my library will be on Windows, Mac or Linux.
How should I handle this?
I'd like to avoid packaging the native files up on one jar myself and having it choose the right one
https://github.com/swannodette/native-deps
This idea was added here: https://cwiki.apache.org/confluence/display/MXNET/Clojure+Package+Contribution+Needs
Would it be a good idea to have a unique cross-platform jar as dependency?
The text was updated successfully, but these errors were encountered: