Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature setup config #6

Closed

Conversation

init-random
Copy link
Contributor

Hello -- This is a simple config setup for the ability to configure the -L/-l blas link_options for your individual environment, e.g. in setup.cfg I needed
[link_options]
link_dir = /opt/openblas/lib
link_library = openblas
The current setup may not be optimal as more platforms are supported, bit it may be a starting point.

@henningpeters
Copy link
Contributor

Thanks for the PR. I am not yet convinced a setup.cfg solution is better than hardcoding the values in setup.py. What I am saying is that the problem is not solved, but just has moved to a different place. Hardly anybody installs nowadays manually via setup.py/distutils, cannot easily tweak setup.cfg and hence has to live with our presets. What would make a great contribution is making the install process truly seamless.

Because we find the idea of vendorizing openblas/atlas quite awful we started looking at good blas alternatives. Any ideas/contributions in that area are greatly appreciated.

@init-random
Copy link
Contributor Author

Hello Henning -- Thanks for taking a look. The install did not work in my environment, so I thought I would
generalize; setup.cfg seems to be a standard place to put header configs. Point taken, however. 
Why is there a hard dependency on blas? Could this be offloaded to scipy? It seems like gensim does thishttp://rare-technologies.com/word2vec-in-python-part-two-optimizing/ (see BLAS section)However, it does not look like it is required.
https://github.com/piskvorky/gensim/blob/5cd434bcc321be0a93bd52714ac93c93470c8bbe/README.rst
If users want to link their scipy to blas, then that would be up to them to have optimized versions.
Install in python does not have much consensus. Did you have initial thought on the direction you would
like to go? If so, I would be happy to take a look.
Thanks,Keith 

On Sunday, February 28, 2016 9:15 AM, Henning Peters <notifications@github.com> wrote:

Thanks for the PR. I am not yet convinced a setup.cfg solution is better than hardcoding the values in setup.py. What I am saying is that the problem is not solved, but just has moved to a different place. Hardly anybody installs nowadays manually via setup.py/distutils, cannot easily tweak setup.cfg and hence has to live with our presets. What would make a great contribution is making the install process truly seamless.Because we find the idea of vendorizing openblas/atlas quite awful we started looking at good blas alternatives. Any ideas/contributions in that area are greatly appreciated.—
Reply to this email directly or view it on GitHub.

@henningpeters
Copy link
Contributor

We are looking for a small and lean vector library that is optimized for performance and easy to distribute/depend on. I think the pattern in which a library falls back on slow paths if some dependency is not present at install-time is bad for a number of reasons, hence we really try to avoid such dependencies. Best candidate so far is eigen which comes as a header-only library and should be easy to distribute along sense2vec. You could help us by integrating it into the current codebase. Would you be up to this? I could provide some pointers where to start...

@honnibal
Copy link
Member

To elaborate on this a little bit:

We're concerned that getting a good, fast linked BLAS is currently quite an arduous task at this time. It's also implicitly a system-level dependency. This is a problem that people very often have to face when working with scientific Python. The Continuum stack makes this much easier, too. But we'd like to support a broader audience, and really we just need one or two functions.

So, it seems to us that it'll be best to vendorize a fast C/C++ math library, so that it can be statically linked, and there will be no system dependencies.

Wrapping all of Eigen will actually be fairly difficult. I've been looking into this. But I think we can get the subset we need without too much work.

@init-random
Copy link
Contributor Author

OK, great. Thanks for the detail. I am a little tied up this week, but I would certainly be happy to look into this
if you do not mind my pinging you until i get up and running. Any thoughts on minieigen?

On Monday, February 29, 2016 11:01 AM, Matthew Honnibal <notifications@github.com> wrote:

To elaborate on this a little bit:We're concerned that getting a good, fast linked BLAS is currently quite an arduous task at this time. It's also implicitly a system-level dependency. This is a problem that people very often have to face when working with scientific Python. The Continuum stack makes this much easier, too. But we'd like to support a broader audience, and really we just need one or two functions.So, it seems to us that it'll be best to vendorize a fast C/C++ math library, so that it can be statically linked, and there will be no system dependencies.Wrapping all of Eigen will actually be fairly difficult. I've been looking into this. But I think we can get the subset we need without too much work.—
Reply to this email directly or view it on GitHub.

@honnibal
Copy link
Member

License is incompatible :(. We're under MIT license, and a lot of our users care that the library stay unencumbered.

@init-random
Copy link
Contributor Author

Got it, good to know.

On Monday, February 29, 2016 11:36 AM, Matthew Honnibal <notifications@github.com> wrote:

License is incompatible :(. We're under MIT license, and a lot of our users care that the library stay unencumbered.—
Reply to this email directly or view it on GitHub.

@init-random
Copy link
Contributor Author

I just wanted to let you know that I am going to start looking into the eigen
integration now. I saw the new USE_BLAS. I think it makes sense to have
the fallback (other) default to eigen if USE_BLAS is not defined, so it wouldreplace the #else in the shim file.
Let me know if you have any particular requirements.

Thanks,Keith

On Monday, February 29, 2016 11:42 AM, Keith Chewning <chewning_keith@yahoo.com> wrote:

Got it, good to know.

On Monday, February 29, 2016 11:36 AM, Matthew Honnibal <notifications@github.com> wrote:

License is incompatible :(. We're under MIT license, and a lot of our users care that the library stay unencumbered.—
Reply to this email directly or view it on GitHub.

@henningpeters
Copy link
Contributor

Cool, looking forward to your contribution... maybe open a new issue in case you want to discuss anything related to eigen.

In fact if we have eigen we would like it to replace both, the BLAS option and the #else in the shim file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants