Support LLVM 12

[KOLANICH]

No description provided.

[esc]

From what I can tell, LLVM 12 has not been released yet. Or did I miss something? Latest available release is 10.0.1?

[KOLANICH]

What do you mean under "released"? There are prebuilt packages for debian-based distros, and they work. I use clang 12 every day and it works fine. pocl linked against llvm 12 works too. So do other llvm-based tools I have ported to llvm 12.

Given that my previous PRs into this repo bringing compatibility to LLVM 10 were just ignored for years, until they got unneeded, effectively my work on creating them has been wasted, I am not going to send here any PRs anymore.

[stuartarchibald]

@abebeos see #639

[almson]

Could you guys start working on supporting LLVM and Python versions before they are released? I'm on a rolling distro and llvmlite/numba make me want to tear my hair out.

P.S. Can you release a no-op numba which does not rely on llvmlite and can be used to satisfy other packages' dependencies?

[esc]

@almson thank you for asking about this on the llvmlite issue tracker, your input is appreciated.

• Regarding Python versions: We are looking toward getting ahead of Python pre-releases this year. Python's development activity has shifted toward breaking backward compatibility on bytecodes more frequently than they did historically, so we are attempting to adjust our own testing system to let us get ahead of that.
• Regarding LLVM versions: Supporting LLVM's faster is always of interest, but LLVM's tendency to break backward compatibility and/or break some platforms we need to support (like PPC or ARM) in non-obvious ways makes us very cautious about upgrading LLVM too quickly.
• Regarding the no-op package: I don't understand what the purpose of a Numba "no-op" package would be. I don't yet see what such a package would look like and what benefits it would bring. Thus, I would appreciate if you could elaborate on this, thank you.

[KOLANICH]

Supporting LLVM's faster is always of interest

If it was the case, my PRs wouldn't have been ignored for years.

[KOLANICH]

However, like with nearly any open-source product which drives commercial products:
=> Priorities come usually from the customers of the commercial products

It may make sense to set up a PR template explicitly making it clear that only PRs implementing the features for which it was paid (unfortunately GH has no feature to disable PRs) are welcome.

[almson]

@esc What I mean is a python library or numba compilation option which would not use system libraries nor look at python bytecode, which would still allow numba-based libraries to function on any Python platform but at interpreted speed. The end result is a separate package which would not have dependencies. (Sorry I'm not a direct numba user may be misunderstanding how numba works.)

[esc]

Dear @abebeos and @KOLANICH, thank you for your suggestions regarding the modification of response time communication on issues in an attempt to manage expectations. I am open to discuss this further as part of the conversation in #686 which @abebeos has kindly initiated.

[esc]

Dear @almson, thank you for your suggestion regarding a no-op package for Numba. The idea may have a multitude of ramifications and I believe some in-depth technical discussion will be necessary to establish the details of the package such as content, name and versioning scheme in addition to specifics regarding the maintenance and release process. Therefore I would propose to continue this discussion on the Numba discourse on the llvmlite category at: https://numba.discourse.group/c/llvmlite/12 -- and I encourage you to initiate a conversation there. Thank you!

[seibert]

Hey, I (wearing my Anaconda hat) just want to comment on some incorrect assumptions that are being made about Anaconda's relationship with Numba/llvmlite in this thread.

Anaconda employs Numba developers because we think it is a useful package for the Python community, the same as we fund or have funded developers for other OSS projects (Dask, Bokeh, in the past Pandas, Jupyter, etc). It is important for us to contribute to open source as we (like many other businesses) depend on it.

Numba is not a direct part of Anaconda's commercial products (beyond being a part of the Anaconda Distribution, along with thousands of other Python packages) and we don't manage development around some internal product roadmap being driven by Anaconda customers. Anaconda makes money in a way that is orthogonal to Numba development. Numba does receive or has received funding for some feature development from various entities (both commercial and non-profit), but we try to always select projects that align with what we think is important for the broader Numba user base.

The complaint in this thread is about our reluctance to quickly update LLVM. This is motivated by a long history of dealing with obscure bugs, testing headaches, and other problems related to LLVM updates. We are cautious not because we are worried about Anaconda's commercial users, we are cautious because we are worried about all of our users, most of whom will see minimal benefit if we move from LLVM version N to LLVM version N+1. LLVM is a large, complex project moving very quickly, and so every update carries risk with it, even with all the testing we already do.

You are free to disagree with that engineering decision, of course, but I just want to be very clear about our motivations.

[advancedwebdeveloper]

It is interesting that some tools, known to be used with LLVM (like Ninja) , were never tested to work on llvmlite.

…

On Tue, Jan 12, 2021 at 4:43 AM Stan Seibert ***@***.***> wrote: Hey, I (wearing my Anaconda hat) just want to comment on some incorrect assumptions that are being made about Anaconda's relationship with Numba/llvmlite in this thread. Anaconda employs Numba developers because we think it is a useful package for the Python community, the same as we fund or have funded developers for other OSS projects (Dask, Bokeh, in the past Pandas, Jupyter, etc). It is important for us to contribute to open source as we (like many other businesses) depend on it. Numba is not a direct part of Anaconda's commercial products (beyond being a part of the Anaconda Distribution, along with thousands of other Python packages) and we don't manage development around some internal product roadmap being driven by Anaconda customers. Anaconda makes money in a way that is orthogonal to Numba development. Numba does receive or has received funding for some feature development from various entities (both commercial and non-profit), but we try to always select projects that align with what we think is important for the broader Numba user base. The complaint in this thread is about our reluctance to quickly update LLVM. This is motivated by a long history of dealing with obscure bugs, testing headaches, and other problems related to LLVM updates. We are cautious not because we are worried about Anaconda's commercial users, we are cautious because we are worried about *all* of our users, most of whom will see minimal benefit if we move from LLVM version N to LLVM version N+1. LLVM is a large, complex project moving very quickly, and so every update carries risk with it, even with all the testing we already do. You are free to disagree with that engineering decision, of course, but I just want to be very clear about our motivations. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#611 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABJHLKRE7YADG3UHU3IFAFLSZOZL5ANCNFSM4PVZVPVQ> .

[esc]

Since multiple threads of discussion have been conflated into this issue, I will close it now and open a fresh issue to cleanly track LLVM 12.* support.

[esc]

The new issue is here: #688