Minutes_2020_06_09
stuartarchibald edited this page Jun 10, 2020
·
2 revisions
Attendees: Siu, Stuart, Aaron, Dominik, Graham, Guilherme, Hameer, Keith, Todd, Juan, Luk, Pearu, Ritwiga, Hugo, Eric, Ethan, Ehsan, Jim, Mathieu.
-
Open meeting agenda
- Quick intoduction from core devs
- Monthly, Tuesdays, 10:30am central.
- Make sure everyone has read and agrees to CoC
- Quick intoduction from core devs
-
Topics from users:
- Discussion forums
- Currently have gitter, mailinglist, github issues. None of which are great for search/back reference.
- Discourse has been suggested previously.
- Luk: comments on searching and development of thoughts on how to do features. Previously relied on sharing notebooks. Long form discussion is really useful for this. A searchable Q&A is also good, faster for community to edit.
- Ethan: mentions open slack. Disadvantage, cost / limits.
- Hugo: discourse has worked well for other things, previous experience suggests it being good for searching.
- Dominik (over IRC): notes that many places use discourse, it's good software.
- Roadmap
- Current one: out of date as soon as it's published :) https://numba.pydata.org/numba-doc/dev/developer/roadmap.html
- First: Request for any pain points or key things that are lacking.
- Ehsan: Notes there should be a balance between new features and dealing with urgent request etc. Process over where to take Numba in the future, balancing general users (expanding reach, making use frictionless) and advanced users (to permit complex things, e.g. Bodo).
- Luk: Setting general direction is a good idea. Roadmap RE: features. and notably different audiences. Target roadmap to audience? Another group of people are library builders, e.g. consumption in SciPy.
- Ehsan: other kind of advanced user, Bodo, Intel SDC, building compiler system based on Numba. Reusing Numba for basic python features, getting IR etc. But also adding extensions and new compiler passes.
- Todd/Siu/Hameer: SciPy interaction needs new things. Pydata/Sparse is a useful offshoot.
- Jim P: representing a group of physicists, Q: what are the restrictions of the things you can do in Numba? Emphasis should be towards people who have sufficient expertise to write Python code but not necessarily beyond that.
- Ehsan: Out of the groups of people, who should be the audience? Python users, NumPy users etc.
- Jim P: Targetting novice users in the above.
- Siu: People building on the base Numba compiler to fit a very specific domain, e.g. SDC/Bodo pandas/analytics. Jim P: structured arrays. Where should the focus be, making it easier to tailor to these groups?
- Ehsan: Actual problems:
- Compilation time
- Caching
- Error reporting
- Basic Python features, should work well and as expected.
- Jim P: Above is on the lines that serve the physicist group target user.
- Mathieu: Error messages are fine to be huge so long as they are precise.
- Jim P: Error message length, jupyter-notebook with massive errors is psychologically impactful.
- Siu: Stuart fixed the above (somewhat).
- Luk: Location of where the actual error is in a given error structure.
- Hameer: From library dev perspective, long tracebacks are annoying on CI, takes up a huge amount of space/makes logs very long. As a pain point, would like to make it easier to extend Numba by oneself, with view of making it easier to contribute back and also use it elsewhere.
- Eric: Shares the experience with Hameer, lot of effort to do things like unpack struct, handle errors, boxing/unboxing.
- Luk: Suggests it'd be a struggle to do what Eric did to make it work. Two groups in library builders, those who want to exist an existing library "those who can get it to work from numba but not from python" but also "those who can only make it work from both python/interpreter and numba"
- Hugo: want users to be able to use Numba and make their extension library work well with Numba too.
- Siu: we've been trying in the last few years to make this easier. Have been adding and changing APIs on how to do extensions. Sorting out the "adding a new type" problem is a common theme and a good place to grow the Numba community. Numba needs to use the same APIs internally too to make it easier for community contributions.
- Luk: Core developer bottleneck:
- Will put in writing, gain opportunity to comment
- Ethan:
- How do we create a new roadmap!?
- Jim P:
- decide on focus and then yes/no to features.
- Todd: - maximise value creation for some domain: - Support all of python - No! - Domain of scientific computing - voting scheme? - feature request list for preference expression.
- Hameer:
- discourse has voting plugins, could use that.
- Ethan: should we use discourse to do this?
- Siu: can try for that.
- Ethan: another type of user, open source contributors that aren't core devs. Could there be things core devs do to make it easier to add things, unblock other people etc.
- Graham: a lot of it is just a question of digging into the code and discovering.
- Pearu: Feature request/milestone. Core devs know best the state of Numba code base/features etc.
- Siu: lots of opportunities, research directions, back tracking etc. Wanted feedback as it helps guide direction.
- Stuart: wrapped up. Mentioned the large group of users who "just want to speed up code". Code base is massive, beyond the point of any one user knowing it all.
- Siu:
- Discussion will continue on discourse
- Thanks to everyone
- Discussion forums
-
Actions:
- Core devs: Set up discourse for discussions.
- All: try out discourse!
-
If time, the usual:
- triage/PRs
- Defer announcing high risk items until next week.
- #5824 - Overload inliner regression in 0.50.0rc1
-
#5823 - Conditional execution of functions in nopython mode
- Suggests need for having some constant propagation
- #5822 - sort function of typed list is very slow
-
#5820 - Add back support for atomics that always return the non-NaN values
- Suggest adding back as nanmax and nanmin
-
#5816 - Unexpected behavior from
prangewhen accessing large arrays- c.f. Siu's demo which does get a speedup
- #5813 - feature request for np.mgrid
- #5809 - Dangerous default arguments in numba
-
#5805 -
pip install numba==0.48does not makebin/numbaexecutable- Do a new build of wheels with later build number
- #5804 - test_const_record_align failing with CUDA 10.1/10.2/11.0
- #5802 - Slicing a string from a Numpy structured array throws an error
-
#5801 - Type unification problem of integer temp variables in a parfor loop
- Suggest workaround / cast
-
#5799 - Meta-class destructor is not called on exit
- More investigation needed
- #5794 - Numba 0.50.0rc1 Checklist
- #5793 - Call to cuInit results in CUDA_ERROR_NO_DEVICE:
- #5825 - Weird behavior for loop
- #5818 - CUDA atomicrmw error on windows
- #5817 - Cannot compile in nopython mode for a "specialized" function
- #5814 - pop() method of the typed list of typed list crashes on numba 0.49
- #5798 - Initilizing a Numpy structured array inside a jitclass constructor throws an error
- #5797 - some cuda tests failing due to changes in error messages
- #5826 - CUDA: Add function to get SASS for kernels
- #5821 - setup.py: add option to disable tbb at compile time
- #5819 - Add support for CUDA 11 and Ampere / CC 8.0
- #5812 - Fix side issue from #5792, @overload inliner cached IR being mutated.
- #5811 - Eliminate all dangerous default values found by pylint
- #5808 - Fix 5665 Block jit(nopython=True, forceobj=True) and suppress njit(forceobj=True)
- #5796 - Add implementation for np.positive
- #5815 - Pin llvmlite to 0.33
- #5810 - Update CHANGE_LOG for 0.50
- #5807 - Skip failing test on POWER8 due to PPC CTR Loop problem.
- #5806 - Temp/test cuda
- #5803 - CUDA Update expected error messages to fix #5797
- #5800 - Fix LLVM 9 IR for NVVM
- #5795 - Restore refct removal pass and make it strict
-
Requests for 0.51
-
0.51 potential tasks (To be updated)
- TypedSet
- base on TypedDict
- Nested modes
- Move SSA pass up the pipeline
- make sure more passes can work in SSA form
- CUDA Dispatcher / kernel interface:
- Share machinery with CPU target (e.g. for multiple signatures, typeconv, etc.)
- TypedSet
- Opening up the numba meeting