Minutes_2018_12_18

Attendees: Siu, Stuart, Ehsan, Stan, Todd

0. Feature Discussion

• Numba 0.42.0 release status
  • llvmlite RC tagged & build
  • numba RC tagged
  • tag numba for final release on 28th
• 2 minute thing... topics for GH page to make Numba discoverable by topic. Numpy has "numpy" and "python"
  • python
  • numpy
  • llvm
  • compiler
  • cuda
  • rocm
  • parallel

Numba Roadmap update

Requested by several groups. Here's a draft version that will go into developer docs:

• Short term: 2019H1

  • Container improvements:
    • Numba dictionary support
    • Refactor lists to follow new container best practices
  • Deprecate Python 2.7 support
  • Improve caching:
    • Full support for functions compiled with ParallelAccelerator
    • Safe caching of generated functions (eval of strings)
    • Expire cache when any function in call chain (even in other files) changes
    • Process for distributing pre-populated cache
  • Continue to improve usability and debugging:
    • Trap more unsupported features earlier in pipeline (especially things that parfors can’t handle)
    • Error messages
    • Diagnostic tools for debugging and understanding performance
    • Better on-boarding for new users and contributors (new homepage, revise docs, more examples)
  • Begin refactoring existing features that cause common bug reports:
    • Enhance description of interfaces provided by Numba functions to give more type information
    • Convert older Numba function implementations to use public extension mechanisms
    • More unit testing and modularization of ParallelAccelerator passes

• Medium Term: 2019H2

  • Unify dispatch of regular functions, ufuncs, and gufuncs
  • Declare Numba 1.0 with stable interfaces
  • Continue to improve usability and debugging:
    • See above
  • Continue refactoring Numba internals to solve common bug reports:
    • See above
  • JIT class review and improvement
  • Improve compilation speed
  • Improve memory management of Numba-allocated memory
  • Better support for writing code transformation passes
  • Make caching and parallel execution features opt-out instead of opt-in
    • add heuristic to determine if parfor passes will be beneficial

• Long Term: 2020 and beyond

  • Unify GPU backends (share more code and interfaces)
  • Improve ahead of time compilation (for low powered devices)
  • Improve cross language connections (C++, JVM?, Julia?, R?)
    • Call Numba from other languages,
    • Call from Numba into other languages
  • Better support for "hybrid" CPU/GPU/TPU/etc programming
  • Partial / deferred compilation of functions
  • Foster integration of Numba into core PyData packages:
    • scipy/scikit-learn/scikit-image/pandas
  • More support for efforts to put Numba into other applications (databases, etc) for compiling user-defined functions
  • More support for usage of Numba as a “compiler toolkit” to create custom compilers (like HPAT, automatic differentiation of functions, etc)
  • Investigate AST-based Numba frontend in addition to existing bytecode-based frontend

1. New issues

• #3613 - Compiled function being forgotten in 0.41
• #3610 - runtests.py issue: test does not work
• #3602 - unsupported keyword arguments when calling Function(<ufunc 'add'>)
• #3601 - internal error
• #3600 - Numba 0.42RC checklist
• #3599 - Njitted permutations
• #3595 - llvm.dbg.declare deprecation
• #3593 - Remove code duplication in cuda real hw vs simulator
• #3591 - Lowering Error, cannot cast none to bool
• #3590 - isinstance(None, numba.types.NoneType) is False
• #3588 - Force compilation caching
• #3586 - empty_inferred typing error
• #3585 - is None construct in nopython mode

Already Closed

• #3603 - Segmentation Fault while trying to do in place np.sin
• #3589 - Internal error with array closure

2. Open PRs

• #3612 - wip: exact_match_flag
• #3607 - Some simplication to lowering
• #3604 - [WIP] Support for np.interp
• #3598 - Implement np.asarray()
• #3592 - WIP: Add dead branch prune before type inference.

Already Merged

• #3611 - gdb test guard needs to ack ptrace permissions
• #3609 - Mention ARMv7 support in installation instructions
• #3608 - Update changelog for 0.42.0
• #3606 - Improve the accuracy of error message wording for undefined type.
• #3605 - Add guard for writing to read only arrays from ufunc calls
• #3597 - Warn users if threading backend init unsafe.
• #3596 - Fix Array Analysis for Global Namedtuples
• #3594 - Changing @stencil docs to correctly reflect func_or_mode param
• #3587 - Fix llvm.memset usage changes in llvm7

Old

• #3572 - Support for np.ptp

  • in-progress

• #3563 - Support for np.roll

  • Waiting on author
  • basically fine

• #3561 - Fixing the boolean copy from data

  • In progress

• #3556 - [WIP] Support for np.trim_zeros

  • in progress

• #3519 - WIP: fix-3457 support of numpy repeat.

• #3468 - Add support for np.clip and ndarray.clip.

  • depends on fix on @overload_method for kwargs

• 3450 - [WIP] generated_jit for CUDA kernels

• 3134 - [WIP] Cfunc x86 abi

  • Needs re-review

• 3046 - Pairwise sum implementation.

• #2999 - Support LowLevelCallable

• #2942 - Fix linkage nature (declspec(dllexport)) of some test functions

• #2894: [WIP] Implement jitclass default constructor arguments.

• #2817: [WIP] Emit LLVM optimization remarks

===========================

4. Next Release: Version 0.42, RC=ASAP, Final=Dec 27, 2018

• LLVM 7
• GDB support
• Other string features/fixes
• Caching bugs