Minutes_2018_09_18

Numba Meeting: 2018-09-18

Attendees: Stan, Siu, Ehsan, Stuart, Todd

0. Feature Discussion

• Start 0.40.0 release process?

  • Checklist: https://github.com/numba/numba/issues/3317

• Brainstorming for 0.41 dev cycle

  • Dates:
    • RC: November 19? Final: November 26?
  • PR cleanup (+1)
  • Parallel diagnostics PR improvements (+1)
  • Type refactor (+1)
    • Use cases to improve:
      • Const arguments
      • Typing of containers (list, set, dict in the future)
      • Pyobject in nopython mode
      • Layer a "view" on a basic type
  • Big gufunc/ufunc dispatch refactor (using xnd)
  • Dictionaries?
    • type inference for key/value types
  • Strings? (+1)
    • ascii?
    • All strings treated as const
    • Need to be able easily create strings as a result of operations
    • format()
  • Fix known caching limitations
    • Make dispatcher carry call graph?
    • Review how Julia does it
  • Lowering error rewrite (if extra time)
  • First pass of replacing low level lowering with usage of extension API
  • Start test suite refactoring
    • Are there redundant tests with significant overlaps in coverage?
  • Closure rewriting mutates IR and breaks everything in confusing ways
    • Make IR immutable!
  • @inlined decorator
    • Force inlining
  • Distributed compilation
    • Worker processes jitting objcode
    • Predict type signatures and compile ahead of time
  • Start blogging about Numba features
  • How do we do Cython-style highlight of code with poor performance?
    • Probably need to be heuristics based since there isn't a single metric that makes sense
    • Note things that are not C/F order arrays
  • two stage compilation?
    • First call produces -O0 fast compile
    • recompile for -O3 in the background and switch in at future time (easier: on next call, harder: in the middle of a call?)

• Azure Pipeline Status

  • Enabled with following configurations:
    • Linux 64-bit
    • Linux 32-bit
    • Windows 64-bit
    • Mac 64-bit
  • Seems to be working well. About 2x faster on test suite than AppVeyor + Travis CI.
  • Will run along side other CI systems for a while to ensure results make sense and match the others.

1. New issues

• 3319 - Improve documentation to show how to call cffi functions taking struct arguments
  • Not possible currently
  • Responded in issue
• 3317 - 0.40.0 Final Release Checklist
  • In progress now
• 3315 - Document spawn workaround when CUDA is used with multiprocessing
  • See #3322 for fix
• 3314 - suspected parfors array analysis failure under reshape
  • Can we handle if reshape is constant?
  • Raise error?
  • Can the "unknownness" be flagged in analysis
• 3312 - 1.16.2.4. Does Numba automatically parallelize code? gufuncs with the target="parallel" example
  • Doc suggestion
  • Maybe need a good gufunc example
• 3308 - interest in arm64 (armv8, aarch64, 64-bit arm) support
  • Need to have more packages and ARMv8 intaller testing
• 3306 - Add doc for NUMBA_DEBUGINFO in Environment variables page
  • Easy doc addition
• 3305 - jitclass method not inlined bug
  • Two methods in jitclass calling each other are not inlined
• 3304 - can't create pycc AOT compiler on Windows
  • Explains mystery error that we see in tests
  • Clearly no one is using AOT on windows
  • Solution is included in issue

2. Open PRs

New

• 3322 - Add FAQ on CUDA + fork issue. Fixes #3315.
• 3263 - Add note about memory leaks with exceptions to docs. Fixes #3263
• 3320 - [WIP] Support for np.partition
• 3316 - Update 0.40 changelog with additional PRs
• 3309 - Workaround race condition with apt
• 3307 - Adding NUMBA_ENABLE_PROFILING envvar, enabling jit event

Old

• 3228 - Reduce redundant module linking
  • Stuart will review
• 3162 Support constant dtype string in nopython mode in functions like numpy.empty.
  • Need to resolve #3195
• 3160 First attempt at parallel diagnostics
  • Stuart will implement Todd's suggestion
• 3134 [WIP] Cfunc x86 abi
  • Needs re-review
• 3124 Fix 3119, raise for 0d arrays in reductions
  • Stuart needs to implement feedback
• 3093 [WIP] Singledispatch overload support for cuda array interface.
  • How do deal with polymorphic (CPU or GPU) types?
• 3046 Pairwise sum implementation.
• #2999 Support LowLevelCallable
• #2983 [WIP] invert mapping b/w binop operators and the operator module
• #2950 Fix dispatcher to only consider contiguous-ness.
• #2942 Fix linkage nature (declspec(dllexport)) of some test functions
• #2894: [WIP] Implement jitclass default constructor arguments.
• #2817: [WIP] Emit LLVM optimization remarks

===========================

4. Next Release: Version 0.41, RC=Nov 19, Final=Nov 26, 2018

• TBD
• Usual collection of bug fixes