Minutes_2023_02_14

Numba Meeting: 2023-02-14

Attendees: Andre Masella, Guilherme, Jim Pivarski, stuart, Siu Kwan Lam, Graham Markall, Todd A. Anderson, Luk, brandon willard, Kaustubh Chaudhari FPOC (last week): Andre FPOC (incoming): Siu

NOTE: All communication is subject to the Numba Code of Conduct.

Please refer to this calendar for the next meeting date.

0. Discussion

• llvmlite linux-aarch64 packaging issues:
  • option 1: llvmlite split llvm14 and llvm11 for 0.40.0
    • details:
      • one version of llvmlite
      • linux-aarch64: llvmlite use llvm11
      • otherwise: llvmlite use llvm14
    • problems:
      • but should we still ship llvm14 on linux-aarch64 for non jit user? if so, how to version?
  • option 2: numba use llvmlite 0.40.0 except on linux-aarch64 which use 0.39.0
    • details:
      • numba uses two versions of llvmlite versions
      • linux-aarch64: numba 0.57 depends on llvmlite 0.39.0 (llvm11)
      • other platforms: numba 0.57 depends on llvmlite 0.40.0 (llvm14)
    • problems:
      • confusion for Numba package maintainer (conda-forge, linux distro...)
      • setup.py depedency is depends on the executing platform
      • no py3.11 support?
  • option 3: others??? Wait for OrcJIT?
  • Conclusion: option 1 is the least horrible
• (Jim) In the Numba IR redesign, can a piece be separated out into its own package (with no dependence on LLVM) that can produce ASTs from Python bytecode? That would enable general macros. An ecosystem could develop on that, if it's possible.
  • Yes, this will take the RVSDG frontend.
  • Jim is willing to work on the AST producer from the above frontend.
  • There are a lot of dead projects that aim to provide macros in Python:
    • https://peps.python.org/pep-0638/ (2 years)
    • https://github.com/lihaoyi/macropy (5 years)
    • https://github.com/alexmojaki/sorcery (2 years)
    • and uncompyle6 might want to use this as well (or anyway, I could replace anything I'd want to use uncompyle6 for with something that turned Python bytecode into ASTs)
• (Todd) - When will Numba IR redesign happen and how will that work be structured? Also, as part of this reimplementation, will the charter expand beyond NumPy to more of the data structures in typical scientific Python?
  • structure:
    • the work will be open with champions of each components.
    • all components are in prototype phase.
    • RVSDG and PIXIE are most concrete ideas currently
  • More than just numpy:
    • Yes, we don't want to make another fast numpy DSL. We want to enable other data-structures esp. in a way that doesn't require Numba maintainer to develop and maintain every data structures. We want to enable the broader community to work on adding new data structures.
• (Guilherme) Would the Numba redesign stop work in other areas? i.e. existing PRs that are waiting for review that touches core parts
  • Work on Numba will continue but avoid starting new large features.
  • There will be trade-off because of effort on researching and developing new tech and components.
• (Luk) will the cache PR be affected by PIXIE work?
  • Stu: It will take 2-3 years for PIXIE to reach the horizon to replace the cache machinery in Numba
• (Luk) AOT
  • Stu: New AOT focus is about letting user control when to compile.

New "Ready for Review" PRs

• numba#5544 - Add support for np.union1d - Graham plans to look over again.

1. New Issues

• numba#8747 - np.roll not working in nopython model (CUDA)
• numba#8749 - Unwanted numba rounding vs native sum
• numba#8753 - Memory overwriting other memory issue?

Closed Issues

• numba#8745 - Failure in test_dispatcher due change in representation of types

2. New PRs

• numba#8744 - Update CtypesLinker::add_cu error message to include fp16 usage
• numba#8748 - Suppress known test failures for py3.11
• numba#8750 - [Experimental] Made closures with yield pass though MakeFunctionToJITFunction compiler pass
• numba#8751 - Recycle test runners more aggressively
• numba#8752 - Flake8 fixes for py311 branch
• numba#8754 - [TESTING ONLY] test pr8752 + pr8748
• llvmlite#905 - Add YouCompleteMe configuration file and ignore vim swap files

Closed PRs

• merged - numba#8746 - Fix failing test_dispatcher test case

3. Next Release: Version 0.57.0/0.40.0, RC Feb 2023

• pending conda packages for dependencies