Skip to content

dev call 20220407

Stepan Sindelar edited this page Apr 11, 2022 · 14 revisions

Dev Call: 7 April 2022

Present

Antonio, Simon, Matti, Ronan, Florian, Tim, Stepan

Agenda

HPy Release 0.0.4

  • HPy release 0.0.4: no objections; just do it.
  • PyPy also started to integrate HPy 0.0.4 already.
  • GraalPython is almost ready. Next official release will be in July but we provide snapshot builds for experimenting.
  • We will ignore (for now) that HPyGlobals cannot be shared between multiple incompatible (sub-)interpreters
    • Implementations can choose to allow sharing of HPyGlobals between their own subintereperters. For example, in CPython we should make an indirection through interpreter state to support that.
    • Sharing between multiple Python implementations: the contract would be to (atomically) write the address of HPyGlobal as its value. The address is then used by the implementations as a token/identifier for given global. We can decide to go for this later.
  • Reason (for the release): The GraalPython team migrated MPL and Kiwisolver to HPy and wants to make those public.

Autogen: separate API and ABI (context)

  • Autogen API/ABI: we should have a real public header that contains the official API.
  • People agree, that separation basically makes sense.
  • Concerns are that API/ABI might run out of sync.
  • Create and issue and continue discussion there: https://github.com/hpyproject/hpy/issues/311

HPy binary compatibility

  • There is some discussion in the Python community. So maybe it's a good time to think about it and make first steps.
  • Our binary stability is that we have the context and if we don't change the functions and just add at the end, we are fine. But that's probably not future-proof.
  • The HPy context already has a version.
  • Rough idea from Tim/Florian how to do it: * The extension somehow needs to request the required context version. * In multi-phase init, the module can request the context version in the spec. * In single-phase init, we need some _protocol_ to request the HPy context in the right version.
  • Idea by Antonio: The extension could just provide some symbol (much like HPyInit_*) that would specify the context. We can generate the symbol in macro HPy_MODINIT.
  • What if an extension calls function of different extension and passes the context. They could both require a different context version. We will need some _trampoline_ or some converter function to call C functions (with HPyContext) of a different extension.
  • The CPython approach is: if you can load and call a function that raises an exception, then the extension can at least handle the incompatibility nicely (without an immediate crash).
  • Tim and Florian prefer the approach of being able to request a specific context version because that would allow to emulate obsolete/removed ABI functions (maybe at the cost of performance).
  • Simon: how hard would it be for impls support older ABI?

Misc

  • HPyGlobal: we are seeing a lot of load global - call API fun - close handle and these could maybe be done more efficiently by having a dedicated API for it. For example, in our NumPy
  • Simon: talked to two students that will do some HPy benchmarking later this year. We invite them to IRC, mailing list, and dev calls with next month.
  • Cython: Florian didn't address the last review comments yet. The PR is still open but priorities are now on Numpy migration. He will eventually finish that ASAP.

Preliminary Benchmark Results for Kiwi (enaml_bench) and Matplotlib (mpl-bench basic benchmark)

Matplotlib CPython C-API, CPython HPy C-ABI, CPython HPy universal

Matplotlib benchmark suite

Matplotlib CPython C-API, CPython HPy C-ABI, CPython HPy universal, GraalPython HPy Universal on LLVM, GraalPython C-API on LLVM

Matplotlib benchmark with GraalPython

Kiwi CPython HPy C-ABI, CPython C-API, CPython HPy universal

Kiwi benchmark

Kiwi CPython HPy C-ABI, CPython HPy universal, CPython HPy C-API, GraalPython HPy Universal Native, GraalPython HPy Universal on LLVM

Kiwi benchmark with GraalPython
Clone this wiki locally