dev call 20220407
Antonio, Simon, Matti, Ronan, Florian, Tim, Stepan
- HPy release 0.0.4
- Autogen: separate API and ABI (context)
- HPy binary compatibility
-
- Assuming that our ABI will always be backwards compatible is a bit naïve. We should start thinking about how to deal with that.
- Inspired by SDL2 paper: https://www.reddit.com/r/linux_gaming/comments/1upn39/sdl2_adds_dynamic_api_magic_to_allow_updating_it/ (sorry, couldn't find the original paper)
- We should think about a policy for supported HPy ABI versions
- Debug mode: The possibility to decide per module makes implementation complicated. Maybe a global decision would be enough (i.e. all contexts are debug contexts or none).
- HPyType_FromSpec: disallow setting any tp_flags not supported by HPy https://github.com/hpyproject/hpy/issues/305
- Remove support for Python 3.6 https://github.com/hpyproject/hpy/issues/307
- NumPy Progress and Plans
-
- NpyIter
- UFunc
- Wiki with some migration notes: https://github.com/hpyproject/hpy/wiki/Numpy-HPy-migration-notes:-blockers-and-concerns
- Open HPy PRs
-
- Capsule PR: https://github.com/hpyproject/hpy/pull/308
- exact type check: https://github.com/hpyproject/hpy/pull/309
- detect HPy_Close/return of context constants in debug: https://github.com/hpyproject/hpy/pull/303
- HPy release 0.0.4: no objections; just do it.
- PyPy also started to integrate HPy 0.0.4 already.
- GraalPython is almost ready. Next official release will be in July but we provide snapshot builds for experimenting.
- We will ignore (for now) that HPyGlobals cannot be shared between multiple incompatible (sub-)interpreters
-
- Implementations can choose to allow sharing of HPyGlobals between their own subintereperters. For example, in CPython we should make an indirection through interpreter state to support that.
- Sharing between multiple Python implementations: the contract would be to (atomically) write the address of HPyGlobal as its value. The address is then used by the implementations as a token/identifier for given global. We can decide to go for this later.
- Reason (for the release): The GraalPython team migrated MPL and Kiwisolver to HPy and wants to make those public.
Autogen: separate API and ABI (context) -----------------------------* Autogen API/ABI: we should have a real public header that contains the official API. * People agree, that separation basically makes sense. * Concerns are that API/ABI might run out of sync. * Create and issue and continue discussion there: https://github.com/hpyproject/hpy/issues/311
- There is some discussion in the Python community. So maybe it's a good time to think about it and make first steps.
- Our binary stability is that we have the context and if we don't change the functions and just add at the end, we are fine. But that's probably not future-proof.
- The HPy context already has a version.
- Rough idea from Tim/Florian how to do it:
- The extension somehow needs to request the required context version.
- In multi-phase init, the module can request the context version in the spec.
- In single-phase init, we need some _protocol to request the HPy context in the right version.
- Idea by Antonio: The extension could just provide some symbol (much like HPyInit_*) that would specify the context. We can generate the symbol in macro HPy_MODINIT.
- What if an extension calls function of different extension and passes the context. They could both require a different context version. We will need some _trampoline or some converter function to call C functions (with HPyContext) of a different extension.
- The CPython approach is: if you can load and call a function that raises an exception, then the extension can at least handle the incompatibility nicely (without an immediate crash).
- Tim and Florian prefer the approach of being able to request a specific context version because that would allow to emulate obsolete/removed ABI functions (maybe at the cost of performance).
- Simon: how hard would it be for impls support older ABI?
- HPyGlobal: we are seeing a lot of load global - call API fun - close handle and these could maybe be done more efficiently by having a dedicated API for it. For example, in our NumPy
- Simon: talked to two students that will do some HPy benchmarking later this year. We invite them to IRC, mailing list, and dev calls with next month.
- Cython: Florian didn't address the last review comments yet. The PR is still open but priorities are now on Numpy migration. He will eventually finish that ASAP.
Preliminary Benchmark Results for Kiwi (enaml_bench) and Matplotlib (mpl-bench basic benchmark) -------------------------
Matplotlib CPython C-API, CPython HPy C-ABI, CPython HPy universal
Matplotlib CPython C-API, CPython HPy C-ABI, CPython HPy universal, GraalPython HPy Universal on LLVM, GraalPython C-API on LLVM
Kiwi CPython HPy C-ABI, CPython C-API, CPython HPy universal
Kiwi CPython HPy C-ABI, CPython HPy universal, CPython HPy C-API, GraalPython HPy Universal Native, GraalPython HPy Universal on LLVM
- 4 April 2024
- 7 March 2024
- 1 February 2024
- 11 January 2024
- 7 December 2023
- 9 November 2023
- 5 October 2023
- 14 September 2023
- 3 August 2023
- 6 July 2023
- 1 June 2023
- 4 May 2023
- 13 April 2023
- 2 March 2023
- 2 February 2023
- 12 January 2023
- 1 December 2022
- 3 November 2022
- 6 October 2022
- 8 September 2022
- 4 August 2022
- 7 July 2022
- 2 June 2022
- 5 May 2022
- 7 April 2022
- 3 March 2022
- 3 February 2022
- 13 January 2022
- 2 December 2021
- 4 November 2021
- 7 October 2021
- 2 September 2021
- 12 August 2021
- 8 July 2021
- 6 May 2021
- 4 March 2021
- 7 January 2021
- 3 December 2020
- 5 November 2020