dev call 20211202

Dev Call: 2 December 2021

Present

Tim, Florian, Stepan, Antonio, Ronan, Matti

Numpy progress

• Ronan managed to port some of the members of ndarary to HPy
• There were problems with stack-allocated dummy objects. Ronan fixed that upstream (?).
• Still one of the main problems: getting an HPyContext to call HPy functions from legacy C code. This is in particular required for Numpy's C API (which is aligend with the Python C API). We would now need to overlay every function with an HPy version (there are about 150 of them).
• Antonio suggested: if we introduce an API to get an HPyContext (out of thin air), it should only be possible in legacy mode.
• Ronan changed his mind on the discussed callback approach (discussed in the previous HPy dev call); he thinks its not feasible for Numpy.
• Matti suggested to go the way via PyBind11 (probably less effort).
• Issue for further discussion regarding API for getting the context: https://github.com/hpyproject/hpy/issues/268

Plans for HPy 0.0.4 -----------------

• Since PyPy is currently low on dev resources, we aim for a release in March 2022.
• We will include everything on master.

General Discussions

• We were discussing the problem of passing generic data across the boundary in a binary compatible. This should be a replacement for fromat string + varargs. One idea is to have sevaral primitve arrays (one for int64, double, and pointer etc).
• Ronan thinks that HPyField is not very nice to use. While in CPython, one usually just uses PyObject * (and can therefore also directly use it for anything else), HPyField always requires a separate load operation. Maybe we can reduce the pain by providing macros that generate accessors.
• We were also discussing if HPyLoad and HPyStore may fail. The conclusions was that it can but only with fatal errors. So, no need for error checking.

Other recent HPy developments

• Stepan: PR #253; extracts code examples to snippets that are actually tested in the CI and referenced from the docs.
• Stepan: PR #256, #259, #264: Debug mode improvements.
• Stepan: PR #256; Added CI job that runs main tests with a Python debug build.
• Ronan: PR #257; Don't clobber legacy tp_dealloc unnecessarily.
• Florian: PR #261; Fix: do not trave uninitialized HPyField.
• Florian: PR #262; Refactor HPyModuleDef; also migrate piconumpy-hpy
• Florian: PR #265; Re-enabled cppcheck in the CI but this is using the version from Ubuntu which is currently 1.90.
• Florian: PR #251; making progress on the call API.
• Florian: PR #8 (ujson-hpy); Migrate to changed HPyModuleDef, fix tests and add GH Actions for tests.
• Florian: Ongoing work on Cython/HPy; publishing PR in the next days.

What people are working on next

• Florian is continuing work on Cython/HPy.
• Florian will create a micro benchmark for HPyField.
• Someone of the Graal team will probably help to make progress with Numpy.
• Stepan will finish PRs in progress/review.