Add support for the Shapely extension

[andreittr]

This change set adds support for the shapely extension in the main Python build.

Depends on Python 3.10 and Numpy:

• Update to upstream Python 3.10.11 #15
• Add support for NumPy #16
• Port upstream NumPy 1.25.0  lib-python-numpy#1

and all their dependencies. Contains all commits from #16.
Commit c26d30d adds shapely support. (Will rebase once numpy is mainlined)

The Shapely microlibrary is a separate PR:

• Port upstream Shapely 2.0.1  lib-python-shapely#1

[StefanJum]

Looks good, but I think we should add the other library dependencies to the Config.uk file (maybe as depends on statements or something), like the intel-intrinsis, libcxx* etc.

[StefanJum]

Why not just select LIBPYTHON_STAPELY if the shapely extension is selected, or do a depends on?

[andreittr]

Same as for numpy, this is the pattern for extensions and shapely isn't special enough to break it.
As for using depends on, in Kconfig a block like

if CONDITION
   config ...
endif

is by definition equivalent to

config ...
depends on CONDITION

so they are functionally identical, but there is an argument to be made for style.

We could change the entire file around to use this format --- it might save a handful of lines --- but it's something that should be done in a python-wide PR.

[StefanJum]

We could change the entire file around to use this format --- it might save a handful of lines --- but it's something that should be done in a python-wide PR.

Agree, let's leave it like it is now.

[andreittr]

Looks good, but I think we should add the other library dependencies to the Config.uk file (maybe as depends on statements or something), like the intel-intrinsis, libcxx* etc.

I strongly disagree. libcxx, intel-intrinsics, etc. are not dependencies of python's bindings to shapely, but of shapely itself (actually, of shapely's dependencies) and should thus logically be expressed in the latter library --- and more generally I'm of the opinion that only direct dependencies should be expressed through Kconfig, with indirect depends and selects being pulled in as required.
That way we have one authoritative source of truth for dependencies, one that we can update in a single, well-defined location when they change.

[StefanJum]

I strongly disagree. libcxx, intel-intrinsics, etc. are not dependencies of python's bindings to shapely, but of shapely itself (actually, of shapely's dependencies) and should thus logically be expressed in the latter library --- and more generally I'm of the opinion that only direct dependencies should be expressed through Kconfig, with indirect depends and selects being pulled in as required. That way we have one authoritative source of truth for dependencies, one that we can update in a single, well-defined location when they change.

True. The final point is that everything should be eventually selected by some Config.uk. For example, is intel-intrinsics stated as a dependency by any library that ends up being selected when using python-shapely (I see that lib-python-shapely and lib-geos do not select it)?

[andreittr]

True. The final point is that everything should be eventually selected by some Config.uk. For example, is intel-intrinsics stated as a dependency by any library that ends up being selected when using python-shapely (I see that lib-python-shapely and lib-geos do not select it)?

intel-intrinsics is selected by numpy, since that lib is the only one that directly requires intrinsics (shapely does not directly use intrinsics in any way, nor does geos -- perfectly reflecting their Config.uk contents).
Same with C++: shapely does not directly use any C++, only numpy and geos do, and they indeed do select libcxx; if either drops C++ as a requirement in the future (numpy existed for many years without C++ so it's pefectly reasonable) it's clear where this change needs to be reflected.

[StefanJum]

intel-intrinsics is selected by numpy, since that lib is the only one that directly requires intrinsics (shapely does not directly use intrinsics in any way, nor does geos -- perfectly reflecting their Config.uk contents). Same with C++: shapely does not directly use any C++, only numpy and geos do, and they indeed do select libcxx; if either drops C++ as a requirement in the future (numpy existed for many years without C++ so it's pefectly reasonable) it's clear where this change needs to be reflected.

Perfect, then it's all good.

[StefanJum]

All good @andreittr, I'll add the tag after all numpy and libgeos dependencies are merged.

[StefanJum]

@andreittr you can rebase this now, #15 is merged 🎉

[andreittr]

@andreittr you can rebase this now, #15 is merged tada

Will rebase it directly onto numpy once it gets merged.

• Add support for NumPy #16

Done

[StefanJum]

Reviewed-by: Stefan Jumarea stefanjumarea02@gmail.com

[mariasfiraiala]

All good with me now, good job!

Reviewed-by: Maria Sfiraiala maria.sfiraiala@gmail.com

[razvand]

Approved-by: Razvan Deaconescu razvand@unikraft.io