CppInterOp based cppyy migration#21261
Open
aaronj0 wants to merge 33 commits intoroot-project:masterfrom
Open
Conversation
aaronj0
requested review from
bellenot,
guitargeek and
vepadulano
as code owners
aaronj0
force-pushed
the
cppyy-interop-migration
branch
from
6ddc3ce to
42586b9
Compare
Test Results 20 files 20 suites 3d 13h 37m 50s ⏱️ For more details on these failures, see this check. Results for commit f9defdb. ♻️ This comment has been updated with latest results. |
aaronj0
force-pushed
the
cppyy-interop-migration
branch
from
54c4fb0 to
45f4502
Compare
aaronj0
force-pushed
the
cppyy-interop-migration
branch
2 times, most recently
from
6d03e1a to
986d7ac
Compare
Contributor
Author
|
Just rebased, windows builds failed with something related to libafterimage |
aaronj0
force-pushed
the
cppyy-interop-migration
branch
from
986d7ac to
c124bcf
Compare
aaronj0
force-pushed
the
cppyy-interop-migration
branch
3 times, most recently
from
9336443 to
66eb416
Compare
Contributor
|
Thanks @aaronj0 for the great progress so far! Just to record where we stand:
|
aaronj0
force-pushed
the
cppyy-interop-migration
branch
3 times, most recently
from
66fbcd0 to
d3cdb55
Compare
aaronj0
force-pushed
the
cppyy-interop-migration
branch
from
70ef9c3 to
3d9b1de
Compare
Contributor
|
Great progress! Similar to #21261 (comment), here the summary:
|
aaronj0
force-pushed
the
cppyy-interop-migration
branch
2 times, most recently
from
3e50982 to
65d8ca1
Compare
Contributor
Author
|
Update: with the last two [tmp] commits: net roottests: |
aaronj0
force-pushed
the
cppyy-interop-migration
branch
4 times, most recently
from
ee3eb5c to
b6e1a3f
Compare
Contributor
Author
|
Update: alma10: 185 (-8) Note: these updates are regression tested, no new failures/early error |
… [upstream] Same as in Pythonize.cxx basic_string name from GetQualifiedCompleteName so the NPOS object is registered for the canonical class name shapes.
ROOT-specific patches on top of the compres fork cppyy/master baseline: __init__.py: - set gInterpereter to gbl.TInterpreter.Instance() - load_library(): use gSystem.Load/FindDynamicLibrary, add Windows winmode=0 for search path - add_library_path(): use gSystem.AddDynamicPath - Drop the CPPYY_API_PATH apipath_extra dispatcher-headers (ROOT installs CPyCppyy API headers) _cpython_cppyy.py: - Wrap `from cppyy_backend import loader` in try/except, fall back to c = None (ROOT does not ship cppyy_backend.so) - Platform: `import libcppyy` vs `import cppyy.libcppyy` (standalone cppyy uses the former; ROOT exposes it as cppyy.libcppyy) - load_reflection_info(): use gSystem.Load ROOT/_facade.py: - Store gInterpreter and gPad in __dict__
Source: compres forks CPyCppyy/master 6dcce64 Brings in fork-ahead developments missing from ROOT's old copy and compatibility with the CppInterOp based backend: - CppInterOp-aligned type system (Cppyy.h) - Type-based CreateConverter / CreateExecutor factory overloads - Instance_FromVoidPtr(scope) overload - PythonGILRAII for exception-safe GIL management - PyError_t RAII rewrite (unique_ptr-based) - GetTemplateArgsTypes for type-based template instantiation - ReduceReturnType/LambdaClass handling - Rvalue forwarding in Dispatcher - GetFailureMsg in Converters - STL string executor returns bound C++ object ROOT-specific CPyCppyyModule.h and CPyCppyyPyModule.cxx are preserved from master.
…tream] Source: ROOT master - Move to using SetGlobalPolicy(ECallFlags, bool) + GlobalPolicyFlags() - Drop the CallContext* argument from UseStrictOwnership() - Guard PyMapping_GetOptionalItemString for Python>3.13 in Dispatcher.cxx
…stream] Source: master
Neither on ROOT-master, nor compres-forks These fields hold C++ type handles, so semantically we should enforce this type
Source: master - Deprecate __mempolicy__ getter/setter with a clear error pointing users to the SetOwnership() pythonization - Drop the per-method memory policy from mp_call (referenced the removed sMemoryPolicy static) - Use GlobalPolicyFlags() for the Clone heuristic - Match the method name prefix before '<' only, so template arguments don't affect Clone detection
…tream] Source: ROOT-master 760b6cc - ToMemory override that lets the array converter copy Python strings (not only buffers) into C++ const char* arrays - ToArrayFromBuffer<> template helper that copies a buffer to an array-converter memory address, lifetime management via SetLifeLine
…NVERTER [upstream] Source: master Adds the 'override' keyword on HasState() in the macro so the derived-class definition matches the base-class virtual declaration.
…eam] Source: ROOT-master - DEFINE_CALL_POLICY_TOGGLE macro generates SetHeuristicMemoryPolicy, SetImplicitSmartPointerConversion, and SetGlobalSignalPolicy used by CallContext::SetGlobalPolicy() - Drop the obsolete kMemoryHeuristics/kMemoryStrict module-level labels (no longer referenced the removed CallContext::kUseStrict) - Add missing guard for Py_INCREF(gThisModule)
… [upstream] Source: ROOT-master - VectorIAdd returns self after the insert. Python += reassigns the lhs to the returned value, so returning the inserted-iterator result silently breaks the idiomatic += pattern on std::vector. Disable by wrapping VectorArray in #if 0 and removes its Utility::AddToClass call (to be fixed at a later stage) - Drop Cppyy::gGlobalScope extern, every caller uses Cppyy::GetGlobalScope()
…d [upstream] This std::span branch went only into the string based overload and not the new type based one that the compres forks use.
… args [upstream] GetQualifiedCompleteName returns the canonical class name with default template args spelled out
…ure mismatches [upstream] In PyFunction_AsCPointer, the CPPOverload and TemplateProxy branches previously returned null when the requested signature does not match the candidates, preventing the generic-Python-callable fall-through (JIT-ed wrapper path), even though CPPOverload and TemplateProxy are valid PyCallable_Check inputs. As a result, std::function/fn ptr parameters could not accept a cppyy free function or template overload whose signature did not match the target.
…[ROOT-patch] Source: ROOT - SignalTryCatch.h: gException resolved to definition from libCore.so - CPPInstance.h: replace CPYCPPYY_IMPORT with explicit extern / __declspec(dllimport) pair. Used by libROOTPythonizations which does not include CommonDefs.h for the definition of CPYCPPYY_IMPORT
…-patch] Source: ROOT-master ROOT runs CPyCppyy and libROOTPythonizations as two shared libs that under the same libcppyy Python extension module. The real PyInit_libcppyy lives in libROOTPythonizations and calls CPyCppyy::Init() to start the extension.
…s [ROOT-patch] Source: ROOT-master - Register a std::span pythonization that overrides begin() / end() with a JIT-compiled __cppyy_internal::ptr_iterator helper. libstdc++ (GCC >= 15) implements std::span::iterator via a private nested tag type, which CallFunc-generated wrappers cannot name without violating access rules — the pointer-based iterator sidesteps that - Add more std::basic_string name variants to the STLWString pythonization
… [ROOT-patch] Source: ROOT - __template_args__ read-only property on TemplateProxy for introspection of a method's template arguments - "ss:__overload__" branch in tpp_overload so callers can select an overload by both signature and template arguments Added for ROOT's Numba-introspection support.
…ype iterators [ROOT-patch] Pythonize.cxx tags begin()-returns as STL iterator types when Cppyy::GetScope succeeds on the return-type string. CppInterOp's GetScope returns the underlying class scopes, leading to raw-pointer iterators (e.g. RVec<T>::iterator = T*) falsely classified as STL iterator types. Restore the earlier behaviour of skipping pointer return values by doing a IsPointerType check on the type-handle. ROOT master's TCling-based GetScope returned null for pointer names and did not hit this false positive.
5cfc2da changed std::string-returning methods to always produce Python str instead of a CPPInstance proxy (gbl.std.string). Update test18_operator_plus_overloads and test34_cstring_template_argument asserts expecting gbl.std.string
…OOT-patch] _generic.pythonize_generic skips the pretty-printer for std::string because ToString returns a quoted ""x"" form, and CPyCppyy already pythonizes std::string with the unquoted shape. With CppInterOp, klass.__cpp_name__ is the canonical fully-qualified template form, so the typedef "std::string" no longer matched the exclude check
aaronj0
force-pushed
the
cppyy-interop-migration
branch
from
0f43a4c to
736d27c
Compare
The `gPad and `gVirtualX` identifiers are injected into ROOT meta via `TGlobalMappedFunction::MakeFunctor()`. However, in the ROOT Python interface we don't want to rely on ROOT meta but use Cling or Clang-Repl directly. Hence, we inject these identifiers manually into the facade. The `TDirectoryPythonAdapter` previously used for `gDirectory` mirrored the live-tracking semantics of the C++ macros: every attribute access re-resolves to the current directory. The same semantics are needed for `gPad` and `gVirtualX`, which on the C++ side are also preprocessor macros that expand to a static accessor call. Therefore, a new `LiveProxy` is introduced for this.
aaronj0
force-pushed
the
cppyy-interop-migration
branch
from
736d27c to
67e4cf4
Compare
…lorNumber ctor param
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Currently in a draft state, based on relatively recent ROOT commit and compiler research forks (Pending another update for the CPyCppyy (compiler-research/CPyCppyy#170, compiler-research/CPyCppyy#167, compiler-research/CPyCppyy#161 cppyy-backend (compiler-research/cppyy-backend#185)