Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide C++ functions to expose objects to Python #798

Closed
Tracked by #813
feltech opened this issue Jan 9, 2023 · 0 comments · Fixed by #929
Closed
Tracked by #813

Provide C++ functions to expose objects to Python #798

feltech opened this issue Jan 9, 2023 · 0 comments · Fixed by #929
Assignees
Labels

Comments

@feltech
Copy link
Member

feltech commented Jan 9, 2023

What

As a hybrid C++/Python host application author
I want functions to expose C++ Manager, Context, LoggerInterface ... objects to Python that were created in C++ (and vice versa?)
Because assets need to be queried and published in both languages sharing the same underlying OpenAssetIO state.

Why

If a host application uses a compatible version of pybind11, then exposing a (bound) OpenAssetIO object to Python is fairly trivial (pybind11::cast(...) et al.). However, we cannot rely on a host application using pybind11 (or a compatible version of it). In OpenAssetIO we deliberately hide the dependency on pybind11 as an internal tool, so it is not accessible directly through the OpenAssetIO library.

So we should expose functions whose only external dependency is CPython (PyObject* et al) and that will expose a C++-created Manager object (for example) to the host application's Python environment.

Acceptance Criteria

  • exposeToPython-like functions for all required cross-language objects (to be designed).
  • Documentation examples updated.
  • Integration tests added (OpenAssetIO-Test-CMake or otherwise).
  • If py::cast fails, raise the string held in PyErr (no custom exception needed)
  • The refcount strategy for the method is conceptually aligned with the CPython API usage.
  • GIL is not acquired by the method (unless required by error handling etc).
  • GIL and ref counting behaviour is documented as part of the method docs.
  • Test lifetimes (including python derived class unowned in Python called from C++)
  • Mis-matched casts raise appropriate non-pybind exception
  • Converters do not assume ownership of the castee

Notes

Workflow sketches required to figure out the API surface area, including which objects must be explicitly exposed by the host application.

@feltech feltech added enhancement New feature or request Python story labels Jan 9, 2023
@foundrytom foundrytom added the good first issue Good for newcomers label Mar 20, 2023
@elliotcmorris elliotcmorris self-assigned this May 4, 2023
elliotcmorris added a commit to elliotcmorris/OpenAssetIO that referenced this issue May 11, 2023
[OpenAssetIO#798] There is a need as a hybrid C++/Python host application author
to convert types between an embedded python interpreter and a c++
context, as some hosts will want to do asset resolution in both
languages, sharing the same configuration.

This adds two methods to python bridge, `castToPyObject`
and `castFromPyObject` that facilitates this.

Prior, the way to do this was to do a pybind cast, which
is what we do here behind the scenes, but by providing
methods to raw CPython types, we eliminate the need
to link against pybind for the consumer.

Signed-off-by: Elliot Morris <elliot.morris@foundry.com>
elliotcmorris added a commit to elliotcmorris/OpenAssetIO that referenced this issue May 11, 2023
[OpenAssetIO#798] There is a need as a hybrid C++/Python host application author
to convert types between an embedded python interpreter and a c++
context, as some hosts will want to do asset resolution in both
languages, sharing the same configuration.

This adds two methods to python bridge, `castToPyObject`
and `castFromPyObject` that facilitates this.

Prior, the way to do this was to do a pybind cast, which
is what we do here behind the scenes, but by providing
methods to raw CPython types, we eliminate the need
to link against pybind for the consumer.

Signed-off-by: Elliot Morris <elliot.morris@foundry.com>
elliotcmorris added a commit to elliotcmorris/OpenAssetIO that referenced this issue May 11, 2023
[OpenAssetIO#798] There is a need as a hybrid C++/Python host application author
to convert types between an embedded python interpreter and a c++
context, as some hosts will want to do asset resolution in both
languages, sharing the same configuration.

This adds two methods to python bridge, `castToPyObject`
and `castFromPyObject` that facilitates this.

Prior, the way to do this was to do a pybind cast, which
is what we do here behind the scenes, but by providing
methods to raw CPython types, we eliminate the need
to link against pybind for the consumer.

Signed-off-by: Elliot Morris <elliot.morris@foundry.com>
elliotcmorris added a commit to elliotcmorris/OpenAssetIO that referenced this issue May 12, 2023
[OpenAssetIO#798] There is a need as a hybrid C++/Python host application author
to convert types between an embedded python interpreter and a c++
context, as some hosts will want to do asset resolution in both
languages, sharing the same configuration.

This adds two methods to python bridge, `castToPyObject`
and `castFromPyObject` that facilitates this.

Prior, the way to do this was to do a pybind cast, which
is what we do here behind the scenes, but by providing
methods to raw CPython types, we eliminate the need
to link against pybind for the consumer.

Signed-off-by: Elliot Morris <elliot.morris@foundry.com>
elliotcmorris added a commit to elliotcmorris/OpenAssetIO that referenced this issue May 16, 2023
[OpenAssetIO#798] There is a need as a hybrid C++/Python host application author
to convert types between an embedded python interpreter and a c++
context, as some hosts will want to do asset resolution in both
languages, sharing the same configuration.

This adds two methods to python bridge, `castToPyObject`
and `castFromPyObject` that facilitates this.

Prior, the way to do this was to do a pybind cast, which
is what we do here behind the scenes, but by providing
methods to raw CPython types, we eliminate the need
to link against pybind for the consumer.

Signed-off-by: Elliot Morris <elliot.morris@foundry.com>
elliotcmorris added a commit to elliotcmorris/OpenAssetIO that referenced this issue May 16, 2023
[OpenAssetIO#798] There is a need as a hybrid C++/Python host application author
to convert types between an embedded python interpreter and a c++
context, as some hosts will want to do asset resolution in both
languages, sharing the same configuration.

This adds two methods to python bridge, `castToPyObject`
and `castFromPyObject` that facilitates this.

Prior, the way to do this was to do a pybind cast, which
is what we do here behind the scenes, but by providing
methods to raw CPython types, we eliminate the need
to link against pybind for the consumer.

Signed-off-by: Elliot Morris <elliot.morris@foundry.com>
elliotcmorris added a commit to elliotcmorris/OpenAssetIO that referenced this issue May 16, 2023
[OpenAssetIO#798] There is a need as a hybrid C++/Python host application author
to convert types between an embedded python interpreter and a c++
context, as some hosts will want to do asset resolution in both
languages, sharing the same configuration.

This adds two methods to python bridge, `castToPyObject`
and `castFromPyObject` that facilitates this.

Prior, the way to do this was to do a pybind cast, which
is what we do here behind the scenes, but by providing
methods to raw CPython types, we eliminate the need
to link against pybind for the consumer.

Also adds an additional CTest project,
`openassetio.internal.python-bridge-no-openassetio-module-test` that
runs tests tagged with `[no_openassetio_module]`, on order to guarantee
test runs without the openassetio having already been loaded/linked.

Signed-off-by: Elliot Morris <elliot.morris@foundry.com>
feltech pushed a commit to feltech/OpenAssetIO that referenced this issue May 17, 2023
[OpenAssetIO#798] There is a need as a hybrid C++/Python host application author
to convert types between an embedded python interpreter and a c++
context, as some hosts will want to do asset resolution in both
languages, sharing the same configuration.

This adds two methods to python bridge, `castToPyObject`
and `castFromPyObject` that facilitates this.

Prior, the way to do this was to do a pybind cast, which
is what we do here behind the scenes, but by providing
methods to raw CPython types, we eliminate the need
to link against pybind for the consumer.

Also adds an additional CTest project,
`openassetio.internal.python-bridge-no-openassetio-module-test` that
runs tests tagged with `[no_openassetio_module]`, on order to guarantee
test runs without the openassetio having already been loaded/linked.

Signed-off-by: Elliot Morris <elliot.morris@foundry.com>
elliotcmorris added a commit to feltech/OpenAssetIO that referenced this issue May 19, 2023
[OpenAssetIO#798] There is a need as a hybrid C++/Python host application author
to convert types between an embedded python interpreter and a c++
context, as some hosts will want to do asset resolution in both
languages, sharing the same configuration.

This adds two methods to python bridge, `castToPyObject`
and `castFromPyObject` that facilitates this.

Prior, the way to do this was to do a pybind cast, which
is what we do here behind the scenes, but by providing
methods to raw CPython types, we eliminate the need
to link against pybind for the consumer.

Also adds an additional CTest project,
`openassetio.internal.python-bridge-no-openassetio-module-test` that
runs tests tagged with `[no_openassetio_module]`, on order to guarantee
test runs without the openassetio having already been loaded/linked.

Signed-off-by: Elliot Morris <elliot.morris@foundry.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Done
3 participants