Support non Python based extension providers #38
Description
This issue's goal is for the community to discuss interest and solutions to support extension providers implemented in languages other than Python.
As a reminder, currently, the toolkit supports three extension providers:
- http: whereby you declare a URL to call and the toolkit does it for you
- process: where you provdie the path to a binary which is executed by the toolkit
- python: where you define a Python function that is imported from a module extension
While Python is considered a good choice for the core and most extensions, we always cared for larger than a single community. @dastergon asked on that subject topic on the community slack and he suggested I should kick the ball with a high-level view of what would need to be done.
Generally speaking, it seems the simplest/easiest integration for calling native code from Python is to export a native library that exports its symbols (much like a C library). When doing that, Python has facilities to call them for you with ctypes.
This is what people seem to generally do:
Alternatives to ctypes are CFFI and cython. The latter is quite interesting because you provide a C-like wrapper on your native extensions and the generated Python code makes it look fairly native. It is popular but requires more work.
There could be two paths:
- The core of the toolkit makes it loud and clear it officially supports ctypes/CFFI and you declare it like this:
{
"type": "probe",
"name": "my-go-blah",
"provider": {
"type": "go",
"lib_name": "my-go-lib.so",
"func": "func_name_in_lib",
"arguments": { ... }
}This is what is done for Python as well but here that would expect simply a native library.
- An extension author wraps entirely the native code inside a Python extension using cython. In that case, the "python" provider is enough and would work as it already does.
I think both are valuable but I wonder what communities would prefer.