-
Notifications
You must be signed in to change notification settings - Fork 90
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MINIFICPP-1961 Python scripting compatibility with multiple python minor versions #1504
Conversation
I've ran the new python compatibility tests on my fork. |
I've followed @szaszm advice and readded ExecuteScript in a3b6e6b, so this PR doesn't break compatibility with older configs, and follows NiFi's way of doing ExecuteScript more closely. This of course intruduces a bit of additional complexity (and maybe restrictions) vs the separate ExecuteLuaScript/ExecutePythonScript approach. Let me know what you think. |
47c3922
to
a3b6e6b
Compare
extensions/python/pythonprocessors/examples/CountingProcessor.py
Outdated
Show resolved
Hide resolved
da748bf
to
13a0c35
Compare
I've also reran the compatibility tests after these changes. |
e0ab06d
to
1d0fa07
Compare
1d0fa07
to
71a3385
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't got to the end of this, but only minor quibbles so far.
extensions/python/pythonprocessors/examples/GaussianDistributionWithNumpy.py
Outdated
Show resolved
Hide resolved
Simplify TestExecuteScriptProcessorWithPythonScript with SingleProcessorTestController Added test for when session.get() returns nullptr (and fixed related bug)
Co-authored-by: Ferenc Gerlits <fgerlits@users.noreply.github.com>
…fCount before overwriting previous pointer
e0fd671
to
9efeaee
Compare
To support binaries targeting multiple libpython minor versions. Closes apache#1504 Signed-off-by: Marton Szasz <szaszm@apache.org> Co-authored-by: Martin Zink <martinzink@apache.org>
To support binaries targeting multiple libpython minor versions. Closes apache#1504 Signed-off-by: Marton Szasz <szaszm@apache.org> Co-authored-by: Martin Zink <martinzink@apache.org>
To support binaries targeting multiple libpython minor versions. Closes apache#1504 Signed-off-by: Marton Szasz <szaszm@apache.org> Co-authored-by: Martin Zink <martinzink@apache.org>
To support binaries targeting multiple libpython minor versions. Closes apache#1504 Signed-off-by: Marton Szasz <szaszm@apache.org> Co-authored-by: Martin Zink <martinzink@apache.org>
To support binaries targeting multiple libpython minor versions. Closes apache#1504 Signed-off-by: Marton Szasz <szaszm@apache.org> Co-authored-by: Martin Zink <martinzink@apache.org>
The bulk of this python rewrite was done by @dam4rus. I've took over because he got other responsibilities that made it unfeasable to finish the PR.
This change replaces the previous Python extension implementation (based on PyBind11) with a pure Python C API implementation (with the limited stable API) https://docs.python.org/3/c-api/stable.html
I've included a new ci workflow that uses the centos tar.gz and verifies this new extension works on a large variety of platforms with large variety of python versions, but I don't think we want to run these on every commit. (Maybe on every release?)
I've also separated the current scripting extensions into lua/python extensions so it will be possible to build them together, and still use them without needing the other one's dependencies.
Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:
Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?
Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
Has your PR been rebased against the latest commit within the target branch (typically main)?
Is your initial contribution a single, squashed commit?
For code changes:
For documentation related changes:
Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible.