New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
dbld
/python
: use python3 for every platform & remove xenial & add type support
#4057
Merged
MrAnno
merged 27 commits into
syslog-ng:master
from
alltilla:dbld-python3-all-platforms
Oct 13, 2022
Merged
dbld
/python
: use python3 for every platform & remove xenial & add type support
#4057
MrAnno
merged 27 commits into
syslog-ng:master
from
alltilla:dbld-python3-all-platforms
Oct 13, 2022
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
MrAnno
reviewed
Jun 24, 2022
alltilla
force-pushed
the
dbld-python3-all-platforms
branch
from
July 15, 2022 14:58
bf92363
to
cad2770
Compare
alltilla
changed the title
Jul 15, 2022
dbld
: use python3 for every platform & remove xenialdbld
/python
: use python3 for every platform & remove xenial & add type support
alltilla
force-pushed
the
dbld-python3-all-platforms
branch
from
July 15, 2022 15:03
cad2770
to
a974f07
Compare
alltilla
force-pushed
the
dbld-python3-all-platforms
branch
from
July 15, 2022 15:08
a974f07
to
94a057f
Compare
alltilla
force-pushed
the
dbld-python3-all-platforms
branch
from
July 15, 2022 15:16
94a057f
to
abf82e1
Compare
alltilla
force-pushed
the
dbld-python3-all-platforms
branch
from
July 15, 2022 15:17
abf82e1
to
a5cda12
Compare
Build FAILURE |
alltilla
force-pushed
the
dbld-python3-all-platforms
branch
from
July 18, 2022 09:00
a5cda12
to
3769fd6
Compare
alltilla
force-pushed
the
dbld-python3-all-platforms
branch
from
July 18, 2022 09:08
3769fd6
to
4ef8372
Compare
alltilla
force-pushed
the
dbld-python3-all-platforms
branch
from
July 18, 2022 09:08
4ef8372
to
37774a5
Compare
Build FAILURE |
alltilla
force-pushed
the
dbld-python3-all-platforms
branch
from
July 18, 2022 09:33
37774a5
to
a94e138
Compare
@kira-syslogng test this please test branch=ose-with-python-3; |
alltilla
force-pushed
the
dbld-python3-all-platforms
branch
from
July 18, 2022 09:40
a94e138
to
33815c9
Compare
Signed-off-by: Attila Szakacs <attila.szakacs@oneidentity.com>
Signed-off-by: Attila Szakacs <attila.szakacs@oneidentity.com>
alltilla
added a commit
to alltilla/syslog-ng
that referenced
this pull request
Oct 13, 2022
Signed-off-by: Attila Szakacs <attila.szakacs@oneidentity.com>
alltilla
force-pushed
the
dbld-python3-all-platforms
branch
from
October 13, 2022 18:34
0b993f6
to
44e6ee6
Compare
Signed-off-by: Attila Szakacs <attila.szakacs@oneidentity.com>
alltilla
added a commit
to alltilla/syslog-ng
that referenced
this pull request
Oct 13, 2022
Signed-off-by: Attila Szakacs <attila.szakacs@oneidentity.com>
alltilla
force-pushed
the
dbld-python3-all-platforms
branch
from
October 13, 2022 18:52
44e6ee6
to
e769faf
Compare
alltilla
added a commit
to alltilla/syslog-ng
that referenced
this pull request
Oct 13, 2022
Signed-off-by: Attila Szakacs <attila.szakacs@oneidentity.com>
alltilla
force-pushed
the
dbld-python3-all-platforms
branch
from
October 13, 2022 18:58
e769faf
to
1da09e0
Compare
python-types provides the same functionality. Signed-off-by: Attila Szakacs <attila.szakacs@oneidentity.com>
python-types offers the same functionality. Signed-off-by: Attila Szakacs <attila.szakacs@oneidentity.com>
…ossible `!py_bytes_or_string_to_string()` checks it for us. Signed-off-by: Attila Szakacs <szakacs.attila96@gmail.com>
Signed-off-by: Attila Szakacs <attila.szakacs@oneidentity.com>
`_dict_clone_value()` limits us, to only be able to get python values as strings. Also, it extracts the python value to C, which leaves us room for error (which we happen to have, because we lose the knowledge about whether it is shown as bytes or str in python) The only 2 users of `_dict_clone_value()` uses it to check a python variable's value, so it is easier to add an assert function for that, which can handle any kinds of types (as you build up a "script", you can inject any types of variables). Signed-off-by: Attila Szakacs <attila.szakacs@oneidentity.com>
Signed-off-by: Attila Szakacs <attila.szakacs@oneidentity.com>
Signed-off-by: Attila Szakacs <attila.szakacs@oneidentity.com>
In the previous 2 commits we have introduced 2 breaking changes: 1. Previously if someone tried to set an NV value with not a string/bytes type in a python code, we threw an error. With the previous commit we changed the behavior and we store it with the correct type. 2. Previously the python NV accessor function always returned string/bytes type. With the previous commit we return the same type as it is stored in the `LogMessage`. The 1. point is not a big deal for backward compatibility reasons, because we can be more forgiving, as long as we do not fail with previously working code. The 2. point is a bigger deal, because older codes expect strings/bytes, but we can return other types, based on how it is stored in the `LogMessage`. To fix these, I have added a compatibility flag to the `PyLogMessage` class. If the flag is set, we throw an error when setting a non string/bytes value in python code, and we return string/bytes typed value when accessing NVs in python code, no matter how it is stored in the `LogMessage`. Unfortunately, setting this flag can only be done where we create the `PyLogMessage` instance in C code, as we only have a reference to the `GlobalConfig` there. We cannot do that in the python code, because the constructor there executes a callback (`py_log_message_new_empty()`), which cannot have any user_data provided. This means we can only set it in parser and destination, and not in source. Not setting it in the source is fine, because there, we have always created an empty `LogMessage`, so there is no risk of accessing NVs with non bytes/string type. Signed-off-by: Attila Szakacs <szakacs.attila96@gmail.com>
Signed-off-by: Attila Szakacs <szakacs.attila96@gmail.com>
It is implemented in a parametrized fashion, which might seems off, but more input sets will be added in the following commits. Signed-off-by: Attila Szakacs <szakacs.attila96@gmail.com>
We want to add the config version 3.x compat to its state in prepare time, so we can create the PyLogMessage accordingly. This commit paves the way for that. Signed-off-by: Attila Szakacs <szakacs.attila96@gmail.com>
Signed-off-by: Attila Szakacs <szakacs.attila96@gmail.com>
Signed-off-by: Attila Szakacs <szakacs.attila96@gmail.com>
IMHO we should not "fix" this problem with the blacklist, but the proper fix would need a lot more work, so for now, it is better to use the blacklist workaround correctly, until we have a proper fix. Signed-off-by: Attila Szakacs <attila.szakacs@oneidentity.com>
Signed-off-by: Attila Szakacs <attila.szakacs@oneidentity.com>
Build FAILURE |
alltilla
force-pushed
the
dbld-python3-all-platforms
branch
from
October 13, 2022 19:23
1da09e0
to
3903daa
Compare
Build FAILURE |
MrAnno
approved these changes
Oct 13, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For the reviewers
Kira fails, because it needs 3 things to work:
python3
instead ofpython2
.python-parser
test config a bit, to work with the more up-to-datehy
module.dbld-kira
image build is needed, to have the latesthy
package. It will only be generated when this PR is merged.Kira tests with dbld-kira image generated from this PR, and with both of the 2 internal MRs:
Why remove xenial
It does not work with python3.
Its standard support ended in April 2021. Also the python3 version on it is python3.5, which's support ended in September 2020.
We have kept xenial until now, because we could use it with python2, but now, as we started to use python3 everywhere, and it does not work here, it is time to remove xenial from the supported platforms.
GitHub Actions devshell run
https://github.com/alltilla/syslog-ng/actions/runs/2746728090
Local package tests
Steps:
test-apt-packages.yml
.Signed-off-by: Attila Szakacs attila.szakacs@oneidentity.com