FIX: Fixed output encoding in WinRMTrigger for WinRMOperator in deferred mode#64154
Open
dabla wants to merge 6 commits intoapache:mainfrom
Open
FIX: Fixed output encoding in WinRMTrigger for WinRMOperator in deferred mode#64154dabla wants to merge 6 commits intoapache:mainfrom
dabla wants to merge 6 commits intoapache:mainfrom
Conversation
eladkal
approved these changes
Mar 24, 2026
eladkal
force-pushed
the
fix/output-encoding-winrm-trigger
branch
from
8884b18 to
f5879fb
Compare
Contributor
|
tests are failing :( |
Contributor
Author
Don't seem to be related to the changes in PR weird |
dabla
requested review from
amoghrajesh,
ashb,
bugraoz93,
gopidesupavan,
jason810496,
jscheffl and
potiuk
as code owners
dabla
force-pushed
the
fix/output-encoding-winrm-trigger
branch
2 times, most recently
from
a2ea719 to
8259da3
Compare
dabla
force-pushed
the
fix/output-encoding-winrm-trigger
branch
from
8259da3 to
021700a
Compare
bugraoz93
approved these changes
Mar 25, 2026
Contributor
bugraoz93
left a comment
bugraoz93
left a comment
There was a problem hiding this comment.
Look good to me! I have a nit related to how we ignore the 0 collected tests. Not blocking as it shouldn't cause any issues immediately
jscheffl
reviewed
Mar 25, 2026
| output_bytes = output.encode("ascii") | ||
| else: | ||
| output_bytes = output | ||
| except UnicodeEncodeError as e: |
Contributor
There was a problem hiding this comment.
We had similar problems in Docker and K8s provider, can you take a look here? https://github.com/apache/airflow/pull/62632/changes
I suspect it is the same root cause, you get chars that are not JSON serializable in logs? In both Docker and K8s the fix was to replace incorrect surrogates, can happen if reads are in bytes and chunks are only "half" unicode chars (e.g. 2-byte char and you read only 1 byte at the end of a line.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WinRM deferrable mode breaks with non-UTF8 output encodings
8fc49f4
providers/microsoft/winrm/src/airflow/providers/microsoft/winrm/triggers/winrm.py | providers/microsoft/winrm/src/airflow/providers/microsoft/winrm/operators/winrm.py
The new deferrable WinRM trigger base64-encodes stdout/stderr and immediately decodes those bytes using the user-supplied output_encoding. Base64 data is ASCII; decoding it with encodings like UTF-16 produces non-ASCII characters. When the operator resumes, _decode() calls base64.standard_b64decode on those strings, which raises because the string is not ASCII. This means deferrable mode will crash for non-ASCII output encodings (e.g., UTF-16LE output from Windows), a regression introduced by this commit.
Was generative AI tooling used to co-author this PR?
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.