You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
>>> from airflow.utils.json import XComEncoder
>>> XComEncoder().encode(set())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/airflow/airflow/utils/json.py", line 91, in encode
return super().encode(o)
File "/usr/local/lib/python3.7/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/local/lib/python3.7/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/opt/airflow/airflow/utils/json.py", line 82, in default
return serialize(o)
File "/opt/airflow/airflow/serialization/serde.py", line 97, in serialize
if isinstance(o, _primitives):
RecursionError: maximum recursion depth exceeded in __instancecheck__
What you think should happen instead
The encoder should correctly encode a value, or raise TypeError if it is not encodable.
How to reproduce
Simply use XComEncoder to encode anything that’s not encodable by the built-in JSONEncoder (i.e. something that’d go through the custom default hook).
Operating System
Any
Versions of Apache Airflow Providers
Irrelevant
Deployment
Official Apache Airflow Helm Chart
Deployment details
No response
Anything else
The problem here is because serialize does not guarantee to return a JSON-serializable object. So when default receives an set, it returns a set, and the encoder would try to serialize that set again, calling default, and eventually the stack explodes.
The simplest solution would be to add an additional flag to serialize so it can return a JSON-compatible type if requested.
Apache Airflow version
main (development)
What happened
What you think should happen instead
The encoder should correctly encode a value, or raise TypeError if it is not encodable.
How to reproduce
Simply use
XComEncoder
to encode anything that’s not encodable by the built-in JSONEncoder (i.e. something that’d go through the customdefault
hook).Operating System
Any
Versions of Apache Airflow Providers
Irrelevant
Deployment
Official Apache Airflow Helm Chart
Deployment details
No response
Anything else
The problem here is because
serialize
does not guarantee to return a JSON-serializable object. So whendefault
receives an set, it returns a set, and the encoder would try to serialize that set again, calling default, and eventually the stack explodes.The simplest solution would be to add an additional flag to
serialize
so it can return a JSON-compatible type if requested.Are you willing to submit PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: