Implements JSON-string connection representation generator #35723
Conversation
Taragolis
requested review from
potiuk,
kaxil,
ashb,
bolkedebruin and
XD-DENG
as code owners
Taragolis
force-pushed
the
json-connection-generator
branch
from
6be5197
to
fc96a40
Compare
airflow/models/connection.py
Outdated
| @@ -496,3 +525,10 @@ def from_json(cls, value, conn_id=None) -> Connection: | |||
| except ValueError: | |||
| raise ValueError(f"Expected integer value for `port`, but got {port!r} instead.") | |||
| return Connection(conn_id=conn_id, **kwargs) | |||
|
|
|||
| @property | |||
| def json_repr(self) -> str: | |||
There was a problem hiding this comment.
I'm not sure about this one. Should it be a property or a method
There was a problem hiding this comment.
In first glance yes it is fine, however if some additional settings would required in the future like indent or resolutions for empty hostname and schema which parsed from the uri we could extend it by optional parameters.
In the other hand it is just a helper for create connection for environment variable backend or some cloud based
There was a problem hiding this comment.
Personally I would do something like as_json()
There was a problem hiding this comment.
"There are only two hard things in Computer Science: cache invalidation and naming things"
I do not have personal preference, so let me change it.
There was a problem hiding this comment.
"There are only two hard things in Computer Science: cache invalidation and naming things"
The second is also off-by-one mistakes.
| @@ -498,7 +498,7 @@ def serialize( | |||
| type_=DAT.SIMPLE_TASK_INSTANCE, | |||
| ) | |||
| elif isinstance(var, Connection): | |||
| return cls._encode(var.to_dict(), type_=DAT.CONNECTION) | |||
| return cls._encode(var.to_json_dict(validate=True), type_=DAT.CONNECTION) | |||
There was a problem hiding this comment.
Isn't that going to be backwards incompatible for older connections?
There was a problem hiding this comment.
It should be backward compatible.
If connection serialized by previous version, it should be deserialized by new one. I've also add test for that check.
I'm also not sure that we ever serialize connection in Airflow.
There was a problem hiding this comment.
Previous version serialize conn_id, description and uri new one every field except uri.
So this is basically two existed ways to create connection from constructor.
Co-authored-by: Vincent <97131062+vincbeck@users.noreply.github.com>
* Implements JSON-string connection representation generator * json_repr -> as_json() * Apply suggestions from code review Co-authored-by: Vincent <97131062+vincbeck@users.noreply.github.com> --------- Co-authored-by: Vincent <97131062+vincbeck@users.noreply.github.com>
Airflow 2.3.0 introduce new way for store connection JSON, however there is no easy way to:
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.