Fix Airflow serialization for namedtuple

[Joffreybvn]

This PR fixes: #36839
In some cases, the namedtuple was not recognized as tuple and failed to serialize.

Side note:
When we return a tuple from a task, the tuple reach the XComEncoder and get serialized as a dict - thanks to this line. However, when we return a list of tuples, the tuples get serialized as lists. The above condition is false, and we fallback to the default json encoding behavior. Thus, the way namedtuple / tuple are handled is inconsistent.

This causes some queries of the Databricks Hook and Operator to fail right now, depending on how the Hook / Operator are parameterized.

This PR fix the namedtuple for such queries, but it doesn't fix the inconsistency: sometimes namedtuple will be encoded as dict, sometimes as list. IMO a definitive fix involve another strategy than qualname(). Because namedtuples have dynamic qualnames, wich will never be "builtins.tuple".

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[bolkedebruin]

I'm picky in this area due for speed and tightness. This gets called a lot. Make sure to check for attributes as much as you can (hasattr > isinstance)

[bolkedebruin]

ping @Joffreybvn :-)

[Joffreybvn]

Sorry, busy week. I'm on it !

[bolkedebruin]

Please keep the changes to a minimum. The only two lines that are required are

if _is_namedtuple(o):
  qn = "builtins.tuple"

[Joffreybvn]

We do need to specify the classname, which should be "builtins.tuple":

The serialize method (in airflow/serialization/serializers/builtin.py) does qualname() on the namedtuple, which returns "airflow.providers.databricks.hooks.Row" (the namedtuple dynamically created in the hook). If this is used as classname, it will fail to deserialize: there won't be any deserializer for it.

If we hardcode it to "builtins.tuple", it gets deserialized correctly.

[potiuk]

Hey @joffreyvbn @bolkedebruin - any chance to get it soon ?

[Joffreybvn]

CI-CD is failing because the generated classname is the name of the namedtuple, instead of "builtins.tuple".
Do we want to serialize namedtuples like tuples ? If yes, we need to specify the classname @bolkedebruin.

Otherwise, how do you want namedtuples to be serialized?

[bolkedebruin]

We do need to specify the classname, which should be "builtins.tuple":

The serialize method (in airflow/serialization/serializers/builtin.py) does qualname() on the namedtuple, which returns "airflow.providers.databricks.hooks.Row" (the namedtuple dynamically created in the hook). If this is used as classname, it will fail to deserialize: there won't be any deserializer for it.

If we hardcode it to "builtins.tuple", it gets deserialized correctly.

Ah sorry @Joffreybvn , I missed that. If you'd like you can revert to the previous / earlier version. That looks and reads better. Add a comment that you are overriding the class name and it makes things clear

[Joffreybvn]

Thanks @bolkedebruin ! I reverted the commit, and added a comment to clarify it.