Rotate fernet key optimisation

[bjankie1]

---

Current implementation of Fernet key rotation implicitly executes all() method on the processed tables leading to loading all rows to memory.
It's been observed that some users store additional data in variable table which is leading to memory issues during the operation.
This change introduces batch processing of fernet key rotation to avoid it. To be consistent across the tables (variable, connection, trigger) the batching operation was added for all of them.

[potiuk]

NIT: Can we extract it to a constant value ?

[potiuk]

Hey @bjankie1 -> we need to revert that one. Our canary builds detected that yield_per is sqlalchemy 2-only feature and we still support sqlalchemy 1 because a number of our users might rely on it via other dependencies.

That's why I revert it for now in #40769

I think we still need to do a conditional work on those queries. We already have in airflow/sqlalchemy_utils:

def is_sqlalchemy_v1() -> bool:
    return version.parse(metadata.version("sqlalchemy")).major == 1

(so it should be redone with it).

[bjankie1]

Thanks. I'll redo it with the condition suggested.