1wjob can't be loaded

[coding-dragon520]

Apache Airflow version

2.10.5

If "Other Airflow 2 version" selected, which one?

2.10.4

What happened?

We can run a single DAG to produce 3K, but for business development, a single DAG needs to support 1W jobs, and there is currently no way for DAG to insert serialized_dag

What you think should happen instead?

We can run a single DAG to produce 3K, but for business development, a single DAG needs to support 1W jobs, and there is currently no way for DAG to insert serialized_dag

How to reproduce

We can run a single DAG to produce 3K, but for business development, a single DAG needs to support 1W jobs, and there is currently no way for DAG to insert serialized_dag

Operating System

k8s + mysql8 + airflow

Versions of Apache Airflow Providers

We can run a single DAG to produce 3K, but for business development, a single DAG needs to support 1W jobs, and there is currently no way for DAG to insert serialized_dag

Deployment

Official Apache Airflow Helm Chart

Deployment details

We can run a single DAG to produce 3K, but for business development, a single DAG needs to support 1W jobs, and there is currently no way for DAG to insert serialized_dag

Anything else?

We can run a single DAG to produce 3K, but for business development, a single DAG needs to support 1W jobs, and there is currently no way for DAG to insert serialized_dag

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[coding-dragon520]

[2025-04-27T08:06:46.314+0000] {processor.py:925} INFO - DAG(s) '20250422163140_T000005' retrieved from /opt/airflow/dags/20250422163140_T000005.py
[2025-04-27T08:07:07.020+0000] {logging_mixin.py:190} INFO - [2025-04-27T08:07:06.118+0000] {override.py:1914} ERROR - Creation of Permission View Error:
(MySQLdb.OperationalError) (2006, 'Server has gone away')
[SQL: INSERT INTO serialized_dag (dag_id, fileloc, fileloc_hash, data, data_compressed, last_updated, dag_hash, processor_subdir) VALUES (%s, %s, %s, %s,
%s, %s, %s, %s)]
[parameters: ('20250422163140_T000005', '/opt/airflow/dags/20250422163140_T000005.py', 178526774
89718545, '{"__version": 1, "dag": {"edge_info": {}, "fileloc": "/opt/airflow/dags/20250422163140_T000005.py", "timezone": "A
sia/Sh ... (108561095 characters truncated) ... equests': {'cpu': '1000m', 'memory': '8000Mi', 'nvidia.com/gpu': None}}}"}, "__type": "operator
"}], "dag_dependencies": [], "params": []}}', None, datetime.datetime(2025, 4, 27, 8, 6, 46, 327318), '42320c17324c79621f353eff88039c9c', '/opt/airflow/da
gs')]

[coding-dragon520]

What configuration do I need to increase so that a single DAG can support 1W jobs?