Triggerer not starting

[ecodina]

Under which category would you file this issue?

Airflow Core

Apache Airflow version

3.2.1

What happened and how to reproduce it?

I upgraded from 3.1.8 to 3.2.1 and the Triggerer didn't start properly for my prod instance.

My staging instance worked well (and has worked well for over a week). It uses the same exact Docker image as prod, and the same airflow.cfg (except the value of parallelism).

The triggerer was sending the heartbeat, but the logs only showed:
[info] Starting the triggerer [airflow.jobs.triggerer_job_runner.TriggererJobRunner] loc=triggerer_job_runner.py:201

I changed the log level to debug and this error appeared. All other logs looked as if it was working correctly:

Stack (most recent call last):
  File "/usr/python/lib/python3.12/threading.py", line 1032, in _bootstrap
    self._bootstrap_inner()
    self.run()
  File "/usr/python/lib/python3.12/threading.py", line 1075, in _bootstrap_inner
  File "/usr/python/lib/python3.12/threading.py", line 1012, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/python/lib/python3.12/asyncio/base_events.py", line 645, in run_forever
    self._run_once()
  File "/usr/python/lib/python3.12/asyncio/events.py", line 88, in _run
    handle._run()
  File "/usr/python/lib/python3.12/asyncio/base_events.py", line 1999, in _run_once
    self._context.run(self._callback, *self._args)
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/execution_api/app.py", line 353, in start_lifespan
    await cm.enter_async_context(app.router.lifespan_context(app))
  File "/usr/python/lib/python3.12/contextlib.py", line 659, in enter_async_context
    result = await _enter(cm)
  File "/usr/python/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
  File "/home/airflow/.local/lib/python3.12/site-packages/fastapi/routing.py", line 216, in merged_lifespan
    async with original_context(app) as maybe_original_state:
  File "/home/airflow/.local/lib/python3.12/site-packages/svcs/fastapi.py", line 66, in __call__
    async with self.registry, cm(app, self.registry) as state:
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/execution_api/app.py", line 96, in lifespan
    registry.register_factory(JWTGenerator, _jwt_generator)
  File "/home/airflow/.local/lib/python3.12/site-packages/svcs/_core.py", line 294, in register_factory
  File "/usr/python/lib/python3.12/logging/__init__.py", line 1527, in debug
    log.debug(
    self._log(DEBUG, msg, args, **kwargs)
  File "/usr/python/lib/python3.12/logging/__init__.py", line 1684, in _log
  File "/usr/python/lib/python3.12/logging/__init__.py", line 999, in format
  File "/usr/python/lib/python3.12/logging/__init__.py", line 1160, in emit
    msg = self.format(record)
    self.emit(record)
  File "/usr/python/lib/python3.12/logging/__init__.py", line 1028, in handle
  File "/usr/python/lib/python3.12/logging/__init__.py", line 1762, in callHandlers
    self.callHandlers(record)
  File "/usr/python/lib/python3.12/logging/__init__.py", line 1700, in handle
    self.handle(record)
    hdlr.handle(record)
    return fmt.format(record)

Airflow sent the tasks to the triggerer, but the trigger code wasn't executed.

I had to downgrade to 3.1.8, which left the task instances and dag runs corrupted, so I had to recover the backup.

Environment

Deployed on docker swarm using the official image (with some requirements installed)
We use LocalExecutor

What you think should happen instead?

No response

Operating System

No response

Deployment

Other Docker-based deployment

Apache Airflow Provider(s)

No response

Versions of Apache Airflow Providers

All latest versions existing the 27/april/2026

Official Helm Chart version

Not Applicable

Kubernetes Version

No response

Helm Chart configuration

No response

Docker Image customizations

The Dockerfile installs the following requirements:

apache-airflow-client
apache-airflow-core[gunicorn]>=3.2.0
apache-airflow[async,graphviz,otel,standard]
apache-airflow[postgres,ftp,http,imap,sftp,smtp,ssh,redis,keycloak,git]

It also installs our provider, that contains the deferrable operator code and custom logging:

log_config.py:

LOGGING_CONFIG["handlers"]["task"] = {
    "class": "example.airflow.provider.logging.example_logs.SlurmSharedFileTaskHandler",
    "formatter": "airflow",
    "base_log_folder": BASE_LOG_FOLDER,
    "filters": ["mask_secrets_core"],
}

The SlurmSharedFileTaskHandler inherits from FileTaskHandler and LoggingMixin and only overwrites the _read() method (which I believe it is only used from the api server).

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct