Electric fails to connect to the database if another instance of Electric is already running

[alco]

The error looks like this in the log output:

2024-01-18T11:12:44.580068439Z ▓ ┌────────────────────┐
2024-01-18T11:12:44.580106777Z ▓ │  CONNECTION ERROR  │
2024-01-18T11:12:44.580113567Z ▓ ┕━━━━━━━━━━━━━━━━━━━━┙
2024-01-18T11:12:44.580118544Z ▓ 
2024-01-18T11:12:44.580125387Z ▓ Failed to establish replication connection to Postgres:
2024-01-18T11:12:44.580133704Z ▓   replication slot "electric_replication_out_defaultdb" is active for PID 38381
2024-01-18T11:12:44.580140172Z ▓ 
2024-01-18T11:12:44.580144738Z ▓ Another instance of Electric appears to be connected to this database.
2024-01-18T11:12:44.580318925Z 11:12:44.579 pid=<0.2795.0> origin=postgres_1 [error] PostgresConnectorSup failed to start child :postgres_producer with reason: {:bad_return_value, {:error, {:error, :error, "55006", :object_in_use, "replication slot \"electric_replication_out_defaultdb\" is active for PID 38381", [file: "slot.c", line: "516", routine: "ReplicationSlotAcquire", severity: "ERROR"]}}}.
2024-01-18T11:12:44.580413865Z 11:12:44.579 pid=<0.2795.0> origin=postgres_1 [info] schedule retry: 2000

This can happen on some hosting platforms where a configuration change in an already deployed instance of Electric will cause a new instance to be started to use the update config. The new instance will not be able to establish a replication connection to the database until the old instance shuts down.

The reason there can only be one Electric instance connected to a database at any given time is that Electric uses a replication connection to listen for changes originating in Postgres. Supporting multiple instances running simultaneously requires careful design and development to ensure that consistency is preserved and that the load is split evenly between all instances.

Until we address the underlying limitation, we suggest using one of these workarounds:

• stop the running instance of Electric before starting a new one
• use a TCP health check (as opposed to an HTTP health check for /api/status) when deploying Electric to a hosting platform

[linear]

VAX-1685 Electric fails to connect to the database if another instance of Electric is already running