Skip to content

fix: handle cloud-sql-connector unhandled exceptions on close#10555

Merged
fredzqm merged 6 commits into
mainfrom
fix/cloud-sql-connector-unhandled-exception
May 26, 2026
Merged

fix: handle cloud-sql-connector unhandled exceptions on close#10555
fredzqm merged 6 commits into
mainfrom
fix/cloud-sql-connector-unhandled-exception

Conversation

@fredzqm
Copy link
Copy Markdown
Contributor

@fredzqm fredzqm commented May 26, 2026

Description

This commit resolves an issue where deploying a Data Connect service backed by Cloud SQL Postgres fails with an unexpected unhandled exception (CloudSQLConnectorError: The connector was closed).

In @google-cloud/cloud-sql-connector version >= 1.7.0, the connector.close() method destroys underlying sockets by asynchronously emitting an error event. Previously, firebase-tools closed the pg.Pool first, which caused the database client to detach all error listeners from its sockets. Consequently, the later connector.close() call caused Node.js to crash due to an unhandled exception on those sockets.

To prevent this:

  1. Re-ordered cleanup sequences so that connector.close() is executed prior to pool.end().
  2. Attached an explicit 'error' handler to pg.Pool to ensure any connection teardown errors on idle clients are safely logged and absorbed.
  3. Wrapped cleanup functions inside try/catch blocks for added defensiveness.

Fixes #10549

Scenarios Tested

  • Successfully ran all dataconnect and cloudsql unit tests.
  • Verified teardown sequence completes gracefully without crash using a local reproduction script.
  • Verified fix handles both connect.ts deployments and dataconnect-sql-shell.ts shell exits.

Sample Commands

firebase deploy --only dataconnect --project <project-id>

Description

Scenarios Tested

Sample Commands

### Description
This commit resolves an issue where deploying a Data Connect service backed by Cloud SQL Postgres fails with an unexpected unhandled exception (`CloudSQLConnectorError: The connector was closed`).

In `@google-cloud/cloud-sql-connector` version `>= 1.7.0`, the `connector.close()` method destroys underlying sockets by asynchronously emitting an `error` event. Previously, `firebase-tools` closed the `pg.Pool` first, which caused the database client to detach all error listeners from its sockets. Consequently, the later `connector.close()` call caused Node.js to crash due to an unhandled exception on those sockets.

To prevent this:
1. Re-ordered cleanup sequences so that `connector.close()` is executed prior to `pool.end()`.
2. Attached an explicit `'error'` handler to `pg.Pool` to ensure any connection teardown errors on idle clients are safely logged and absorbed.
3. Wrapped cleanup functions inside `try/catch` blocks for added defensiveness.

Fixes #10549

### Scenarios Tested
- Successfully ran all dataconnect and cloudsql unit tests.
- Verified teardown sequence completes gracefully without crash using a local reproduction script.
- Verified fix handles both `connect.ts` deployments and `dataconnect-sql-shell.ts` shell exits.

### Sample Commands
`firebase deploy --only dataconnect --project <project-id>`
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds error handling to the PostgreSQL pool and wraps cleanup operations in try/catch blocks to prevent unhandled exceptions during Data Connect deployments. However, the reviewer identified critical issues where resource leaks can occur if pool.connect() fails, as the cleanup functions would not be executed. Additionally, declaring conn after cleanUpFn in connect.ts can trigger linter warnings due to referencing a variable before its declaration.

Comment thread src/gcp/cloudsql/connect.ts Outdated
pool.on("error", (err) => {
logger.debug("PostgreSQL pool error:", err);
});
const conn: pg.PoolClient = await pool.connect();
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If pool.connect() fails and throws an error, the command action will reject immediately without executing the cleanup block at the end of the function. This results in a resource leak where the connector and pool are left open.

Wrapping pool.connect() in a try/catch block to clean up resources on failure prevents this leak.

Suggested change
const conn: pg.PoolClient = await pool.connect();
let conn: pg.PoolClient;
try {
conn = await pool.connect();
} catch (err) {
try {
connector.close();
} catch (closeErr) {
logger.debug("Error closing Cloud SQL connector:", closeErr);
}
try {
await pool.end();
} catch (endErr) {
logger.debug("Error ending pg pool:", endErr);
}
throw err;
}

@fredzqm fredzqm closed this May 26, 2026
@fredzqm fredzqm reopened this May 26, 2026
@fredzqm fredzqm requested a review from joehan May 26, 2026 19:23
Copy link
Copy Markdown
Member

@joehan joehan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even better than pinning - thanks!

@fredzqm fredzqm enabled auto-merge (squash) May 26, 2026 21:20
@fredzqm fredzqm merged commit fc87fca into main May 26, 2026
50 of 56 checks passed
@fredzqm fredzqm deleted the fix/cloud-sql-connector-unhandled-exception branch May 26, 2026 21:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

firebase deploy --only dataconnect fails with "CloudSQLConnectorError: The connector was closed" (transitive @google-cloud/cloud-sql-connector ≥1.7.0)

3 participants