Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(server): introduce maxResultSize limit and fix pg errors handling #911

Merged
merged 1 commit into from
Mar 27, 2025

Conversation

avallete
Copy link
Member

@avallete avallete commented Mar 25, 2025

What kind of change does this PR introduce?

  • Fixes the way we handled pg.Pool non "database" error (connection level errors) to avoid throwing unhandled exception, but instead make sure it get caught and propagated to the caller in the response via poolerQueryHandleError
  • Going along with feat: add maxResultSize limit node-postgres#1 introducing a way to cap the maximum size we allow a query to take in memory before aborting with an error.

With those changes the running server should be more robust, die less often and leverage killing the connections sooner than later.

Fixes PGMETA-45

Copy link
Member Author

@avallete avallete left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will need more testing on staging, and will need close monitoring on prod deploy. But should help reduce the spikes of 5xx errors we see.

if (Array.isArray(res)) {
res = res.reverse().find((x) => x.rows.length !== 0) ?? { rows: [] }
}
return { data: res.rows, error: null }
} catch (error: any) {
if (error instanceof DatabaseError) {
if (error.constructor.name === 'DatabaseError') {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note

With the new error propagation, had to slightly tweak this as the instanceof wasn't properly working. Even tough the constructor was the right one, the instance kept being an Error because of the promise wrapping.

@coveralls
Copy link

coveralls commented Mar 26, 2025

Pull Request Test Coverage Report for Build 14112748589

Details

  • 61 of 65 (93.85%) changed or added relevant lines in 4 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.5%) to 75.947%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/server/constants.ts 4 5 80.0%
src/lib/db.ts 53 56 94.64%
Totals Coverage Status
Change from base Build 13958195217: 0.5%
Covered Lines: 4991
Relevant Lines: 6490

💛 - Coveralls

@avallete avallete marked this pull request as ready for review March 27, 2025 00:03
@avallete avallete requested review from a team as code owners March 27, 2025 00:03
Copy link

linear bot commented Mar 27, 2025

@avallete avallete force-pushed the avallete/pg-add-maximum-query-resultsize-limit branch from 25bec58 to 9d831d7 Compare March 27, 2025 17:17
@avallete avallete requested a review from soedirgo March 27, 2025 17:20
@avallete avallete merged commit 1be9d45 into master Mar 27, 2025
5 checks passed
@avallete avallete deleted the avallete/pg-add-maximum-query-resultsize-limit branch March 27, 2025 20:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants