Skip to content

Complex collections cause OOM with Postgres #12341

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jeepman32 opened this issue May 8, 2025 · 2 comments · Fixed by #12193
Closed

Complex collections cause OOM with Postgres #12341

jeepman32 opened this issue May 8, 2025 · 2 comments · Fixed by #12193

Comments

@jeepman32
Copy link
Contributor

jeepman32 commented May 8, 2025

Describe the Bug

Fix

PR is available and open here

When

  • Using pnpm dev
  • With collections with select fields that have large enums
  • Using Postgres

Then

The Node instance Payload is runing on will eventually crash due to an OOM.

Where

The schema comparison step inside packages/drizzle/src/utilities/pushDevSchema.ts:26

Why

The method of comparing the old and new schemas uses node:assert.deepStrictEqual which has poor performance, especially with larger objects. Over time, Payload projects using Postgres with large/complex schemas will become slower, eventually running into physical memory limits.

Resources

Previously posted discussion

Memory Usage Time to completion
deepStrictEquals timing

Link to the code that reproduces this issue

https://github.com/jeepman32/schema-pulling-oom

Reproduction Steps

These steps aren't required for the reproduction repo, but are useful if trying to create a fresh reproduction.

  1. Create a project using the Payload blank template, using Postgres
  2. Create many collections, at least 45, that use a select field with at least 1500 options
  3. Attempt to run pnpm/npm/yarn run dev
  4. Payload will crash due to OOM.

Which area(s) are affected? (Select all that apply)

db-postgres, area: core

Environment Info

Binaries:
  Node: 22.14.0
  npm: 11.3.0
  Yarn: 1.22.22
  pnpm: 10.10.0
Relevant Packages:
  payload: 3.37.0
  next: 15.3.0
  @payloadcms/db-postgres: 3.37.0
  @payloadcms/graphql: 3.37.0
  @payloadcms/next/utilities: 3.37.0
  @payloadcms/richtext-lexical: 3.37.0
  @payloadcms/translations: 3.37.0
  @payloadcms/ui/shared: 3.37.0
  react: 19.1.0
  react-dom: 19.1.0
Operating System:
  Platform: linux
  Arch: x64
  Version: #24-Ubuntu SMP PREEMPT_DYNAMIC Fri Mar 14 18:13:56 UTC 2025
  Available memory (MB): 28805
  Available CPU cores: 16
@jeepman32 jeepman32 added status: needs-triage Possible bug which hasn't been reproduced yet validate-reproduction Auto-added tag on create to tell bot to check recreation URL, removed after check. labels May 8, 2025
@github-actions github-actions bot removed the validate-reproduction Auto-added tag on create to tell bot to check recreation URL, removed after check. label May 8, 2025
@r1tsuu r1tsuu linked a pull request May 8, 2025 that will close this issue
@github-actions github-actions bot removed the status: needs-triage Possible bug which hasn't been reproduced yet label May 8, 2025
Copy link
Contributor

🚀 This is included in version v3.38.0

Copy link
Contributor

This issue has been automatically locked.
Please open a new issue if this issue persists with any additional detail.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 23, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant