fix: set pulp-api --max-requests to 5000 to prevent OOM

[dkliban]

Problem

After removing the explicit --max-requests=20 override (which was causing SLI histogram corruption), pulp-api workers rely on pulpcore's default. However, there is a memory leak (~60 MiB/hr) that causes workers to grow from ~850 MiB to 3.8+ GiB within 24 hours, resulting in OOMKilled events (observed on pod pulp-api-5864b7b559-zfczh at 2026-04-29T00:01:41Z).

Fix

Set --max-requests=5000 (jitter 500) explicitly on the pulp-api command. This:

• Prevents OOM: Workers recycle well before hitting the 5120 MiB limit
• Preserves SLI accuracy: Counter values reach ~5000 before reset, making coincidental value matches (the root cause of SLI > 1) statistically negligible compared to the previous value of 20

Test plan

• Deploy to stage — monitor memory usage, confirm no OOM over 24h
• Deploy to production — confirm APIWriteRequestLatency SLI stays ≤ 1 and no OOMKilled events

🤖 Generated with Claude Code

Summary by Sourcery

Configure pulp-api Gunicorn workers to recycle after a bounded number of requests to mitigate memory growth and OOM events while preserving metrics accuracy.

Enhancements:

• Add configurable max-requests and jitter settings to the pulp-api Gunicorn command in the ClowdApp deployment manifest.
• Introduce new PULP_API_GUNICORN_MAX_REQUESTS and PULP_API_GUNICORN_MAX_REQUESTS_JITTER parameters with defaults tuned to balance memory usage and SLI accuracy.

[sourcery-ai]

Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

Configures pulp-api Gunicorn workers to recycle after ~5000 requests with jitter to mitigate a memory leak and prevent OOMs while preserving SLI accuracy, by wiring new max-requests settings into the deployment manifest as parameters and command-line flags.

Flow diagram for Gunicorn worker lifecycle with max-requests and jitter

flowchart TD
  Start[Worker start] --> Init[Initialize request_count to 0<br>Load max_requests from PULP_API_GUNICORN_MAX_REQUESTS<br>Load jitter from PULP_API_GUNICORN_MAX_REQUESTS_JITTER]

  Init --> ComputeLimit[Compute effective_limit = max_requests + random_jitter]
  ComputeLimit --> WaitReq[Wait for next HTTP request]

  WaitReq --> HandleReq[Handle request]
  HandleReq --> IncCount[Increment request_count]

  IncCount --> CheckLimit{request_count >= effective_limit?}
  CheckLimit -->|No| WaitReq
  CheckLimit -->|Yes| GracefulShutdown[Finish in-flight work and shut down worker]

  GracefulShutdown --> WorkerRestart[Master starts replacement worker]
  WorkerRestart --> Start

Loading

File-Level Changes

Change	Details	Files
Configure pulp-api Gunicorn workers with explicit max-requests and jitter settings via deployment parameters and container command arguments.	Extend the pulp-api container command to pass --max-requests and --max-requests-jitter flags using environment-driven values. Introduce new ClowdApp parameters for PULP_API_GUNICORN_MAX_REQUESTS and PULP_API_GUNICORN_MAX_REQUESTS_JITTER with defaults of 5000 and 500 respectively, including descriptive comments. Keep existing timeout and workers configuration unchanged while integrating the new options into the existing command line in a backward-compatible way.	deploy/clowdapp.yaml

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've reviewed your changes and they look great!

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}