Activity delivery error stores unbounded response body in memory, causing memory pressure under high failure rates

[dahlia]

Summary

When sendActivity() fails due to a non-2xx HTTP response, the entire response body is read via response.text() without any size limit and stored in both the error message string and the log output. Many remote servers (especially those behind Cloudflare or other reverse proxies) return full HTML error pages (50–100 KB each) on failure. Under production conditions with many unreachable inboxes, this creates significant memory pressure and contributes to V8 OOM crashes.

Affected Code

In packages/fedify/src/federation/send.ts, inside sendActivityInternal():

if (!response.ok) {
  let error;
  try {
    error = await response.text(); // Reads entire response body without limit
  } catch (_) {
    error = "";
  }
  logger.error(
    "Failed to send activity {activityId} to {inbox} ({status} " +
      "{statusText}):\n{error}",
    { activityId, inbox: inbox.href, status: response.status,
      statusText: response.statusText, error },
  );
  throw new Error(
    `Failed to send activity ${activityId} to ${inbox.href} ` +
      `(${response.status} ${response.statusText}):\n${error}`,
    // ^^^^^^ Full HTML body embedded in Error message
  );
}

This pattern also exists in the main branch (2.0.0-dev) with SendActivityError, where the full body is stored in both message and the responseBody property.

Actual Behavior

In a production Hackers' Pub deployment (3 instances, Fedify 1.10.0), we observed:

• ~1,300 failed activity sends per instance in a single lifecycle
• Error responses from dead/misconfigured servers include full Cloudflare error pages (~50–100 KB HTML each), phpBB error pages, and other large HTML documents
• These large strings are allocated for each failure: once in response.text(), once in the Error message, and once in the log output
• Combined with other memory pressure (e.g., frequent retries via the outbox retry policy), this contributes to repeated “Fatal JavaScript out of memory: Ineffective mark-compacts near heap limit” crashes at ~510 MB V8 heap

Expected Behavior

The error response body should be bounded to a reasonable size (e.g., 1 KB) to prevent unbounded memory allocation from error responses. The HTTP status code and status text alone are usually sufficient for debugging delivery failures.

Proposed Solution

Truncate the response body before storing it:

if (!response.ok) {
  let error;
  try {
    const raw = await response.text();
    error = raw.length > 1024 ? raw.substring(0, 1024) + "… (truncated)" : raw;
  } catch (_) {
    error = "";
  }
  // ...
}

Alternatively:

• Skip reading the body entirely when Content-Type is text/html (HTML error pages are rarely useful for debugging ActivityPub delivery failures)
• Use a streaming approach to read only the first N bytes instead of buffering the entire response

Environment

• Fedify: 1.10.0 (also confirmed in main branch / 2.0.0-dev)
• Runtime: Deno (Docker, linux/arm64)
• 3 app instances sharing a PostgresMessageQueue
• Many followers on dead/unreachable servers (Cloudflare tunnels down, expired domains, DNS failures, etc.)