inspector,http: support builtin http request bodies

[GrinZero]

Summary

This PR adds builtin http/https request-body support to network inspection, so Network.getRequestPostData works for text request bodies while preserving the existing rejection behavior for binary request bodies.

It also moves builtin http response-body tracking to a raw-byte hook before IncomingMessage decoding, so response inspection remains correct even when user code calls response.setEncoding(...).

This closes part of the remaining postData gap tracked in the network-inspection stabilization issue.

Problem

Builtin http/https network inspection currently emits:

• Network.requestWillBeSent
• Network.responseReceived
• Network.loadingFinished

However, it does not emit request-body data for the builtin http client path. As a result, Network.getRequestPostData cannot return POST data for builtin http/https requests.

In the tracking issue for stabilizing network inspection, builtin http/https request postData is still marked as needing further investigation. This change targets that specific gap.

While working on that, this PR also addresses an important response-side edge case: listening to IncomingMessage 'data' events is not a stable source of raw bytes.

When user code calls:

response.setEncoding('utf8')

the chunks observed by userland change from Buffer objects into strings. The inspector protocol expects byte-oriented payloads for Network.dataReceived and Network.dataSent, so reconstructing bytes from already-decoded strings is only a best-effort fallback and can lose the original payload.

Approach

1. Reuse the existing request buffering pipeline

This change does not modify the CDP schema or the C++ buffering logic in NetworkAgent.

Instead, it reuses the existing pipeline already used by other transports:

Network.dataSent(...) -> NetworkAgent::getRequestPostData(...)

2. Add builtin http request-body diagnostics events

The builtin http client now publishes:

• http.client.request.bodyChunkSent
• http.client.request.bodySent

These events are emitted from the ClientRequest write path before HTTP framing is applied, so the inspector sees the original user payload rather than chunked-transfer framing bytes.

That makes the behavior consistent with the existing undici and http2 network-inspection implementations.

3. Capture builtin http response bytes before decoding

For responses, this PR intentionally avoids relying on IncomingMessage.on('data') in network_http.js.

Instead, it adds:

• http.client.response.bodyChunkReceived

from the HTTP parser body callback in _http_common.js.

That hook runs before Readable.setEncoding() transforms chunks for userland, so the inspector always receives raw bytes. This avoids issues such as:

• missing dataLength or wrong event shape when user code receives strings
• loss of byte-for-byte fidelity when decoded strings are re-encoded
• protocol mismatches for Network.dataReceived

4. Prefer raw bytes over string re-encoding

A temporary compatibility fix could convert string chunks back into Buffers, but that is not equivalent to preserving the original bytes:

• text decoding may already have normalized or replaced invalid sequences
• binary responses observed after setEncoding() are no longer raw bytes
• the inspector should reflect transport-level bytes, not post-decoding reconstruction

So the final implementation moves the builtin http response path to the same principle used by external tooling: capture bytes first, decode later if needed.

This is also consistent with the general handling in node-network-devtools, where network payload processing is built around Buffer data rather than post-decoding string chunks.

Behavior

After this change:

• builtin http and https POST requests with UTF-8 text bodies are available through Network.getRequestPostData
• binary request bodies still reject with the existing inspector error behavior
• builtin http response inspection continues to work even if user code calls response.setEncoding('utf8')

Tests

This PR adds and extends coverage in:

• test/parallel/test-diagnostics-channel-http.js
• test/parallel/test-inspector-network-http.js

The updated tests cover:

• request body chunk and request body finished diagnostics events
• text request bodies split across write() and end()
• binary request bodies
• http and https Network.getRequestPostData for text bodies
• binary request-body rejection semantics
• response inspection when the client calls response.setEncoding('utf8')

Verification

Verified locally with:

python3 tools/test.py \
  parallel/test-diagnostics-channel-http \
  parallel/test-inspector-network-http

Both tests passed locally with the built out/Release/node.

Refs

• Tracking issue: Stabilizing network inspection in Node.js #53946

[nodejs-github-bot]

Review requested:

• @nodejs/http
• @nodejs/inspector
• @nodejs/net

[GrinZero]

E2E Test:

const http = require("http");

const DEFAULT_TARGET_URL = "http://jsonplaceholder.typicode.com/posts";
const DEFAULT_PORT = Number(process.env.PORT || 3000);
const DEFAULT_HOST = "127.0.0.1";
const targetUrl = process.argv[2] || process.env.TARGET_URL || DEFAULT_TARGET_URL;
const defaultPayload = {
  title: "node inspect demo",
  body: "post payload from local trigger service",
  userId: 123,
};

if (!targetUrl.startsWith("http://")) {
  console.error(
    `[config] This script only uses the http module for outbound requests. Use an http:// URL, got: ${targetUrl}`
  );
  process.exit(1);
}

function readRequestBody(req) {
  return new Promise((resolve, reject) => {
    let body = "";

    req.setEncoding("utf8");
    req.on("data", (chunk) => {
      body += chunk;
    });
    req.on("end", () => {
      resolve(body);
    });
    req.on("error", reject);
  });
}

function buildPayload(rawBody) {
  if (!rawBody || !rawBody.trim()) {
    return defaultPayload;
  }

  const parsed = JSON.parse(rawBody);
  if (!parsed || Array.isArray(parsed) || typeof parsed !== "object") {
    throw new Error("payload must be a JSON object");
  }

  return {
    ...defaultPayload,
    ...parsed,
  };
}

function sendOutboundPost(url, payload) {
  return new Promise((resolve, reject) => {
    const body = JSON.stringify(payload);

    console.log(`\n[outbound] request -> POST ${url}`);
    console.log(`[outbound] request payload: ${body}`);

    const outboundReq = http.request(
      url,
      {
        method: "POST",
        headers: {
          "user-agent": "node-inspect-http-demo",
          accept: "application/json,text/plain,*/*",
          "content-type": "application/json; charset=utf-8",
          "content-length": Buffer.byteLength(body),
        },
      },
      (outboundRes) => {
        let responseBody = "";
        outboundRes.setEncoding("utf8");

        console.log(
          `[outbound] response <- ${outboundRes.statusCode} ${outboundRes.statusMessage || ""}`.trim()
        );
        console.log("[outbound] response headers:", outboundRes.headers);

        outboundRes.on("data", (chunk) => {
          responseBody += chunk;
        });

        outboundRes.on("end", () => {
          console.log("[outbound] body preview:");
          console.log(responseBody.slice(0, 300) || "<empty>");

          resolve({
            statusCode: outboundRes.statusCode,
            statusMessage: outboundRes.statusMessage,
            headers: outboundRes.headers,
            bodyPreview: responseBody.slice(0, 300),
          });
        });
      }
    );

    outboundReq.on("socket", (socket) => {
      socket.on("connect", () => {
        console.log(
          `[outbound] socket connected -> ${socket.remoteAddress}:${socket.remotePort}`
        );
      });
    });

    outboundReq.on("finish", () => {
      console.log("[outbound] request finished");
    });

    outboundReq.on("error", (error) => {
      console.error("[outbound] request error:", error.message);
      reject(error);
    });

    outboundReq.write(body);
    outboundReq.end();
  });
}

function writeJson(res, statusCode, payload) {
  const body = JSON.stringify(payload, null, 2);
  res.writeHead(statusCode, {
    "content-type": "application/json; charset=utf-8",
    "content-length": Buffer.byteLength(body),
  });
  res.end(body);
}

const server = http.createServer(async (req, res) => {
  const url = new URL(req.url || "/", `http://${req.headers.host || "127.0.0.1"}`);

  if (req.method === "GET" && url.pathname === "/") {
    return writeJson(res, 200, {
      ok: true,
      message: "local trigger service is running",
      endpoints: {
        trigger: "POST /trigger",
      },
      targetUrl,
      defaultPayload,
    });
  }

  if (req.method === "POST" && url.pathname === "/trigger") {
    console.log(`\n[inbound] trigger <- ${req.method} ${url.pathname}`);

    try {
      const rawBody = await readRequestBody(req);
      const payload = buildPayload(rawBody);
      const outbound = await sendOutboundPost(targetUrl, payload);

      return writeJson(res, 200, {
        ok: true,
        targetUrl,
        payload,
        outbound,
      });
    } catch (error) {
      return writeJson(res, 500, {
        ok: false,
        error: error.message,
      });
    }
  }

  return writeJson(res, 404, {
    ok: false,
    error: "not found",
  });
});

server.listen(DEFAULT_PORT, DEFAULT_HOST, () => {
  console.log(`[server] listening on http://${DEFAULT_HOST}:${DEFAULT_PORT}`);
  console.log(`[server] outbound target: ${targetUrl}`);
  console.log("[usage] node --inspect-wait --experimental-network-inspection inspect-http.js");
  console.log(
    `[usage] curl -X POST http://${DEFAULT_HOST}:${DEFAULT_PORT}/trigger -H 'content-type: application/json' -d '{"title":"manual trigger"}'`
  );
});

[metcoder95]

lgtm, it just seems that is gonna need a documentation for the new dcs

[GrinZero]

Thanks for the feedback. I’ve updated the documentation in doc/api/diagnostics_channel.md to address the review comments and include the new HTTP diagnostics channel events.

Specifically, I added/updated entries for:

• http.client.request.bodyChunkSent
• http.client.request.bodySent
• http.client.response.bodyChunkReceived

I also aligned the wording and structure with the existing Node.js docs style for built-in channels.

[GrinZero]

A slight push, hoping that a maintainer will notice.

[GrinZero]

Sorry for the ping, @legendecas

If you have some time, could you please help review this PR and trigger CI?

This PR fixes an issue where the network inspector does not show postData for http / https requests.

Thanks a lot!

[GrinZero]

push, request-ci plz

[marco-ippolito]

Ill try to review the PR later, I think it's important we make sure it doesn't leak memory or keep the body buffer reference alive, which can exhaust resource quickly @ShogunPanda ptal

[GrinZero]

Ill try to review the PR later, I think it's important we make sure it doesn't leak memory or keep the body buffer reference alive, which can exhaust resource quickly @ShogunPanda ptal

I'll go explore the issue you mentioned

[GrinZero]

Thanks for raising this concern.

I did a focused memory/lifetime check for this PR, mainly around two questions:

1. whether the inspector keeps the original userland request/response body buffers alive;
2. whether the new body buffering path can grow without bound.

Based on the current implementation and local testing, I do not see evidence that this PR keeps the original JS buffers alive.

For both request and response bodies, the payload bytes are copied into inspector-owned storage before being retained by NetworkAgent. I also verified this with a WeakRef/GC test: the original 4 MiB request body buffer and the observed response chunk both became collectible after the request completed, while Network.getRequestPostData() / Network.getResponseBody() could still return the payloads.

So the inspector is retaining copied payload bytes, not the original userland Buffer objects.

I also ran a repeated-request memory test with explicit Network.enable() limits. Memory increased initially, as expected from inspector-side buffering, but then reached a plateau and older requests were evicted. That behavior looks consistent with bounded retention by the existing request buffer manager, not an unbounded leak.

[GrinZero]

The test plan above was proposed by AI, and I reviewed it.

For the first test, it uses WeakRef, manual GC, and process.memoryUsage() to verify object lifetime. I think the underlying testing principle is reasonably credible.

The second test is a stress test. It lowers maxTotalBufferSize and maxResourceBufferSize, then sends multiple requests with relatively large bodies. I’m less familiar with this approach.

[GrinZero]

I merged the main branch code to avoid CI errors.