Encountering "501 FRAME_ERROR" during asynchronous batch publishing

[fanmaomao]

I'm running into a bit of a strange issue when trying to publish messages in batches asynchronously using this library.
I'm consistently getting a "501 FRAME_ERROR" when processing a specific set of messages.

Here's the code I'm using for publishing:

const messages = []; // my datas
const amqp = new AMQPClient(process.env.RabbitMQ_URL);
const connection = await amqp.connect();
const channel = await connection.channel();
const queue = await channel.queue(process.env.RabbitMQ_QUEUE);
await channel.confirmSelect();
await Promise.all(messages.map(message => {
  return queue.publish(
    JSON.stringify(message),
    {
      contentType: 'application/json',
      deliveryMode: 2,
    }
  );
}));

And this is the error output I see:

connection closed by server 501 FRAME_ERROR - type 3, first 16 octets = <<"{\"reqId\":\"4\",\"en">>: {invalid_frame_end_marker,0} 0 0
channel 0 closed: connection closed: FRAME_ERROR - type 3, first 16 octets = <<"{\"reqId\":\"4\",\"en">>: {invalid_frame_end_marker,0} (501)
channel 1 closed: connection closed: FRAME_ERROR - type 3, first 16 octets = <<"{\"reqId\":\"4\",\"en">>: {invalid_frame_end_marker,0} (501)
amqp-client connection closed connection closed: FRAME_ERROR - type 3, first 16 octets = <<"{\"reqId\":\"4\",\"en">>: {invalid_frame_end_marker,0} (501)

---

Initially, I suspected the issue might be related to the size of the messages, perhaps if they were too large. So, I tried creating a batch with larger messages to test this:

const messages = [
  { cloudfrontReqId: '1', msgId: Buffer.from(crypto.getRandomValues(new Uint8Array(1000))).toString('base64') },
  { cloudfrontReqId: '2', msgId: Buffer.from(crypto.getRandomValues(new Uint8Array(1000))).toString('base64') },
  { cloudfrontReqId: '3', msgId: Buffer.from(crypto.getRandomValues(new Uint8Array(1000))).toString('base64') },
  { cloudfrontReqId: '4', msgId: Buffer.from(crypto.getRandomValues(new Uint8Array(1000))).toString('base64') },
];

Interestingly, with these larger messages, the error did not occur. This made me think it might be something specific about the content of the messages rather than just their size.

---

Here is the exact batch of data that triggers the "501 FRAME_ERROR":

const messages = [
  {
    reqId: '1',
    entry: {
      entryType: 'resource',
      startTime: 2213551,
      duration: 1745.300000011921,
      initiatorType: 'xmlhttprequest',
      deliveryType: '',
      nextHopProtocol: 'h2',
      renderBlockingStatus: 'non-blocking',
      workerStart: 0,
      redirectStart: 0,
      redirectEnd: 0,
      fetchStart: 2213551,
      domainLookupStart: 2213551,
      domainLookupEnd: 2213551,
      connectStart: 2213551,
      secureConnectionStart: 2213551,
      connectEnd: 2213551,
      requestStart: 2214132.200000018,
      responseStart: 2215295.5,
      firstInterimResponseStart: 0,
      finalResponseHeadersStart: 2215295.5,
      responseEnd: 2215296.300000012,
      transferSize: 384,
      encodedBodySize: 84,
      decodedBodySize: 68,
      responseStatus: 200,
      serverTiming: []
    }
  },
  {
    reqId: '2',
    entry: {
      entryType: 'resource',
      startTime: 2215299.5,
      duration: 167.2000000178814,
      initiatorType: 'xmlhttprequest',
      deliveryType: '',
      nextHopProtocol: 'h2',
      renderBlockingStatus: 'non-blocking',
      workerStart: 0,
      redirectStart: 0,
      redirectEnd: 0,
      fetchStart: 2215299.5,
      domainLookupStart: 2215299.5,
      domainLookupEnd: 2215299.5,
      connectStart: 2215299.5,
      secureConnectionStart: 2215299.5,
      connectEnd: 2215299.5,
      requestStart: 2215374.600000024,
      responseStart: 2215466.100000024,
      firstInterimResponseStart: 0,
      finalResponseHeadersStart: 2215466.100000024,
      responseEnd: 2215466.700000018,
      transferSize: 1822,
      encodedBodySize: 1522,
      decodedBodySize: 4028,
      responseStatus: 200,
      serverTiming: []
    }
  },
  {
    reqId: '3',
    entry: {
      entryType: 'resource',
      startTime: 2215468.700000018,
      duration: 205.7999999821186,
      initiatorType: 'xmlhttprequest',
      deliveryType: '',
      nextHopProtocol: 'h2',
      renderBlockingStatus: 'non-blocking',
      workerStart: 0,
      redirectStart: 0,
      redirectEnd: 0,
      fetchStart: 2215468.700000018,
      domainLookupStart: 2215468.700000018,
      domainLookupEnd: 2215468.700000018,
      connectStart: 2215468.700000018,
      secureConnectionStart: 2215468.700000018,
      connectEnd: 2215468.700000018,
      requestStart: 2215546.800000012,
      responseStart: 2215673.800000012,
      firstInterimResponseStart: 0,
      finalResponseHeadersStart: 2215673.800000012,
      responseEnd: 2215674.5,
      transferSize: 10506,
      encodedBodySize: 10206,
      decodedBodySize: 69146,
      responseStatus: 200,
      serverTiming: []
    }
  },
  {
    reqId: '4',
    entry: {
      entryType: 'resource',
      startTime: 2215970.400000006,
      duration: 501.09999999403954,
      initiatorType: 'xmlhttprequest',
      deliveryType: '',
      nextHopProtocol: 'h2',
      renderBlockingStatus: 'non-blocking',
      workerStart: 0,
      redirectStart: 0,
      redirectEnd: 0,
      fetchStart: 2215970.400000006,
      domainLookupStart: 2215970.400000006,
      domainLookupEnd: 2215970.400000006,
      connectStart: 2215970.400000006,
      secureConnectionStart: 2215970.400000006,
      connectEnd: 2215970.400000006,
      requestStart: 2215970.900000006,
      responseStart: 2216470.600000024,
      firstInterimResponseStart: 0,
      finalResponseHeadersStart: 2216470.600000024,
      responseEnd: 2216471.5,
      transferSize: 384,
      encodedBodySize: 84,
      decodedBodySize: 68,
      responseStatus: 200,
      serverTiming: []
    }
  }
];

This issue is blocking me from reliably publishing this type of data in batches.

---

Here are the relevant versions:

• amqp-client: v3.2.1
• Node.js: v20
• RabbitMQ: 3.13.7

Any help or pointers on what might be going wrong here would be greatly appreciated!
Thanks so much for your time.

[geminigeek]

exact same issue, started happening in lavinmq since 2.3.0

[geminigeek]

exact same issue, started happening in lavinmq since 2.3.0

checked with bun1.2.3 it works , i think the issue is somewhere in node

[carlhoerberg]

Hmm, could not reproduce locally :/

[fanmaomao]

Following up on this issue, I've done a bit more digging and wanted to share my latest findings.

Initially, I first hit this problem when using AWS's RabbitMQ service with the amqps protocol.

After doing some further testing locally, I'm starting to suspect the issue might specifically be related to using TLS/SSL.

The good news is, I can consistently reproduce this locally now! This should hopefully make it easier to pinpoint the root cause.

Here's a step-by-step breakdown of how I can reliably reproduce the "FRAME_ERROR" I'm seeing when connecting via AMQPS with TLS:

1. Generate Self-Signed Certificates:
   First, I generate a basic set of self-signed certificates for testing.

mkdir rabbitmq_certs
cd rabbitmq_certs

# 1. Generate CA private key
openssl genrsa -out ca_key.pem 2048
# 2. Generate CA certificate
openssl req -new -x509 -days 3650 -key ca_key.pem -out ca_certificate.pem \
  -subj "/C=CN/ST=GD/L=SZ/O=MyOrg/OU=MyOU/CN=MyRabbitMQCA"
# 3. Generate server private key
openssl genrsa -out server_key.pem 2048
# 4. Generate server CSR (Certificate Signing Request)
openssl req -new -key server_key.pem -out server_csr.pem \
  -subj "/C=CN/ST=GD/L=SZ/O=MyOrg/OU=MyServer/CN=localhost"
# 5. Sign the server CSR with the CA key to get the server certificate
echo "subjectAltName = DNS:localhost,IP:127.0.0.1" > extfile.cnf
openssl x509 -req -days 365 -in server_csr.pem -CA ca_certificate.pem -CAkey ca_key.pem \
  -CAcreateserial -out server_certificate.pem -extfile extfile.cnf
# 6. Clean up intermediate files (optional)
rm server_csr.pem extfile.cnf ca_key.pem ca.srl

cd ..

2. Configure RabbitMQ:

# ./rabbitmq.conf

listeners.tcp.default = 5672
listeners.ssl.default = 5671

ssl_options.cacertfile = /etc/rabbitmq/certs/ca_certificate.pem
ssl_options.certfile   = /etc/rabbitmq/certs/server_certificate.pem
ssl_options.keyfile    = /etc/rabbitmq/certs/server_key.pem
ssl_options.verify     = verify_none # For simplicity in testing
ssl_options.fail_if_no_peer_cert = false # For simplicity in testing


management.tcp.port = 15672
management.listener.port = 15671
management.listener.ssl = true
management.listener.ssl_opts.cacertfile = /etc/rabbitmq/certs/ca_certificate.pem
management.listener.ssl_opts.certfile   = /etc/rabbitmq/certs/server_certificate.pem
management.listener.ssl_opts.keyfile    = /etc/rabbitmq/certs/server_key.pem
management.listener.ssl_opts.verify     = verify_none # For simplicity
management.listener.ssl_opts.fail_if_no_peer_cert = false # For simplicity

3. Run RabbitMQ in Docker:

docker run --rm --name rabbitmq-tls \
  -p 5671:5671 \
  -p 5672:5672 \
  -p 15671:15671 \
  -p 15672:15672 \
  -v "./rabbitmq_certs:/etc/rabbitmq/certs:ro" \
  -v "./rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf:ro" \
  -e RABBITMQ_DEFAULT_USER=test \
  -e RABBITMQ_DEFAULT_PASS=test123 \
  rabbitmq:3.13.7-management-alpine

opening https://localhost:15671 in your browser

4. Change the Test Code:

import { readFileSync } from 'node:fs';
// const amqp = new AMQPClient('amqp://test:test123@localhost:5672/%2f?heartbeat=10'); // it works 😊
const amqp = new AMQPClient('amqps://test:test123@localhost:5671/%2f?heartbeat=10', {
  cert: readFileSync('./rabbitmq_certs/server_certificate.pem'),
  key: readFileSync('./rabbitmq_certs/server_key.pem'),
  ca: readFileSync('./rabbitmq_certs/ca_certificate.pem'),
});

I hope these detailed steps help you reproduce the issue on your end.
Please let me know if anything is unclear or if you need any more information from me.

[carlhoerberg]

Thank you! That helps!

[fanmaomao]

Hi @carlhoerberg, just checking in to see if there have been any updates on the bug fix.
Totally understand if you're busy — no rush, just hoping to stay in the loop.
Thanks again for your work on this project!