Pandaproxy slow first message produce?

[danthegoodman1]

I notced that if I do a request like:

curl -s \
    -X POST \
    "http://localhost:8082/topics/my-topic" \
    -H "Content-Type: application/vnd.kafka.json.v2+json" \
    -d '{
    "records":[{
            "key":1,
            "value": {
              "num": 1
            }
        }]
    }'

The first request can have a latency upwards of 10s, however after the first one is completed then I observe super low latency requests. Is this something that is undocumented or unintended?

[BenPope]

Hi @danthegoodman1 - 10s seems like a long time, but I have just verified it locally.

There are a couple of things that can slow it down:

1. If the topic doesn't exist, and it's being auto created
2. Pandaproxy is in-process with Redpanda, but still needs to connect clients to each of the Redpanda nodes - this process is started on the first request to Pandaproxy
3. The clients have an exponential backoff on error, which is likely the major cause of the delay - check the configuration options
4. The client does probably over-aggressive batching at 100ms - again this can be tuned in the configuration

My suggestion is to optionally explicitly create the topic and then to issue a list of topics prior to producing:

rpk create topic my-topic
curl -s localhost:8082/topics

I've raised an issue: #3321

[danthegoodman1]

I bet it’s topic creation, thanks for the swift response!

[danthegoodman1]

I don’t suspect the second because I was running a single node (at least in this case)