Kong startup logs [error] still in larger deployments

[jeremyjpj0916]

Is there an existing issue for this?

• I have searched the existing issues

Kong version ($ kong version)

3.9.0

Current Behavior

I have brought this up in the past but still seeing it in our bigger proxy environment(more kong routes/services/plugins etc. Probably 5000+ proxies in total) But startup logs look like this, running Kong in traditional mode:

2025-03-04T00:36:38.376574776-06:00 
2025-03-04T00:36:38.655150112-06:00 nginx: [warn] the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /usr/local/kong/nginx.conf:10
2025-03-04T00:36:38.655581907-06:00 nginx: [warn] load balancing method redefined in /usr/local/kong/nginx.conf:117
2025-03-04T00:36:38.736729604-06:00 2025/03/04 06:36:38 [notice] 26#0: [lua] client.lua:161: new(): [dns_client] supported types: srv ipv4 ipv6 
2025-03-04T00:36:38.814449460-06:00 2025/03/04 06:36:38 [notice] 26#0: [lua] client.lua:161: new(): [dns_client] supported types: srv ipv4 
2025-03-04T00:36:40.124236219-06:00 2025/03/04 06:36:38 [notice] 26#0: [lua] init.lua:794: init(): [request-debug] token for request debugging: 60a4ff7d-cdb8-4f9e-8722-d38743a910a4
2025-03-04T00:36:40.269840926-06:00 2025/03/04 06:36:38 [notice] 26#0: using the "epoll" event method
2025-03-04T00:36:40.269865693-06:00 2025/03/04 06:36:38 [notice] 26#0: openresty/1.25.3.2
2025-03-04T00:36:40.269868859-06:00 2025/03/04 06:36:38 [notice] 26#0: OS: Linux 5.14.0-427.50.1.el9_4.x86_64
2025-03-04T00:36:40.269872185-06:00 2025/03/04 06:36:38 [notice] 26#0: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2025-03-04T00:36:40.269891932-06:00 2025/03/04 06:36:38 [notice] 26#0: start worker processes
2025-03-04T00:36:40.273296529-06:00 2025/03/04 06:36:38 [notice] 26#0: start worker process 39
2025-03-04T00:36:40.276214517-06:00 2025/03/04 06:36:38 [notice] 26#0: start worker process 40
2025-03-04T00:36:40.279107768-06:00 2025/03/04 06:36:38 [notice] 26#0: start worker process 41
2025-03-04T00:36:40.280811604-06:00 2025/03/04 06:36:38 [notice] 26#0: start worker process 42
2025-03-04T00:36:40.297028766-06:00 2025/03/04 06:36:40 [notice] 39#0: *1 [lua] broker.lua:215: init(): event broker is ready to accept connections on worker #0, context: init_worker_by_lua*
2025-03-04T00:36:40.361148511-06:00 2025/03/04 06:36:40 [notice] 39#0: *1 [lua] warmup.lua:113: single_dao(): Preloading 'services' into the core_cache..., context: init_worker_by_lua*
2025-03-04T00:36:40.375630437-06:00 2025/03/04 06:36:40 [notice] 41#0: *3 [lua] globalpatches.lua:75: sleep(): executing a blocking 'sleep' (0.001 seconds), context: init_worker_by_lua*
2025-03-04T00:36:40.376148034-06:00 2025/03/04 06:36:40 [notice] 40#0: *2 [lua] globalpatches.lua:75: sleep(): executing a blocking 'sleep' (0.001 seconds), context: init_worker_by_lua*
2025-03-04T00:36:40.376593635-06:00 2025/03/04 06:36:40 [notice] 41#0: *3 [lua] globalpatches.lua:75: sleep(): executing a blocking 'sleep' (0.002 seconds), context: init_worker_by_lua*
2025-03-04T00:36:40.377299598-06:00 2025/03/04 06:36:40 [notice] 40#0: *2 [lua] globalpatches.lua:75: sleep(): executing a blocking 'sleep' (0.002 seconds), context: init_worker_by_lua*
2025-03-04T00:36:40.378805580-06:00 2025/03/04 06:36:40 [notice] 41#0: *3 [lua] globalpatches.lua:75: sleep(): executing a blocking 'sleep' (0.004 seconds), context: init_worker_by_lua*
2025-03-04T00:36:40.379555716-06:00 2025/03/04 06:36:40 [notice] 40#0: *2 [lua] globalpatches.lua:75: sleep(): executing a blocking 'sleep' (0.004 seconds), context: init_worker_by_lua*
2025-03-04T00:36:40.383917560-06:00 2025/03/04 06:36:40 [notice] 41#0: *3 [lua] globalpatches.lua:75: sleep(): executing a blocking 'sleep' (0.008 seconds), context: init_worker_by_lua*
2025-03-04T00:36:40.383932949-06:00 2025/03/04 06:36:40 [notice] 40#0: *2 [lua] globalpatches.lua:75: sleep(): executing a blocking 'sleep' (0.008 seconds), context: init_worker_by_lua*
2025-03-04T00:36:40.391123262-06:00 2025/03/04 06:36:40 [notice] 41#0: *3 [lua] globalpatches.lua:75: sleep(): executing a blocking 'sleep' (0.016 seconds), context: init_worker_by_lua*
2025-03-04T00:36:40.391830116-06:00 2025/03/04 06:36:40 [notice] 40#0: *2 [lua] globalpatches.lua:75: sleep(): executing a blocking 'sleep' (0.016 seconds), context: init_worker_by_lua*
2025-03-04T00:36:40.407349180-06:00 2025/03/04 06:36:40 [notice] 41#0: *3 [lua] globalpatches.lua:75: sleep(): executing a blocking 'sleep' (0.032 seconds), context: init_worker_by_lua*
2025-03-04T00:36:40.408077044-06:00 2025/03/04 06:36:40 [notice] 40#0: *2 [lua] globalpatches.lua:75: sleep(): executing a blocking 'sleep' (0.032 seconds), context: init_worker_by_lua*
2025-03-04T00:36:40.605634649-06:00 2025/03/04 06:36:40 [notice] 39#0: *1 [lua] warmup.lua:162: single_dao(): finished preloading 'services' into the core_cache (in 245ms), context: init_worker_by_lua*
2025-03-04T00:36:40.605649497-06:00 2025/03/04 06:36:40 [notice] 39#0: *1 [lua] warmup.lua:113: single_dao(): Preloading 'consumers' into the cache..., context: init_worker_by_lua*
2025-03-04T00:36:40.715027742-06:00 2025/03/04 06:36:40 [notice] 39#0: *1 [lua] warmup.lua:162: single_dao(): finished preloading 'consumers' into the cache (in 0ms), context: init_worker_by_lua*
2025-03-04T00:36:40.715044845-06:00 2025/03/04 06:36:40 [notice] 39#0: *1 [lua] warmup.lua:113: single_dao(): Preloading 'acls' into the cache..., context: init_worker_by_lua*
2025-03-04T00:36:45.615918505-06:00 2025/03/04 06:36:45 [error] 42#0: *6 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:36:45.624461612-06:00 2025/03/04 06:36:45 [error] 40#0: *651 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:36:45.626305162-06:00 2025/03/04 06:36:45 [error] 41#0: *679 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:36:50.631674221-06:00 2025/03/04 06:36:50 [error] 42#0: *1936 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:36:50.650011916-06:00 2025/03/04 06:36:50 [error] 41#0: *1938 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:36:50.662620436-06:00 2025/03/04 06:36:50 [error] 40#0: *1940 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:36:55.670112252-06:00 2025/03/04 06:36:55 [error] 42#0: *1942 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:36:55.683588029-06:00 2025/03/04 06:36:55 [error] 40#0: *1944 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:36:55.689511410-06:00 2025/03/04 06:36:55 [error] 41#0: *1946 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:00.693422577-06:00 2025/03/04 06:37:00 [error] 42#0: *1948 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:00.717759015-06:00 2025/03/04 06:37:00 [error] 40#0: *1950 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:00.729000676-06:00 2025/03/04 06:37:00 [error] 41#0: *1952 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:05.711611345-06:00 2025/03/04 06:37:05 [error] 42#0: *1954 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:05.757056078-06:00 2025/03/04 06:37:05 [error] 40#0: *1956 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:05.766073179-06:00 2025/03/04 06:37:05 [error] 41#0: *1958 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:10.730712565-06:00 2025/03/04 06:37:10 [error] 42#0: *1960 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:10.791566825-06:00 2025/03/04 06:37:10 [error] 41#0: *1962 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:10.802934583-06:00 2025/03/04 06:37:10 [error] 40#0: *1964 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:15.761436284-06:00 2025/03/04 06:37:15 [error] 42#0: *1966 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:15.825371009-06:00 2025/03/04 06:37:15 [error] 40#0: *1968 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:15.834548031-06:00 2025/03/04 06:37:15 [error] 41#0: *1970 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:20.775059714-06:00 2025/03/04 06:37:20 [error] 42#0: *2025 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:20.838767170-06:00 2025/03/04 06:37:20 [error] 40#0: *2030 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:20.846771519-06:00 2025/03/04 06:37:20 [error] 41#0: *2032 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:25.799809406-06:00 2025/03/04 06:37:25 [error] 42#0: *2091 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:25.866307890-06:00 2025/03/04 06:37:25 [error] 40#0: *2093 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:25.871060402-06:00 2025/03/04 06:37:25 [error] 41#0: *2095 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:30.817564245-06:00 2025/03/04 06:37:30 [error] 42#0: *2120 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:30.882516140-06:00 2025/03/04 06:37:30 [error] 41#0: *2122 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:30.884853111-06:00 2025/03/04 06:37:30 [error] 40#0: *2124 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:35.860038926-06:00 2025/03/04 06:37:35 [error] 42#0: *2149 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:35.898458067-06:00 2025/03/04 06:37:35 [error] 40#0: *2151 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:35.930674479-06:00 2025/03/04 06:37:35 [error] 41#0: *2153 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:40.910393871-06:00 2025/03/04 06:37:40 [error] 42#0: *2161 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:40.947125236-06:00 2025/03/04 06:37:40 [error] 41#0: *2163 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:40.947849503-06:00 2025/03/04 06:37:40 [error] 40#0: *2165 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:45.928647812-06:00 2025/03/04 06:37:45 [error] 42#0: *2191 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:45.975420752-06:00 2025/03/04 06:37:45 [error] 41#0: *2193 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:45.985366646-06:00 2025/03/04 06:37:45 [error] 40#0: *2195 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:50.943635818-06:00 2025/03/04 06:37:50 [error] 42#0: *2214 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:51.004468848-06:00 2025/03/04 06:37:51 [error] 40#0: *2216 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:51.016171939-06:00 2025/03/04 06:37:51 [error] 41#0: *2218 [lua] worker.lua:291: communicate(): failed to connect: failed to receive response header: timeout, context: ngx.timer
2025-03-04T00:37:52.410209201-06:00 2025/03/04 06:36:40 [notice] 39#0: *1 [lua] warmup.lua:162: single_dao(): finished preloading 'acls' into the cache (in 0ms), context: init_worker_by_lua*
2025-03-04T00:37:52.410275847-06:00 2025/03/04 06:36:40 [notice] 39#0: *1 [lua] warmup.lua:113: single_dao(): Preloading 'certificates' into the core_cache..., context: init_worker_by_lua*
2025-03-04T00:37:52.466804161-06:00 2025/03/04 06:36:40 [notice] 39#0: *1 [lua] warmup.lua:162: single_dao(): finished preloading 'certificates' into the core_cache (in 0ms), context: init_worker_by_lua*
2025-03-04T00:37:52.466841071-06:00 2025/03/04 06:36:40 [notice] 39#0: *1 [lua] warmup.lua:113: single_dao(): Preloading 'jwt_secrets' into the cache..., context: init_worker_by_lua*
2025-03-04T00:37:52.556407248-06:00 2025/03/04 06:36:40 [notice] 39#0: *1 [lua] warmup.lua:162: single_dao(): finished preloading 'jwt_secrets' into the cache (in 0ms), context: init_worker_by_lua*
2025-03-04T00:37:52.556446222-06:00 2025/03/04 06:36:40 [notice] 39#0: *1 [lua] warmup.lua:113: single_dao(): Preloading 'oauth2_credentials' into the cache..., context: init_worker_by_lua*
2025-03-04T00:37:52.658270899-06:00 2025/03/04 06:36:40 [notice] 39#0: *1 [lua] warmup.lua:162: single_dao(): finished preloading 'oauth2_credentials' into the cache (in 0ms), context: init_worker_by_lua*
2025-03-04T00:37:53.020146846-06:00 2025/03/04 06:37:53 [notice] 39#0: *2424 [lua] warmup.lua:46: warming up DNS entries ..., context: ngx.timer
2025-03-04T00:37:53.025000898-06:00 2025/03/04 06:37:53 [error] 39#0: *2333 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025033590-06:00 2025/03/04 06:37:53 [error] 39#0: *2334 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025042346-06:00 2025/03/04 06:37:53 [error] 39#0: *2335 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025109634-06:00 2025/03/04 06:37:53 [error] 39#0: *2336 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025204573-06:00 2025/03/04 06:37:53 [error] 39#0: *2337 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025216365-06:00 2025/03/04 06:37:53 [error] 39#0: *2338 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025305573-06:00 2025/03/04 06:37:53 [error] 39#0: *2339 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025434727-06:00 2025/03/04 06:37:53 [error] 39#0: *2340 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025442883-06:00 2025/03/04 06:37:53 [error] 39#0: *2341 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025581494-06:00 2025/03/04 06:37:53 [error] 39#0: *2342 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025604367-06:00 2025/03/04 06:37:53 [error] 39#0: *2343 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025653280-06:00 2025/03/04 06:37:53 [error] 39#0: *2344 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025662297-06:00 2025/03/04 06:37:53 [error] 39#0: *2345 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025668999-06:00 2025/03/04 06:37:53 [error] 39#0: *2346 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025680912-06:00 2025/03/04 06:37:53 [error] 39#0: *2347 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025725676-06:00 2025/03/04 06:37:53 [error] 39#0: *2348 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025745374-06:00 2025/03/04 06:37:53 [error] 39#0: *2349 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025753689-06:00 2025/03/04 06:37:53 [error] 39#0: *2350 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025756845-06:00 2025/03/04 06:37:53 [error] 39#0: *2351 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025759811-06:00 2025/03/04 06:37:53 [error] 39#0: *2352 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025765291-06:00 2025/03/04 06:37:53 [error] 39#0: *2353 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025786401-06:00 2025/03/04 06:37:53 [error] 39#0: *2354 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025795228-06:00 2025/03/04 06:37:53 [error] 39#0: *2355 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025822740-06:00 2025/03/04 06:37:53 [error] 39#0: *2356 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025893954-06:00 2025/03/04 06:37:53 [error] 39#0: *2357 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025900045-06:00 2025/03/04 06:37:53 [error] 39#0: *2358 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025914944-06:00 2025/03/04 06:37:53 [error] 39#0: *2359 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025930002-06:00 2025/03/04 06:37:53 [error] 39#0: *2360 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025953697-06:00 2025/03/04 06:37:53 [error] 39#0: *2361 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025982872-06:00 2025/03/04 06:37:53 [error] 39#0: *2362 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025989785-06:00 2025/03/04 06:37:53 [error] 39#0: *2363 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025995896-06:00 2025/03/04 06:37:53 [error] 39#0: *2364 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.025998962-06:00 2025/03/04 06:37:53 [error] 39#0: *2365 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.026079945-06:00 2025/03/04 06:37:53 [error] 39#0: *2366 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.026084774-06:00 2025/03/04 06:37:53 [error] 39#0: *2367 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.026106375-06:00 2025/03/04 06:37:53 [error] 39#0: *2368 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.026117646-06:00 2025/03/04 06:37:53 [error] 39#0: *2369 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.026126022-06:00 2025/03/04 06:37:53 [error] 39#0: *2370 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.026217645-06:00 2025/03/04 06:37:53 [error] 39#0: *2371 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.026224628-06:00 2025/03/04 06:37:53 [error] 39#0: *2372 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.026227554-06:00 2025/03/04 06:37:53 [error] 39#0: *2373 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.026230419-06:00 2025/03/04 06:37:53 [error] 39#0: *2374 [lua] broker.lua:235: run(): failed to init socket: failed to flush response header: nginx output filter error, client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.026357910-06:00 2025/03/04 06:37:53 [notice] 40#0: *2324 [lua] worker.lua:304: communicate(): worker #1 is ready to accept events from unix:/usr/local/kong/sockets/we, context: ngx.timer
2025-03-04T00:37:53.026509005-06:00 2025/03/04 06:37:53 [notice] 41#0: *2326 [lua] worker.lua:304: communicate(): worker #2 is ready to accept events from unix:/usr/local/kong/sockets/we, context: ngx.timer
2025-03-04T00:37:53.026590809-06:00 2025/03/04 06:37:53 [notice] 42#0: *2322 [lua] worker.lua:304: communicate(): worker #3 is ready to accept events from unix:/usr/local/kong/sockets/we, context: ngx.timer
2025-03-04T00:37:53.071008644-06:00 2025/03/04 06:37:53 [notice] 39#0: *2375 [lua] broker.lua:263: run(): worker #3 connected to events broker (worker pid: 42), client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.071135113-06:00 2025/03/04 06:37:53 [notice] 39#0: *2376 [lua] broker.lua:263: run(): worker #1 connected to events broker (worker pid: 40), client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.071399762-06:00 2025/03/04 06:37:53 [notice] 39#0: *2377 [lua] broker.lua:263: run(): worker #2 connected to events broker (worker pid: 41), client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.075160120-06:00 2025/03/04 06:37:53 [notice] 39#0: *2421 [lua] worker.lua:304: communicate(): worker #0 is ready to accept events from unix:/usr/local/kong/sockets/we, context: ngx.timer
2025-03-04T00:37:53.085835611-06:00 2025/03/04 06:37:53 [notice] 39#0: *3063 [lua] broker.lua:263: run(): worker #0 connected to events broker (worker pid: 39), client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:38:01.632549402-06:00 2025/03/04 06:38:01 [notice] 39#0: *2424 [lua] warmup.lua:80: finished warming up DNS entries' into the cache (in 8612ms), context: ngx.timer

After these logs Kong is healthy and then starts regularly handling traffic. Still no idea why all the [error] logs in the startup phase of a larger Kong instance though?

Expected Behavior

No response

Steps To Reproduce

No response

Anything else?

No response

[jeremyjpj0916]

Straight up [error] logs throwing from 06:36:45 to 06:37:53 in those logs, little past a full minute.

Related issue from prior: #13250

[jeremyjpj0916]

Is this like the sweet sauce that lets us know the errors are resolved at this point?

2025-03-04T00:37:53.026357910-06:00 2025/03/04 06:37:53 [notice] 40#0: *2324 [lua] worker.lua:304: communicate(): worker #1 is ready to accept events from unix:/usr/local/kong/sockets/we, context: ngx.timer
2025-03-04T00:37:53.026509005-06:00 2025/03/04 06:37:53 [notice] 41#0: *2326 [lua] worker.lua:304: communicate(): worker #2 is ready to accept events from unix:/usr/local/kong/sockets/we, context: ngx.timer
2025-03-04T00:37:53.026590809-06:00 2025/03/04 06:37:53 [notice] 42#0: *2322 [lua] worker.lua:304: communicate(): worker #3 is ready to accept events from unix:/usr/local/kong/sockets/we, context: ngx.timer
2025-03-04T00:37:53.071008644-06:00 2025/03/04 06:37:53 [notice] 39#0: *2375 [lua] broker.lua:263: run(): worker #3 connected to events broker (worker pid: 42), client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.071135113-06:00 2025/03/04 06:37:53 [notice] 39#0: *2376 [lua] broker.lua:263: run(): worker #1 connected to events broker (worker pid: 40), client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.071399762-06:00 2025/03/04 06:37:53 [notice] 39#0: *2377 [lua] broker.lua:263: run(): worker #2 connected to events broker (worker pid: 41), client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"
2025-03-04T00:37:53.075160120-06:00 2025/03/04 06:37:53 [notice] 39#0: *2421 [lua] worker.lua:304: communicate(): worker #0 is ready to accept events from unix:/usr/local/kong/sockets/we, context: ngx.timer
2025-03-04T00:37:53.085835611-06:00 2025/03/04 06:37:53 [notice] 39#0: *3063 [lua] broker.lua:263: run(): worker #0 connected to events broker (worker pid: 39), client: unix:, server: kong_worker_events, request: "GET / HTTP/1.1", host: "localhost"

Where finally we see "ready to accept events" ? I guess the question is why the tons of errors prior and the full minute of it I suppose. Slow startup procedure stuff for larger Kong runtimes leading to early runtime initial error logs?