kong 500 error：received: 500; {"message":"An unexpected error occurred"}

[zhangxinars]

Is there an existing issue for this?

• I have searched the existing issues

Kong version ($ kong version)

Kong 3.0.0

Current Behavior

No response

Expected Behavior

No response

Steps To Reproduce

No response

Anything else?

These error logs exist. I'm not sure if they are relevant.

2024/03/27 17:07:29 [error] 1124#0: *62 [lua] handler.lua:415: reconfigure failed: ...l/share/lua/5.1/kong/runloop/balancer/healthcheckers.lua:419: bad argument #1 to 'pairs' (table expected, got nil), context: ngx.timer

2024/03/29 07:24:29 [error] 1126#0: *58047224 upstream prematurely closed connection while reading response header from upstream

[StarlightIbuki]

Is there any other error log with the error or the critical level?
Also, we always recommend updating to the latest version.

[zhangxinars]

I didn't see any other error logs.

[StarlightIbuki]

How often does this occur? It sounds like a previously-fixed bug. Could you also try with the latest version?

[chronolaw]

Kong gateway 3.0 is EOL, could you upgrade it to the latest or LTS version and try again?

[git-hulk]

@StarlightIbuki @chronolaw This issue seems to have happened while get_all_upstreams returns nil + error, but error was swallowed : https://github.com/Kong/kong/blob/3.0.0/kong/runloop/balancer/healthcheckers.lua#L419.

It should have the same issue in latest version, should we check the error and don't iterate the get_all_upstreams if it returns nil + error? like below:

function healthcheckers_M.stop_healthcheckers(delay)
  local upstreams, err =  upstreams.get_all_upstreams()
  if err ~= nil {
    return err
  }
  for _, id in pairs() do
  ...
end

And log the error message where calls the stop_healthcheckers.

[chronolaw]

@oowl , could you take a look? thanks.

[oowl]

Hi @git-hulk, We have fixed this bug in the master branch, Please be patient to wait for the next major version release. It will be included.

[git-hulk]

@oowl That's cool, thank you!

[dubuqingfeng]

failed the initial dns/balancer resolve for 'UPSTREAM_HOST' with: failed to get from node cache: could not acquire callback lock: timeout

Hello, we have checked the relevant logs and found that they should be related to this.
And it can be confirmed that 500 is not related to the fix above.

[dubuqingfeng]

Can this be reopened?

[oowl]

@dubuqingfeng Hi, You can open a new issue, and I will be happy to help you investigate.