bug: failed to run balancer_by_lua*: /usr/local/share/lua/5.1/apisix/balancer.lua:99: attempt to index local 'checker' (a nil value)

[kuberxy]

Issue description

2020/12/29 18:38:10 [error] 9718#9718: 69019801 failed to run balancer_by_lua: /usr/local/share/lua/5.1/apisix/balancer.lua:99: attempt to index local 'checker' (a nil value)
stack traceback:
/usr/local/share/lua/5.1/apisix/balancer.lua:99: in function 'create_obj_fun'
/usr/local/share/lua/5.1/apisix/core/lrucache.lua:92: in function 'lrucache_checker'
/usr/local/share/lua/5.1/apisix/balancer.lua:135: in function 'fetch_healthchecker'
/usr/local/share/lua/5.1/apisix/balancer.lua:208: in function 'pick_server'
/usr/local/share/lua/5.1/apisix/balancer.lua:272: in function 'load_balancer'
/usr/local/share/lua/5.1/apisix/init.lua:748: in function 'http_balancer_phase'
balancer_by_lua:2: in main chunk while connecting to upstream, client: 192.168.2.11, server: , request: "GET /question/bag/getQuestions?id=bGl1c2hlbmd5aV2QuPZIGWaI&offset=0&limit=20&pushId=4caf4e1b-a65a-48d1-ab63-86e8a6dd26f6 HTTP/1.0", host: "hyb.com"

Environment

• apisix version (cmd: apisix version): 2.1
• OS: (cmd: uname -a) 4.15.0-111-generic Ubuntu 18.04
• OpenResty / Nginx version: (cmd: nginx -V or openresty -V) openresty/1.15.8.3

[tokers]

@kuberxy How this problem occurred in your environment, is it reproducible? If so, could you please paste a minimal case to help us reproduce it.

[kuberxy]

这是我司的生产环境，我在一台备用节点上部署了apisix，通过Goreplay将生产环境的GET请求复制到apisix上，昨天上午一直都是正常运行的，到了晚上出现了大量的500错误。从上午到下午的这段时间我并没有调整任何配置。

[membphis]

@kuberxy I created a related issue: #3169

we can add more Lua code to capture the error case.

If you can provide more information(eg: fully error log), or the mini step to reproduce this issue, it should be helpful.

[kuberxy]

@kuberxy I created a related issue: #3169

we can add more Lua code to capture the error case.

If you can provide more information(eg: fully error log), or the mini step to reproduce this issue, it should be helpful.

thanks. I try it

[kuberxy]

I capture this error

2021/01/05 14:21:04 [error] 10698#10698: *196787178 [lua] balancer.lua:96: create_obj_fun(): fail to create healthcheck instance: failed to create 'healthy' timer: too many p
ending timers while connecting to upstream, client: 192.168.1.11, server: , request: "GET /lesson?offset=0&limit=20&pushId=99ce0c71-248f-44
e8-9da2-f82660ecbdf9 HTTP/1.0", host: "hyb.com"

Is this a resource limitations or a bug or other ?

[membphis]

https://github.com/openresty/lua-nginx-module#lua_max_pending_timers

you can add this directive to your nginx.conf ^_^