help request: Apisix sometimes return 503 sometimes return 200 even if request valid and backend up.

[ynsdl]

Description

I am new at apisix.

ı have apisix between nginx and jboss.

if ı make small load test with siege, sometimes Apisix return 200 sometimes 503. My test is 2 minutes with 5 concurrent.

if ı remove apisix and repeat my test again all my request's response http 200.

[@~]$ siege -t2 -c4 --content-type "application/json" -H 'api-key: xxxxxzu' 'https://.....com/rest/3.0/getProfileAttributes?extId=erdem'
** SIEGE 4.1.1
** Preparing 4 concurrent users for battle.
The server is now under siege...
HTTP/1.1 200 0.27 secs: 218 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.34 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.29 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 200 1.26 secs: 218 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.28 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.23 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 200 2.27 secs: 218 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.25 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.29 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 200 3.26 secs: 218 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.32 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.27 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 200 3.01 secs: 218 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.26 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.28 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 200 3.29 secs: 218 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.24 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.27 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.32 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 200 3.16 secs: 218 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.26 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.27 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 200 3.13 secs: 218 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.27 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.24 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 200 2.99 secs: 218 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.27 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.32 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 200 3.00 secs: 218 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.29 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.25 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 200 3.00 secs: 218 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.25 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.24 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 200 2.86 secs: 218 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.24 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem
HTTP/1.1 503 0.25 secs: 38 bytes ==> GET /rest/3.0/getProfileAttributes?extId=erdem

Environment

• APISIX version (run apisix version): 2.15.0
• Operating system (run uname -a): Linux devops02 3.10.0-693.el7.x86_64 change: added doc of how to load plugin. #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
• OpenResty / Nginx version (run openresty -V or nginx -V):
• etcd version, if relevant (run curl http://127.0.0.1:9090/v1/server_info): 3.4.0
• APISIX Dashboard version, if relevant:
• Plugin runner version, for issues related to plugin runners:
• LuaRocks version, for installation issues (run luarocks --version): /usr/local/bin/luarocks 3.8.0

[tokers]

Try to see error.log

[ynsdl]

I defined a limit-req. I was asumming delete this define but it was active. I deleted all defines it is correct now.
Thanks.

error.log;

2022/09/06 06:07:04 [warn] 45#45: *40032252 [lua] plugin.lua:901: run_plugin(): limit-req exits with http status code 503, client: 10.34.169.14, server: _, request: "GET /rest/3.0/getProfileAttributes?extId=erdem HTTP/1.1", host: "xxx.com"