help request: why apisix dont update healthcheck upstream ip when upstream ip is no use

[shan307]

Description

i sacle my server from 2 to 4 instance，apisix debug logs:

but when i scale my server to two instance，the APISIX healthcheck still detects the IP that has been scale down.

and i noticed that there are many unhealthy IPs in the healthcheck still remaining in the detection list. (This route only has two upstream IPs for me.)

for my route is：

"checks": {
      "active": {
        "concurrency": 10,
        "healthy": {
          "http_statuses": [
            200,
            302
          ],
          "interval": 5,
          "successes": 2
        },
        "http_path": "/ping",
        "timeout": 3,
        "type": "http",
        "unhealthy": {
          "http_failures": 3,
          "http_statuses": [
            404,
            500,
            501,
            502,
            503,
            504,
            505
          ],
          "interval": 5,
          "tcp_failures": 2,
          "timeouts": 3
        }
      }
    },
    "scheme": "http",
    "discovery_type": "kubernetes",
    "pass_host": "pass",
    "service_name": "mgb-namespace/my-service:http",

Environment

• APISIX version (run apisix version):
• Operating system (run uname -a):
• OpenResty / Nginx version (run openresty -V or nginx -V):
• etcd version, if relevant (run curl http://127.0.0.1:9090/v1/server_info):
• APISIX Dashboard version, if relevant:
• Plugin runner version, for issues related to plugin runners:
• LuaRocks version, for installation issues (run luarocks --version):

[Baoyuantop]

Hi @shan307, please provide your environment information.