rpc error: code = Unavailable desc = Starting

[wolf666666]

If you are reporting any crash or any potential security issue, do not
open an issue in this repo. Please report the issue via emailing
envoy-security@googlegroups.com where the issue will be triaged appropriately.

Title: rpc error: code = Unavailable desc = Starting

Description:

One of the cortex ingester instance was killed by the OOM killer because of the memory limit, then the ingester start to startup. but the ingester is still receive request at the same time, so it generate a lot error message:
"Apr 27 17:16:43 cortex-1.16.0[3069602]: ts=2024-04-27T15:16:43.536890158Z caller=grpc_logging.go:64 level=warn method=/cortex.Ingester/QueryStream duration=55.725µs err="rpc error: code = Unavailable desc = Starting" msg=gRPC"

It seems that the distributor still route request to this ingester instance even it have not startup successfully.
and the client that send request to cortex distributor get error message at the same time:
"Request failed. Status: 500 Body: maxFailure (quorum) on a given error family, rpc error: code = Code(500) desc = addr=xxxxxx:port state=ACTIVE zone=www, rpc error: code = Unavailable desc = Starting
"
expect:
the cortex ingester receive request only when it startup totally.

[adisuissa]

Is this an issue that should be reported to https://github.com/cortexproject/cortex ?
If it is Envoy related, can you provide a more detailed explanation related to Envoy?

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted" or "no stalebot". Thank you for your contributions.