Milvus fail to get shard leaders from QueryCoord: no replica available error after standalone restart

[Yashwanth1712094]

MIlvus version-2.1.3
Deployment mode(standalone or cluster): standalone

• MQ type(rocksmq, pulsar or kafka): default
• SDK version(e.g. pymilvus v2.0.0rc2): pymilvus 2.1.3

After the standalone pod restarted we are facing the following error
attempt #1:fail to get shard leaders from QueryCoord: no replica available
attempt #2:fail to get shard leaders from QueryCoord: no replica available
attempt #3:fail to get shard leaders from QueryCoord: no replica available
attempt #4:fail to get shard leaders from QueryCoord: no replica available
attempt #5:fail to get shard leaders from QueryCoord: no replica available
attempt #6:fail to get shard leaders from QueryCoord: no replica available
attempt #7:fail to get shard leaders from QueryCoord: no replica available
attempt #8:context deadline exceeded

[yhmo]

2.1.3 is really an ancient version(this version is released 3 years ago). Why not use the latest version. The v2.3.13 is much stable than v2.1
https://milvus.io/docs/v2.3.x/install_standalone-docker-compose.md

[Yashwanth1712094]

1.But there is already data in milvus. How to retrive that data.
2.Can we attach any sql databases to milvus for data recovery in case of any milvus issue.please suggest any backup setup for milvus data.

[yhmo]

Upgrade milvus from v2.1.x to v2.3.x: https://milvus.io/docs/v2.3.x/upgrade_milvus_standalone-docker.md

Note: you need to migrate the metadata before any upgrade from v2.1.x to v2.3.15.

[Yashwanth1712094]

Thank you , i'll try that one.
can we attach sql databases or gcp cloud storage to milvus instead of default generated volumes??

[yhmo]

A Milvus cluster dependencies:

• Etcd. Stores milvus runtime metadata, including collections schema, addresses of nodes, paths of data files in S3, etc.
• Pulsar/Kafka. As a message to store WAL(write-ahead-log) for insert/delete requests, and time-tick message for data consistency.
• S3/MinIO. As data storage to store data files, index files of each segment.

Etcd and pulsar/kafka are independent services, have their own storage configurations.
Yes you can use google cloud storage for milvus: https://milvus.io/docs/v2.3.x/object_storage_operator.md. But it is only for storing data files. Milvus requires the metadata in etcd to be strongly consistent with the data files in s3/minio. If you only backup the data in s3, it cannot work.

The milvus-backup tool is the recommended tool to backup/restore data of milvus.