Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Too much retry for unimplemented error #1938

Closed
1 task done
binbinlv opened this issue Feb 29, 2024 · 3 comments
Closed
1 task done

[Bug]: Too much retry for unimplemented error #1938

binbinlv opened this issue Feb 29, 2024 · 3 comments
Assignees
Labels
kind/bug Something isn't working
Milestone

Comments

@binbinlv
Copy link

Is there an existing issue for this?

  • I have searched the existing issues

Describe the bug

Too much retry for unimplemented error

[2024-02-28 19:29:02 - WARNING - pymilvus.decorators]: [describe_alias] retry:4, cost: 0.27s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:29:02 - WARNING - pymilvus.decorators]: [describe_alias] retry:5, cost: 0.81s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:29:03 - WARNING - pymilvus.decorators]: [describe_alias] retry:6, cost: 2.43s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:29:06 - WARNING - pymilvus.decorators]: [describe_alias] retry:7, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:29:09 - WARNING - pymilvus.decorators]: [describe_alias] retry:8, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:29:12 - WARNING - pymilvus.decorators]: [describe_alias] retry:9, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:29:15 - WARNING - pymilvus.decorators]: [describe_alias] retry:10, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:29:18 - WARNING - pymilvus.decorators]: [describe_alias] retry:11, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:29:21 - WARNING - pymilvus.decorators]: [describe_alias] retry:12, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:29:24 - WARNING - pymilvus.decorators]: [describe_alias] retry:13, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:29:27 - WARNING - pymilvus.decorators]: [describe_alias] retry:14, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:29:30 - WARNING - pymilvus.decorators]: [describe_alias] retry:15, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:29:33 - WARNING - pymilvus.decorators]: [describe_alias] retry:16, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:29:36 - WARNING - pymilvus.decorators]: [describe_alias] retry:17, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:29:40 - WARNING - pymilvus.decorators]: [describe_alias] retry:18, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:29:43 - WARNING - pymilvus.decorators]: [describe_alias] retry:19, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:29:46 - WARNING - pymilvus.decorators]: [describe_alias] retry:20, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:29:49 - WARNING - pymilvus.decorators]: [describe_alias] retry:21, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:29:52 - WARNING - pymilvus.decorators]: [describe_alias] retry:22, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:29:55 - WARNING - pymilvus.decorators]: [describe_alias] retry:23, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:29:58 - WARNING - pymilvus.decorators]: [describe_alias] retry:24, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:30:01 - WARNING - pymilvus.decorators]: [describe_alias] retry:25, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:30:04 - WARNING - pymilvus.decorators]: [describe_alias] retry:26, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:30:07 - WARNING - pymilvus.decorators]: [describe_alias] retry:27, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:30:10 - WARNING - pymilvus.decorators]: [describe_alias] retry:28, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:30:13 - WARNING - pymilvus.decorators]: [describe_alias] retry:29, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:30:16 - WARNING - pymilvus.decorators]: [describe_alias] retry:30, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:30:19 - WARNING - pymilvus.decorators]: [describe_alias] retry:31, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:30:22 - WARNING - pymilvus.decorators]: [describe_alias] retry:32, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:30:26 - WARNING - pymilvus.decorators]: [describe_alias] retry:33, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:30:29 - WARNING - pymilvus.decorators]: [describe_alias] retry:34, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:30:32 - WARNING - pymilvus.decorators]: [describe_alias] retry:35, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:30:35 - WARNING - pymilvus.decorators]: [describe_alias] retry:36, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:30:38 - WARNING - pymilvus.decorators]: [describe_alias] retry:37, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:30:41 - WARNING - pymilvus.decorators]: [describe_alias] retry:38, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:30:44 - WARNING - pymilvus.decorators]: [describe_alias] retry:39, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:30:47 - WARNING - pymilvus.decorators]: [describe_alias] retry:40, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:30:50 - WARNING - pymilvus.decorators]: [describe_alias] retry:41, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:30:53 - WARNING - pymilvus.decorators]: [describe_alias] retry:42, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:30:56 - WARNING - pymilvus.decorators]: [describe_alias] retry:43, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:30:59 - WARNING - pymilvus.decorators]: [describe_alias] retry:44, cost: 3.00s, reason: <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService> (decorators.py:94)
[2024-02-28 19:31:02 - ERROR - pymilvus.decorators]: RPC error: [describe_alias], <MilvusException: (code=<bound method _MultiThreadedRendezvous.code of <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNIMPLEMENTED
	details = "unknown method DescribeAlias for service milvus.proto.milvus.MilvusService"
	debug_error_string = "UNKNOWN:Error received from peer  {created_time:"2024-02-28T19:31:02.933745+08:00", grpc_status:12, grpc_message:"unknown method DescribeAlias for service milvus.proto.milvus.MilvusService"}"

Expected Behavior

Could we not retry for unimplemented error

Steps/Code To Reproduce behavior

call "describe_alias" interface to not supported milvus version (like 2.2.16)

Environment details

- Hardware/Softward conditions (OS, CPU, GPU, Memory):
- Method of installation (Docker, or from source):
- Milvus version (v0.3.1, or v0.4.0): v2.2.16
- Milvus configuration (Settings you made in `server_config.yaml`):

Anything else?

No response

@binbinlv
Copy link
Author

/assign @XuanYang-cn

XuanYang-cn added a commit to XuanYang-cn/pymilvus that referenced this issue Mar 1, 2024
See also: milvus-io#1938

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
XuanYang-cn added a commit to XuanYang-cn/pymilvus that referenced this issue Mar 1, 2024
See also: milvus-io#1938

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
XuanYang-cn added a commit to XuanYang-cn/pymilvus that referenced this issue Mar 1, 2024
See also: milvus-io#1938

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
sre-ci-robot pushed a commit that referenced this issue Mar 5, 2024
See also: #1938

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
sre-ci-robot pushed a commit that referenced this issue Mar 5, 2024
See also: #1938

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
@XuanYang-cn
Copy link
Contributor

/assign @binbinlv
/unassign
Please help review

@sre-ci-robot sre-ci-robot assigned binbinlv and unassigned XuanYang-cn Mar 5, 2024
@XuanYang-cn XuanYang-cn added this to the 2.4.0 milestone Mar 5, 2024
@XuanYang-cn XuanYang-cn added the kind/bug Something isn't working label Mar 5, 2024
@binbinlv
Copy link
Author

binbinlv commented Mar 7, 2024

Verified and fixed.

milvus: v2.2.16
pymilvus: 2.4.0rc47

result:

>>> client.describe_alias(alias)
grpc RpcError: [describe_alias], <_MultiThreadedRendezvous: StatusCode.UNIMPLEMENTED, unknown method DescribeAlias for service milvus.proto.milvus.MilvusService>, <Time:{'RPC start': '2024-03-07 18:59:51.464918', 'gRPC error': '2024-03-07 18:59:51.545166'}>
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/binbin/milvus_latest/lib/python3.8/site-packages/pymilvus/milvus_client/milvus_client.py", line 962, in describe_alias
    return conn.describe_alias(alias, timeout=timeout, **kwargs)
  File "/Users/binbin/milvus_latest/lib/python3.8/site-packages/pymilvus/decorators.py", line 154, in handler
    raise e from e
  File "/Users/binbin/milvus_latest/lib/python3.8/site-packages/pymilvus/decorators.py", line 136, in handler
    return func(*args, **kwargs)
  File "/Users/binbin/milvus_latest/lib/python3.8/site-packages/pymilvus/decorators.py", line 175, in handler
    return func(self, *args, **kwargs)
  File "/Users/binbin/milvus_latest/lib/python3.8/site-packages/pymilvus/decorators.py", line 90, in handler
    raise e from e
  File "/Users/binbin/milvus_latest/lib/python3.8/site-packages/pymilvus/decorators.py", line 86, in handler
    return func(*args, **kwargs)
  File "/Users/binbin/milvus_latest/lib/python3.8/site-packages/pymilvus/client/grpc_handler.py", line 888, in describe_alias
    response = rf.result()
  File "/Users/binbin/milvus_latest/lib/python3.8/site-packages/grpc/_channel.py", line 881, in result
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNIMPLEMENTED
	details = "unknown method DescribeAlias for service milvus.proto.milvus.MilvusService"
	debug_error_string = "UNKNOWN:Error received from peer  {created_time:"2024-03-07T18:59:51.544769+08:00", grpc_status:12, grpc_message:"unknown method DescribeAlias for service milvus.proto.milvus.MilvusService"}"
>

@binbinlv binbinlv closed this as completed Mar 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants