Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Indexnode fails ( can't load collections) #34608

Open
1 task done
sfisli opened this issue Jul 11, 2024 · 3 comments
Open
1 task done

[Bug]: Indexnode fails ( can't load collections) #34608

sfisli opened this issue Jul 11, 2024 · 3 comments
Assignees
Labels
help wanted Extra attention is needed

Comments

@sfisli
Copy link

sfisli commented Jul 11, 2024

Is there an existing issue for this?

  • I have searched the existing issues

Environment

- Milvus version: 2.2.13
- Deployment mode(standalone or cluster): cluster
- MQ type(rocksmq, pulsar or kafka):  kafka
- OS(Ubuntu or CentOS): kubernetes

Current Behavior

  • i cant load collections in milvus-attu , it stucks at 0%
  • error in the indexnode pod:
`2024-07-11 10:08:36,500 | INFO | default | [SEGCORE][ProcessFormattedStatement][milvus] [AWS LOG] [ERROR] 2024-07-11 10:08:36.500 AWSClient [139881923671808] HTTP response code: 400
Resolved remote host IP address: 52.95.156.81
Request ID: 
Exception name: 
Error message: No response body.
6 response headers:
connection : close
content-type : application/xml
date : Thu, 11 Jul 2024 10:08:35 GMT
server : AmazonS3
x-amz-id-2 : ETNGC8+s6QxVz2GgqKyfFNYxmHcNG1BaxlZrZP8k6CC3/bPQ/dqsdqsdsqdzdz=
x-amz-request-id : ECK2V0ZTSSSEAA66EGTN3R`
pure virtual method called
terminate called without an active exception

Expected Behavior

when i do load collection it loads without issues.

Steps To Reproduce

No response

Milvus Log

No response

Anything else?

No response

@sfisli sfisli added kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Jul 11, 2024
@yanliang567
Copy link
Contributor

I guess you did not access to aws s3 service successfully, please try to check the config is correct. https://milvus.io/docs/aws.md
also I suggest you retry on latest milvus 2.3.18 or 2.4.5, which fixed a lot of issues. @sfisli
/assign @sfisli
/unassign

@sre-ci-robot sre-ci-robot assigned sfisli and unassigned yanliang567 Jul 12, 2024
@yanliang567 yanliang567 added help wanted Extra attention is needed and removed kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Jul 12, 2024
@sfisli
Copy link
Author

sfisli commented Jul 12, 2024

@yanliang567
Milvus policy:

`        {
            "Action": [
                "s3:*"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::milvus-match-video-objects-bucket-prod",
                "arn:aws:s3:::milvus-match-video-objects-bucket-prod/*"
            ]
        }`

externalS3:

externalS3:
  enabled: true
  host: "s3.eu-west-3.amazonaws.com"
  port: "80"
  accessKey: "milvus-accesskey"
  secretKey: "milvus-secretkey"
  cloudProvider: "aws"
  useSSL: false
  bucketName: "milvus-match-video-objects-bucket-prod"
  rootPath: ""
  useIAM: false
  iamEndpoint: ""

here is my config & it used to work ! ``

@yanliang567
Copy link
Contributor

/assign @LoveEachDay
could you please help to take a look
/unassign

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

3 participants