Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

On AWS, Cassandra doesn't come up after restoration - A node with address already exists #1324

Closed
c3-clement opened this issue May 23, 2024 · 1 comment
Labels
bug Something isn't working done Issues in the state 'done'

Comments

@c3-clement
Copy link
Contributor

What happened?

On AWS (EKS and S3 Bucket), the Cassandra Statefulset won't come up sometimes after restoration.
On a K8ssandra cluster with 1 dc of 3 nodes, 2 are not reaching readiness probe.
image

On failed pods, the following error message is showing up in logs:

java.lang.RuntimeException: A node with address /172.0.238.69:7000 already exists, cancelling join. Use cassandra.replace_address if you want to replace this node.
	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:749)
	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:1024)
	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:874)
	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:819)
	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:418)
	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:759)
	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:893)

Did you expect to see something different?

How to reproduce it (as minimally and precisely as possible):

Environment

  • K8ssandra Operator version:

    Insert image tag or Git SHA here

  • Kubernetes version information:

    kubectl version

  • Kubernetes cluster kind:

    insert how you created your cluster: kops, bootkube, etc.

  • Manifests:

insert manifests relevant to the issue
  • K8ssandra Operator Logs:
insert K8ssandra Operator logs relevant to the issue here

Anything else we need to know?:

@c3-clement c3-clement added the bug Something isn't working label May 23, 2024
@c3-clement
Copy link
Contributor Author

Duplicate of #1325

@c3-clement c3-clement closed this as not planned Won't fix, can't repro, duplicate, stale Jun 4, 2024
@adejanovski adejanovski added the done Issues in the state 'done' label Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working done Issues in the state 'done'
Projects
Status: Done
Development

No branches or pull requests

2 participants