Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slow shutdown with multi-controller clusters #174

Open
tombentley opened this issue Sep 10, 2023 · 5 comments
Open

Slow shutdown with multi-controller clusters #174

tombentley opened this issue Sep 10, 2023 · 5 comments
Labels
bug Something isn't working

Comments

@tombentley
Copy link
Contributor

Describe the bug

When using multiple controllers a KafkaCluster takes 5 minutes to shutdown, every time.

To Reproduce

@ExtendWith(KafkaClusterExtension.class)
class ExampleTest {
    @Test
    public void test(
            @BrokerCluster(numBrokers = 3)
            @KRaftCluster(numControllers = 3)
            KafkaCluster cluster) {
        System.out.println("###### Waiting 10s");
        Thread.sleep(10_000);
        System.out.println("###### End of test");
    }
}

Expected behavior

The cluster would shutdown more quickly.

Logs
logs.txt

@tombentley tombentley added the bug Something isn't working label Sep 10, 2023
@tombentley
Copy link
Contributor Author

@SamBarker
Copy link
Member

Should we default to using separate brokers and controllers? Rather than combined mode?

I'd say the extra memory consumption is worth it for faster tests but I don't know that's a reasonable position for the library to take without an opt out?

@tombentley
Copy link
Contributor Author

I think that might be reasonable. We could either add

  • a combinedMode=true attribute on `@KraftCluster
  • or a separate @CombinedMode annotation to be used alongside @BrokerCluster and @KraftCluster

The former feels like it should be sufficient.

@SamBarker
Copy link
Member

Yeah I think a combinedMode attribute is enough.

I have a loose sense that we might want to consider topology API where users can control things like ID's and racks etc but I don't have any thoughts on what form that would take or if it would really be useful. I sort of see it as being helpful for re-creating topologies in tests but not really clear if there is really value in that.

I'm happy to pick this up but probably wont be for a couple days so if you want a faster fix you would need to pick it up.

@tombentley
Copy link
Contributor Author

@SamBarker I've started looking at this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants