[AWS] Still Lack of server resource

[ghkdqhrbals]

Desciprtion

Finally I make CI/CD pipelines. But, EC2's cpu resource gets heavy loads. We need big improvements in our resource management.

[ghkdqhrbals]

Since I run the servers in local, It was a pretty predictable performance in AWS.

• Local ( environment : MBP CPU 2 GHz 4 Cores Intel Core i5) docker stats

[ghkdqhrbals]

Assumptions

1. Repeated topic connections that fail to resolve

When Kafka Broker sets the offset management topic ( __consumer_offset ), it sets the replica default to 3. Due to lack of server resources, I only set up kafka broker. So, the Kafka broker repeatedly informed me that there were not enough brokers, and this seemed to be a resource-consuming process.

2. Too much Tomcat max-connections

I set tomcat in chatting-server with 10000 max-connections and 1000 threads.max. 1000 threads will take memory usage of 1000*2MB which is almost 2GB. My EC2 server only has 1GB memory, so this could be the main problems.

[ghkdqhrbals]

1. Repeated topic connections that fail to resolve

I reset the topics and their connected brokers. And found that they don't consumes big resources.

2. Too much Tomcat max-connections

Since our chatting-server has 1000 max NIO-threads. So this is a problem causing lack of resources. So I reduce thread usage and cpu consumption is little bit lower than before.