Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

High Cpu node 100% #25968

Open
maxthetor opened this issue Jun 23, 2022 · 4 comments
Open

High Cpu node 100% #25968

maxthetor opened this issue Jun 23, 2022 · 4 comments

Comments

@maxthetor
Copy link

Description:

from time to time our rocket stops working.
today analyzing better, we noticed an abnormal traffic
coming from the IP address 51.81.47.27.

when blocking this address and restarting the rocket everything is normal.
When we unlock this IP, the node returns to 100% cpu. when this happens the rocket stops working.

this IP address seems to be from the rocket cloud.
what's the logic?

de vez em quando nosso rocket para de funcionar.
hoje analisando melhor, percebemos um trafego anormal
vindo do endereco IP 51.81.47.27.

ao bloquear esse endereco e reiniciar o rocket tudo fica normal.
qunado desbloqueamos esse IP, o node volta a ficar com 100% de cpu. quando isso acontece o rocket para de funcionar.

esse endereco IP parece ser do rocket cloud.
qual a logica?

Steps to reproduce:

  1. if the rocket stops working, and the node is at 100%
  2. tcpdump -n -i eth0 net 51.81.0.0/16
  3. netstat -anp | grep 51.81.47.27 | cat -n
    (69 tcp 0 0 177.22.32.247:443 51.81.47.27:31706 ESTABELECIDA 3842/nginx: worker)

Expected behavior:

rocket stops working

Server Setup Information:

  • Version of Rocket.Chat Server: 3.18.3
  • Operating System: Ubuntu 16.04.4 LTS
  • Deployment Method: snap
  • Number of Running Instances: 1
  • DB Replicaset Oplog:
  • NodeJS Version: 12.22.1 - x64
  • MongoDB Version: 3.6.14

erro_rocket_1

Relevant logs:

when the node is at 100% cpu, no log is generated

@maxthetor maxthetor changed the title HIGH CPU NODE 100% High Cpu node 100% Jun 23, 2022
@LuluGO
Copy link
Contributor

LuluGO commented Jun 28, 2022

Hello, thanks for reporting this, do you use any omnichannel app in your server? for example facebook, twitter, instagram, whatsapp, etc.

@maxthetor
Copy link
Author

yes

@nileshA-addweb
Copy link

nileshA-addweb commented Jun 12, 2023

+1

We are also getting same issue where RC server and client versions are as below. We are using Docker based RC setup in single EC2 (2vCPU, 8GB RAM, 80G SSD Disk)

Server Setup Information:

Version of Rocket.Chat Server: 6.2.2
Operating System: Ubuntu 20.04.6 LTS
Deployment Method: Docker
Number of Running Instances: 1
DB Replicaset Oplog: Yes
NodeJS Version: 14.21.2 - x64
MongoDB Version: 5.0.15
MongoDB Engine: wiredTiger
USE_NATIVE_OPLOG=true

Client Setup Information

Desktop App: App 3.8.13
Operating System: Ubuntu 22.04.2 LTS

@nileshA-addweb
Copy link

Can anyone help here to overcome this issue which is happening anytime on random basis?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants