-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cpu usage 100% caused by silence and fs-manager #1088
Comments
From the other reports I've seen on the dev mailing list, your couchdb instance has been hacked and someone has installed a bitcoin mining instance on your service. |
@garrensmith Yep, you were right. These were crypto miners. I found a mailing list mail from Sinan Gabel explaining how to turn off the miner. Unfortunately I do not know how to link to Sinan's mail so here is the gist: I needed to remove a cron job created from user couchdb:
So thanks a lot to you and Sinan Gabel. Of course I will also have to rebuild my servers now. My biggest concern now is: How to prevent this from happening again, as these couch instances were not in admin mode. |
If you upgrade to the latest CouchDB you will be fine. We released Security advisories about this late last year.
Get Outlook for iOS<https://aka.ms/o0ukef>
…________________________________
From: Alexander Gabriel <notifications@github.com>
Sent: Thursday, January 4, 2018 6:11:42 PM
To: apache/couchdb
Cc: garren smith; Mention
Subject: Re: [apache/couchdb] cpu usage 100% caused by silence and fs-manager (#1088)
Closed #1088<#1088>.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#1088 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAK9AovuXTP7F09XGuuWab6s341FPvtGks5tHPg-gaJpZM4RS4TQ>.
|
@garrensmith: thanks a lot for your great help! |
I was also hit by this invasion. I am using CouchDB 1.6.1 from stable PPA https://launchpad.net/~couchdb/+archive/ubuntu/stable How can I update it to solve this security vulnerability, since there are no updates in the PPA? Or there is an available workaround? |
Found a solution: #787 |
I would not update. Eliminating the cron job is not enough. In my case the
problem reappeared inside minutes.
I rebuilt the servers with v2.1.1, then synced my DB. Don't know if syncing
could even be risky...
But so far it works.
vom Handy
Am 12.01.2018 9:23 nachm. schrieb "Paulo Coghi" <notifications@github.com>:
… I was also hit by this invasion. I am using CouchDB 1.6.1 from stable PPA
https://launchpad.net/~couchdb/+archive/ubuntu/stable
<https://launchpad.net/%7Ecouchdb/+archive/ubuntu/stable>
How can I update it to solve this security vulnerability, since there are
no updates in the PPA? Or there is an available workaround?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
<#1088 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAo4V_GHG68fBQ-qEIwg8ztIjHQVSJ9Bks5tJ74DgaJpZM4RS4TQ>
.
|
@barbalex , this happened in which version? I upgraded to version 2.1.1 after using 1.6.1 and experiencing this problem. I used So far, everything working perfectly. The only detail was that |
@barbalex , just removing the cron jobs is certainly not enough since the vulnerability remains open if you do not upgrade. |
This was during the short period before I rebuilt the servers. So still with version 1.6.1. And you may be better off not migrating the users: The "bad guys" added a few users. And also a few db's (with only one document each). |
Just for reference if anyone is looking for the process name. It's |
I've had the same problem. A Monero miner was installed through a curl request to 192.99.142.232:8220/logo3.jpg which is in turn a bash script. When it starts with 192., doesn't it refer to a local network? |
@AlbertDavid94 No, the private IP space is only The full set of private IP space blocks is documented in RFC 1918. |
Expected Behavior
Usually CPU usage should be below 10%.
Current Behavior
CPU usage is at 100% 95% of the time.
On 5 different production servers I have noticed that cpu usage has been at 100% for most of the time during at least the last 30 days.
From running
top
it seems that most often Silence is the main culprit, using 98% of the cpu. Sometimesfs-manager
andSilence
both use 49% each. And on one server it seems thatfs-manager
uses 98%.There are no active tasks running.
Two of these servers couches contain very low number of docs (a few hundred). The others have about 30'000 docs.
What could cause this behaviour?
How problematic is it?
How can I prevent it?
Your Environment
The text was updated successfully, but these errors were encountered: