Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check idle CPU consumption #299

Open
Kiruel opened this issue Feb 8, 2019 · 5 comments

Comments

@Kiruel
Copy link

commented Feb 8, 2019

CPU usage is ~7% when no swap or nothing, i guess it's not normal.

  • What OS do you use?
    Mac OS 10.14.2 (18C54)
  • What marketmaker version do you run?
    mm2
@ArtemGr

This comment has been minimized.

Copy link
Collaborator

commented Feb 8, 2019

Note that we're using a debug build currently and unoptimized builds can be slow, this is quite expected.

(In a scripting language you have a thin layer of always slow scripting code running on top of an aisberg of highly optimized native code; whereas in a natively compiled language the entire aisberg might be slow or fast, depending on how you're compiling it; this is especially true for Rust, because the entire graph of dependencies is currently compiled using the same set of settings, including cryptography, I/O, collections and the rest of the low-level stuff; so it's normal to see a huge difference in speed between the debug and the release builds).

We should profile of course, but from an optimized release build.

@ArtemGr ArtemGr added this to the Beta release milestone Feb 8, 2019

@ArtemGr ArtemGr changed the title CPU usage not normal Check idle CPU consumption Feb 10, 2019

@cipig

This comment has been minimized.

Copy link

commented Apr 17, 2019

my mm2 instances are using 100% CPU all the time
if i do sudo strace -f -p 68941 (68941 is the PID of mm2) i see mostly this messages

[pid 68949] futex(0x5581daa1faa8, FUTEX_WAIT_PRIVATE, 0, {tv_sec=0, tv_nsec=705617}) = -1 ETIMEDOUT (Connection timed out)
[pid 68949] futex(0x5581daa29ec0, FUTEX_WAKE_PRIVATE, 1) = 0
[pid 68949] recvfrom(154, 0x7ff2f0000e70, 8192, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
[pid 68949] recvfrom(186, 0x7ff2f0003080, 8192, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
[pid 68949] recvfrom(187, 0x7ff2f0005290, 8192, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)

the fds shown (154,186 and 187) are this ones (from lsof -p 68941 | grep -E "(154|186|187)")

mm2     68941 flow  154u     IPv4 1312946716       0t0        TCP Debian-98-stretch-64-minimal:41981->static.176.116.201.195.clients.your-server.de:47772 (ESTABLISHED)
mm2     68941 flow  186u     IPv4 1312946717       0t0        TCP Debian-98-stretch-64-minimal:62249->static.18.87.4.46.clients.your-server.de:47772 (ESTABLISHED)
mm2     68941 flow  187u     IPv4 1312946718       0t0        TCP Debian-98-stretch-64-minimal:17671->static.11.78.4.46.clients.your-server.de:47772 (ESTABLISHED)

i don't know if that is the reason for the high CPU usage, but could be

@artemii235

This comment has been minimized.

Copy link
Collaborator

commented Apr 17, 2019

@cipig Hi, should be something different from the initial issue as there were only 7%, going to check this today.

artemii235 added a commit that referenced this issue Apr 17, 2019
@artemii235

This comment has been minimized.

Copy link
Collaborator

commented Apr 17, 2019

@cipig Should be fixed now, please try to update your node. There is a loop in LPinit checking MM context existence, 100% load was caused by this, I added 1 second sleep between checks that solved CPU load. Also removed a couple of useless threads which should reduce the load a bit too.

@cipig

This comment has been minimized.

Copy link

commented Apr 17, 2019

Thanks a lot, that fixes it.
@Kiruel can you please check if that fixes your problem too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.