High CPU usage when using SOCKS proxy

[Alcedema]

Since recently upgrading Jackett, the CPU load from the service ends up going through the roof; constant 60-80% which is slowing down plex transcoding.

Service restart works ok, don't know how long after starting the service, the CPU issue arises. Auto update is not on, but I had checked it on in the past but then disabled it.

I am using jackett through a socks5 proxy.

Have looked at the other issues relating to this problem, but they have been closed with no real fix apart from installing on a separate VM which isn't really a fix..

Jackett version: 0.8.676 & 0.8.692
Windows 10.0.16299

---

log.txt

[supremacy2k]

Might be related to
#2498

[kaso17]

Can you please try running it without the socks proxy?
The socks proxy implementation is a big workaround with a lot of potential bugs.

[Alcedema]

Will do, and will let you know. Although the only two sources I have in jackett require a proxy due to ISP blocking, so if this works it may not be just down to the proxy, but something else handling/parsing the requests.

[Alcedema]

9 hours with no problems. I put the proxy settings back and set my IPT address to a defunct url, tested the connection and instantly Jackett's CPU is now permanently ~ 50%. Maybe this problem occurs whenever the connection to an URL is dropped when using a proxy?

[nordicblue]

Following. Continuing to have CPU issues with Jackett still. Tried running on both Windows and Ubuntu and still the same high CPU. I haven't tried disabling the SOCKS5, so going to try that, now that my Linux has permanent VPN connection running...

[evilvibes]

Also having this issue.

[nordicblue]

Removing the SOCKS5 tremendously helped and has lowered the CPU usage. Still spiking every few days, but I am taking care of it with monit and simply calling a restart if it's over 40% for 5 cycles (min). I know it's a bandaid, but I'm not willing to give up on Jackett, I love it too much.

[lawrenceadams]

Having same issue as well (Docker, SOCKS enabled, auto update disabled), restarting usually solves the problem for a while so I'm hoping a cron job to restart it every 24 hours or so should do the job

[extric99]

Having the same issue (Docker on Synology, SOCKS disabled, auto update disabled) Restarting the docker image solves it for a certain period 6h-2days. My Jackett Version is 0.8.737.0

[halsafar]

Also having this issue. Sonarr does the same thing probably for the same reason.

[kaso17]

For everyone having this problem (especially in combination with socks).
Please try using the httpclient2 (-c httpclient2 command line option).

[halsafar]

I have added this option and removed my auto-restarting of Jackett. The problem typically manifests itself at least once a day so we shouldn't have to wait long.

[halsafar]

Well that did not take long to test. Jackett is consuming all my CPU at this moment. Even with -c httpclient2 it is getting into this state.

[ElMoselYEE]

I've been having this issue as well for a while. Today I noticed not just high CPU but also very high memory. 200% CPU usage and 4.2G memory usage.

Jackett Version 0.8.770.0
SOCKS5 enabled

Just gonna have Jackett auto-restart for now.

[fbicknel]

Jackett Version 0.8.994.0

Using SOCKS5 proxy.

... and still doing it.

[kaso17]

It's confirmed that using a socks proxy causes high CPU load. This is most likely caused by some problem or misuse of the https://github.com/postworthy/SocksWebProxy/ library.
I'm not planning to look into this, some help would be very much appreciated.

[codymhorton]

Seeing the same behavior disabled for now.

[fbicknel]

I wrote a very simple Perl script to watch Jackett for misbehavior and restart the service if it does start misbehaving. https://github.com/fbicknel/watchdog.git

Use at your own risk and peril, although at least it's simple enough for you to easily determine if it will be harmful or not. In my own testing, I've found that it's not.

About all I can say for it is that it does the job. Well. I can also say that it doesn't rely on ps to average the cpu over the life of the process, but it does sample 2-minute intervals and computes the cpu average over that two minutes. So, that's a plus.

It just logs to stdout and has no command line options or configuration. It also could rip Jacket out from under a Sonarr or Radarr instance that's using it, which might indeed be mildly harmful. So it's really just a proof of concept.

Hope someone else finds it useful.

[ExorcistF1]

For me jackett service can't start automatically (after restart PC) then i use socks5 (he try to running but after 10 second stopped). With http proxy no problem.

[jbmorgado]

I confirm I also have this issue. I'm not using socks and I have auto-update disabled.

[n1nj4888]

I wrote a very simple Perl script to watch Jackett for misbehavior and restart the service if it does start misbehaving. https://github.com/fbicknel/watchdog.git

Use at your own risk and peril, although at least it's simple enough for you to easily determine if it will be harmful or not. In my own testing, I've found that it's not.

About all I can say for it is that it does the job. Well. I can also say that it doesn't rely on ps to average the cpu over the life of the process, but it does sample 2-minute intervals and computes the cpu average over that two minutes. So, that's a plus.

It just logs to stdout and has no command line options or configuration. It also could rip Jacket out from under a Sonarr or Radarr instance that's using it, which might indeed be mildly harmful. So it's really just a proof of concept.

Hope someone else finds it useful.

I'm also having the same issue with jackett sometimes hitting 100% cpu until restarted... It seems to be fine for a day or so then the same thing happens again - I'm also using SOCKS and jackett is running as a docker container...

Thanks for the script above which restarts the jackett service but does anyone have a similar script that checks for the jackett docker container CPU usage being high and, if above a threshold, restarts the container?

I'm thinking the script needs to do something like:

Run "docker stats --no-stream <JACKETT CONTAINER NAME>" and parse the CPU % column returned for the jackett container. If CPU % over a specific threshold (ideally over a number of runs), then run "docker restart <JACKETT CONTAINER NAME>"

Any good bash scripters out there that could point me in the right direction for this?

Thanks!

[fbicknel]

Try this branch

Sorry for the rudimentary script with no docs.

Try something like:

$ ./watchdog.pl --docker <name of docker container>

Let me know how it works for you.

Working on a more general version with a config file so the threshold can be configured, etc.

[n1nj4888]

Try this branch

Sorry for the rudimentary script with no docs.

Try something like:

$ ./watchdog.pl --docker <name of docker container>

Let me know how it works for you.

Working on a more general version with a config file so the threshold can be configured, etc.

This would be great! Thanks!

There seems to be an issue with the script currently in that it appears to be reading the output MEM USAGE column from "docker stats" command currently rather than the actual "CPU %" column for CPU usage??

docker stats --no-stream linuxserver-jackett

CONTAINER             CPU %               MEM USAGE / LIMIT   MEM %               NET I/O             BLOCK I/O           PIDS
linuxserver-jackett   0.06%               102MiB / 512MiB     19.92%              37.2kB / 162B       0B / 0B             0

./watchdog.pl --docker linuxserver-jackett
Wed Sep 19 08:48:01 +08 2018
Argument "100.6MiB" isn't numeric in numeric gt (>) at ./watchdog.pl line 44.
DockerID: ad2c47cfe66f (linuxserver-jackett)
cpu was 100.6%
restarting service

[fbicknel]

Well, that's odd... I tested that on my Ubuntu system and there it appears in the third column. I'll look closer at it later. bick-ubtu3 ~$ docker stats --no-stream test-web CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS 984400e3ae97 test-web 0.00% 0B / 0B 0.00% 0B / 0B 0B / 0B 0 bick-ubtu3 ~$ docker --version Docker version 18.06.1-ce, build e68fc7a bick-ubtu3 ~$

On Tue, Sep 18, 2018, 20:49 n1nj4888 ***@***.***> wrote: Try this branch <https://github.com/fbicknel/watchdog/tree/feature/docker-support> Sorry for the rudimentary script with no docs. Try something like: $ ./watchdog.pl --docker <name of docker container> Let me know how it works for you. Working on a more general version with a config file so the threshold can be configured, etc. This would be great! Thanks! There seems to be an issue with the script currently in that it appears to be reading the output MEM USAGE column from "docker stats" command currently rather than the actual "CPU %" column for CPU usage?? docker stats --no-stream linuxserver-jackett CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS linuxserver-jackett 0.06% 102MiB / 512MiB 19.92% 37.2kB / 162B 0B / 0B 0 ./watchdog.pl --docker linuxserver-jackett Wed Sep 19 08:48:01 +08 2018 Argument "100.6MiB" isn't numeric in numeric gt (>) at ./watchdog.pl line 44. DockerID: ad2c47cfe66f (linuxserver-jackett) cpu was 100.6% restarting service — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2602 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABmTGPNjvS8yRs1GpF750SjGY8YVmpwmks5ucZSzgaJpZM4SJRO-> .

--

…

Sent from my stone tablet - Frank

[n1nj4888]

I’m running docker 17.05.0-ce which is the standard Synology NAS docker package. If docker have reordered the columns between versions, I think there is the ability to specify the format of the stats returned using

docker stats --format

I’ll see if I can find out some more info on this...

Usage: docker stats [OPTIONS] [CONTAINER...]

Display a live stream of container(s) resource usage statistics

Options:
-a, --all Show all containers (default shows just running)
--format string Pretty-print images using a Go template
--help Print usage
--no-stream Disable streaming stats and only pull the
first result

[fbicknel]

Ok - please try the latest push to that same branch. See if that fixes it.

I feel guilty we're having this conversation in Jackett's space. :)

[ghost]

Agree @WHiZSTA The Jackett team should never have accepted the PR for proxy support. It has been problematic since day one. Have added a warning to the Jackett dashboard. If no one from the community is prepared to spend time on fixing it, then proxy option will be removed from Jackett

[n1nj4888]

Hi @sv01a - Any update on this please? Actually both this and #4862. @FlightLevel indicates that if no-one will support proxy in jackett it maybe removed.

I’m sure there’s a number of us that would like to prevent that happening!

[WHiZSTA]

just my 2 cents. I had written very basic socks proxy code over 10yrs ago for an app, it was super simple to implement. upon an open connection, you send a binary string with the realip/port you want to connect to, and then it gives you a response code when it connects/fails and tunnels the traffic.

granted I never messed with authentication stuff back then which I'm sure complicates it. that said, tech has changed since then, you can now setup proxies to auto auth/allow access from diff ips or trusted computers including localhost, etc.

if the authentication is the hard part of implementing simple proxy support for jackett, just rip it out and people can reconfigure the proxy. i'd rather have that then no support at all.

if the proxy code itself is as bad as some here have suggested, remove the proxy code and add in a network interface option for the network bind. that is even simpler, normally 1 line of code, and would solve the problem in a different way for many of us. aka, I had to set up a local host proxy to tunnel the traffic over a VPN interface because I can't select which interface to bind to. if I could select that, then I wouldn't need a proxy. others could use this in a similar fashion to create tunnels/proxies.

[sv01a]

we can try to switch from https://github.com/postworthy/SocksWebProxy to https://github.com/extremecodetv/SocksSharp .

I haven't much time for digging SocksWebProxy. Especially that i have't any issues with socks5 on win10 and can't repeat it as well

[WHiZSTA]

we can try to switch from https://github.com/postworthy/SocksWebProxy to https://github.com/extremecodetv/SocksSharp .

I haven't much time for digging SocksWebProxy. Especially that i have't any issues with socks5 on win10 and can't repeat it as well

im all for that, willing to try anytjing, as i write this i just had to restart jackett again. it was eating 17% constantly on my i7 for hours. now it's at 0%.

[halsafar]

FYI this bug only happens with SOCKS.
HTTP proxy works perfect, no high CPU usage bug.

[sv01a]

In Jackett is used very old fork of SocksWebProxy - https://github.com/lloydsparkes/SocksWebProxy . I found out that in original repo was added disposing of webproxy. in theory It's could fix this issue.

I think that we could to try to update SocksWebProxy first and if it would't work - move to SocksSharp. Moving to SocksSharp requires much more time that update

[WHiZSTA]

FYI this bug only happens with SOCKS.
HTTP proxy works perfect, no high CPU usage bug.

thanks for the heads up if they cant get socks working right ill setup an http proxy instead.

[n1nj4888]

Hi @sv01a, Any progress with this? I don’t mind testing out some builds if this helps?

[sv01a]

someone tryed to use httpclient2(-c httpclient2) except @halsafar ? to ensure that client is seted up you have to see Info Using HTTP Client: HttpWebClient2 in log.

[n1nj4888]

Not sure what you mean @sv01a? Do you simply mean changing the proxy type from SOCKS to HTTP in the Jackett web GUI? Or running jackett with a “-c httpclient2” command line switch?

Even if the HTTP proxy works, this wouldn’t fix the actual SOCKS proxy issues?

Out of interest, does the HTTP proxy support both HTTP and HTTPS?

[sv01a]

I mean "switch jackett client in command line by passing -c httpclient2 parameter" and check then that an issue still present.

[sv01a]

httpclient2 use same library, but a little bit in different way. And it's works for me, but i need to know what it works for someone else

[ghost]

Is anyone still having this issue using the standalone version of Jackett?

[n1nj4888]

Hi @FlightLevel - I’ve got a script running which checks and logs the CPU usage of the jackett container every 2mins. Since pointing it to the latest jackett container, it hasn’t triggered a restart of the container. It used to trigger a restart of the previous jackett containers on average once a couple of days... so will need to give it a few more days but, from my observations so far, this is no longer an issue...

[florensie]

@n1nj4888 Did any more restarts trigger? I wonder what change could've fixed it.

[n1nj4888]

I am using @fbicknel ‘s watchdog script (docker branch) that is mentioned in this thread above. It checks if the CPU usage of the jackett docker container every 2 mins. If the jackett container uses > 30% CPU on two successive polls, it restarts the container and logs this to file - So far no restarts of the jackett container since using the new default httpclient (and socks5 proxy) in over a week

[florensie]

Is that in the latest stable build or do I need to get an experimental one?

[n1nj4888]

Latest stable

[jackettbot]

Hi @Alcedema,

No response has been received for 7 days. To prevent issue tracker clutter, this issue will now be closed. To re-open the issue, please provide the information requested and the issue will automatically re-open.

[Derkades]

I am still having the issue (on FreeBSD, according to pkg no update available)

[mRaRn]

I also still have the CPU issue. Latest version on Synology/Docker.

[dequeant]

I'm still having this issue.

I'm running the standalone version on both linux (DietPi, latest stable) and Windows 10. It's happening on both machines. Uninstalling jackett and using usenet only until either this is addressed, or developers ask to test a fix.

Running jackett 0.11.385.0.
Running PIA socks5 proxy on both versions. Disabling the proxy fixes the issue on both versions.

[halsafar]

They call it Jackett because it keeps your server warm.

[lsaadeh]

any update on this one? are any of the proxies working for secure searching