TabNine consumes excessive CPU for extended periods

[seren]

I'm using TabNine 1.0.9 in SublimeText 3.1.1 on OS X 10.13.6.

Much of the time, the TabNine process shows that it's using 100-120% CPU. The rest of the time it bounces between 0% and 40% CPU. I'm not sure what is causing this; I thought it might be related to backup activity causing a lot of filesystem events, but now I'm not sure.

Assuming this is not expected behavior, I'm happy to help troubleshoot this.

I should add that I really like TabNine's functionality, so I've been putting up with the high CPU except when running on battery.

[zxqfl]

Does the high CPU usage stop once you stop typing? If so, it's likely TabNine updating the index in the background to reflect the changing state of the current file. This process is not very efficient at the moment, and is a high priority for improvement within the next month or so.

[seren]

Thanks for the response. No, the high CPU usage continues even when Sublime is in the background.

I tried to see if there was a correlation between typing in Sublime as well as saving changes to disk when TabNine wasn't consuming much CPU, but there doesn't seem to be one: CPU usage it briefly jumps a bit but it doesn't begin one of these extended periods (I haven't measured it, but 10s of minutes) of maxing out a core.

[zxqfl]

This could be caused by an issue which I just fixed which could cause indexing threads to get stuck in an infinite loop. Once I release the fix I'll comment here and you can let me know if it fixed the problem. I plan to include better error reporting in the next release as well.

In the meantime, as a workaround you could try creating a .tabnineignore file in your project root which lists every folder, causing TabNine to ignore everything except the file you're currently editing. You can then remove folders from the list and check that the issue doesn't reoccur.

Thanks for the report.

[seren]

Thanks. I'll keep an eye out for the update.

[zxqfl]

The update is released! (Full changelog)

Unfortunately I did not include the improved error reporting in this release.

[maciej-trebacz]

Hello,

I'm using version 1.0.10, but the high CPU usage bug seems to be still present. I've recorded the CPU usage while Sublime was in the background: https://transfer.sh/vj81U/tabnine.mov . This is on latest macOS Mojave on a Macbook Pro.

Edit: Another data point: I killed the TabNine process by hand and when I started typing it was recreated, but now it doesn't consume CPU while in background.

[seren]

The CPU usage seems a little better these days. The TabNine process isn't usually pegging a core at 100%. It does tend to hang between 20% and 70% for extended periods however.

[seren]

Just an update that recently, it's been hanging closer to 75% a lot of the time. This is with and without any interaction with SublimeText. I notice it because my system will slow if there's something else competing with it for large amounts of CPU.

The most recent example is when I started up Sublime, TabNine hung around 70-75% CPU for about 10 minutes. I took a process sample via Activity Monitor around which point (could easily be coincidence) the CPU % rose to around 120% for a few minutes. Then it dropped down to 0-10%. Maybe this is normal startup behavior, but it sometimes happens even when Sublime has been running for a while.

If there's any way to get debug logging or performance profiling, I'm happy to take a look. I've uploaded the process samples in case they're helpful:

Idle High CPU:
https://pastebin.com/khvLbPBH

Idle Ultra-high CPU:
https://pastebin.com/r26bwjsF

Idle Low CPU:
https://pastebin.com/c5DGvQGT

[deevus]

Getting the same issue with the deoplete-tabnine plugin in vim.

The log doesn't appear to be getting thrashed though.

[2019-02-25][13:43:32][server][DEBUG] [autocomplete] 7 ms
[2019-02-25][13:43:32][server][DEBUG] [autocomplete] 8 ms
[2019-02-25][13:43:32][server][DEBUG] [autocomplete] 3 ms
[2019-02-25][13:43:32][server][DEBUG] [autocomplete] 7 ms
[2019-02-25][13:43:33][server][DEBUG] [autocomplete] 8 ms
[2019-02-25][13:43:33][server][DEBUG] [autocomplete] 10 ms
[2019-02-25][13:43:33][server][DEBUG] [autocomplete] 6 ms
[2019-02-25][13:43:34][server][DEBUG] [autocomplete] 12 ms
[2019-02-25][13:43:34][server][DEBUG] [autocomplete] 7 ms
[2019-02-25][13:43:34][server][DEBUG] [autocomplete] 11 ms
[2019-02-25][13:43:36][server][DEBUG] [autocomplete] 4 ms
[2019-02-25][13:43:36][server][DEBUG] [autocomplete] 5 ms
[2019-02-25][13:43:38][server][DEBUG] [autocomplete] 2 ms
[2019-02-25][13:43:42][server][DEBUG] [autocomplete] 14 ms
[2019-02-25][13:43:42][server][DEBUG] [autocomplete] 15 ms
[2019-02-25][13:43:43][server][DEBUG] [autocomplete] 10 ms
[2019-02-25][13:43:43][server][DEBUG] [autocomplete] 7 ms
[2019-02-25][13:43:43][server][DEBUG] [autocomplete] 2 ms
[2019-02-25][13:43:45][server][DEBUG] [autocomplete] 17 ms
[2019-02-25][13:43:45][server][DEBUG] [autocomplete] 16 ms
[2019-02-25][13:43:45][server][DEBUG] [autocomplete] 11 ms
[2019-02-25][13:43:46][server][DEBUG] [autocomplete] 8 ms
[2019-02-25][13:43:50][server][DEBUG] [autocomplete] 9 ms
[2019-02-25][13:43:53][server][DEBUG] [autocomplete] 18 ms
[2019-02-25][13:43:54][server][DEBUG] [autocomplete] 26 ms
[2019-02-25][13:43:54][server][DEBUG] [autocomplete] 10 ms
[2019-02-25][13:43:55][server][DEBUG] [autocomplete] 5 ms
[2019-02-25][13:43:56][server][DEBUG] [autocomplete] 16 ms
[2019-02-25][13:43:56][server][DEBUG] [autocomplete] 11 ms
[2019-02-25][13:43:57][server][DEBUG] [autocomplete] 3 ms
[2019-02-25][13:43:57][server][DEBUG] [autocomplete] 16 ms

[deevus]

35% of memory is roughly 5GB on my system

[Stuk]

I'm seeing the same issue on macOS 10.14.2 with Sublime Text 3, even when I'm not using Sublime or modifying the repo with the command line.

99% CPU usage, 2.2 GB memory usage.

[maciej-trebacz]

@zxqfl Is TabNine abandoned already? This issue is a dealbreaker and I cannot use TabNine even though I paid for it.

[seren]

Yeah, I recently had to remove it so that my battery would last more than an hour. It's too bad, because this is the only thing keeping me from buying it.

[zxqfl]

TabNine is not abandoned. Is the repository causing this issue open source? This may allow me to replicate and fix the issue.

[zxqfl]

I'm sorry for the inconvenience. You can contact me at jacob@jacobjackson.com to receive a full refund.

[Deewiant]

@zxqfl I put a comment with some instructions on how to replicate this with OpenWrt over at #43, along with some analysis of what looks like a bug (and possibly even the cause).

[0xpr03]

I also have high cpu usage over time without doing anything, it's a full core permanently utilized as you can see below:

I could provide the repository.
Edit: Currently blasting full speed since 5 minutes. It could be related to errors in macro expansions. As it works perfect again if I remove those errors and restart vs-code.

[seren]

As a follow-up, after removing TabNine I discovered that Sublime was burning CPU for a different reason: a .git folder in my home directory.

Thus I may have blamed TabNine unfairly due to it being installed around then same time that this issue cropped up. I've re-installed it and will report back if the high CPU usage returns.

[0xpr03]

Thus I may have blamed TabNine unfairly due to it being installed around then same time that this issue cropped up. I've re-installed it and will report back if the high CPU usage returns.

But as you can see on my post above it also happens on a completely unrelated editor and without that one going nuts. (VSCode) So I wouldn't close the issue.

[seren]

No, I'll leave it open for the other issues. I'm not sure mine is resolved either.

[deevus]

@zxqfl Any progress?

[tbodt]

@zxqfl Is there any chance you can share the source code with me so I can diagnose this? It's getting really annoying.

[0xpr03]

I disabled it for now because on some projects I've had a constant 100% CPU usage on one core I couldn't stop..

[zxqfl]

I'm working on open sourcing the code that TabNine uses to index projects. Once this is done, you'll be able to investigate the issue yourself if you wish.

In the meantime, I can look into it if you can provide steps that I can use to reproduce the issue.

[trishume]

Other people who find this issue: Check or think whether it might be an LSP server.

Personally sometimes the Rust LSP server triggered by TabNine consumes all my cores to build the project in the background. One of my processor monitoring apps attributes this CPU to Sublime rather than the LSP server process because it's under Sublime in the tree. I'm not sure what Activity Monitor shows by default. This isn't a problem with TabNine though.

Anyhow if you're having CPU usage issues consider whether this might be the case and maybe you want to ditch your LSP server and just use raw TabNine with TabNine::no_sem

[kylepolich]

I'm a new user giving TabNine a try and experiencing this same issue.

[ionling]

Same with me: macOS, Emacs, and TabNine 2.0.4

[nchase]

this is happening for me too. I can provide more information if it helps

[sersorrel]

Is there any way to provide logs that could help debug this?

~/.vim/plugged/tabnine-vim/binaries/2.1.5/x86_64-unknown-linux-gnu/TabNine is the process that's currently using 100% CPU for me. There doesn't appear to be anything in either of the stdout or stderr files which its arguments point to.

[BugCWP]

When using vscode, the CPU usage does not reach 70% when inputting, causing the computer to be stuck.

[oolongtea]

I'm seeing the same kind of issue. CPUs are being maxed out for minutes at a time.

$ ./TabNine --version
TabNine 2.1.5 (x86_64-unknown-linux-musl)
Jacob Jackson <jacob@tabnine.com>

$ nvim --version
NVIM v0.3.8
Build type: RelWithDebInfo
LuaJIT 2.0.5

# using github.com/tbodt/deoplete-tabnine, git hash 5ba086d08f1da93ec9bbe9276dbc62fcb301aeed

update on 2019-09-26: I have 126gb of memory and after kill -9ing TabNine my used memory went from ~90 to ~20.

[larebsyed]

Happening to me as well. Also, the completion are slow to arrive and halt the PC at time. This has started happening recently. I am using Emacs with Spacemacs configuration.

[koreno]

Hey @zxqfl - what can we do to get this issue looked into? TabNine is amazing, but I'm finding I have to terminate it to save battery/energy.
Please help!

[p5a0u9l]

been using via coc.nvim, disabling after experiencing the same, and then seeing this on answered thread.

[joelcox22]

I'm having the same issue on an ec2 t2.medium instance with the tab nine vim plugin. This seems to completely lock up my instance every few hours, to the point where I need to reboot it to continue.

[mpfaff]

I've been having the same issue. I'm on macOS Mojave using VSCode. As a temporary fix, I've disabled TabNine on a per-workspace basic, because I use VSCode to type my notes and do programming at school and the battery life was getting really bad (~ 2 and a half hours).

[Shougo]

You can disable TabNine Local feature in TabNine::config.

[jansol]

Possibly related, I can't even open the config on my work machine. Whenever I type TabNine::config it opens the browser yes, but the page load times out and TabNine pegs all CPU cores to 90% until I kill it. EDIT: and that's 64 logical cores so it's not like the problem is that the PC is too weak...

[zmitchell]

I'm getting this when I'm just writing Markdown. It pegs the CPU while I'm typing, and then goes back to idle. I'm using the tabnine-vim plugin. Since I'm writing Markdown, that should eliminate any kind of LSP issues, correct?

[e-kwsm]

By default, tabnine seems to use the half of the available cores: for example, CPU usage of TabNine --benchmark-local is nearly 400% on an 8-core machine.
If "Power saving mode" is enabled, the load is reduced to about 130%.

However, sometimes CPU usage of TabNine exceeds 300% while editing a python file in power saving mode; the editor is Neovim and LSP is disabled.

Is it possible to set the number of threads?

---

$ TabNine -v
TabNine 2.1.20 (x86_64-unknown-linux-musl)

[5udu3r]

i have the same issue in mac book pro + tabnine+ pycharm

[drorweiss]

Hi All, we have a fix in beta (version 2.3.4) for excessive CPU and memory consumption by TabNine.
We'd appreciate it if you try it out and let us know if it solves the issue for you.
To try the fix, you have to enable the beta version. Just open TabNine's config by typing TabNine::config in your editor, and then check both Update TabNine automatically and Receive beta releases of TabNine Then, restart your editor, and press TabNine::config to verify you have TabNine 2.3.4. or newer. Please let us know if this version solves the problem for you. We plan to make it available to all users in a couple of weeks.

[0xpr03]

I have no such settings in the VS-Code extension

[ionling]

Here is top command output with version 2.3.4 and Deep TabNine Local enabled. About 1.9G memory get used.

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
 7755 vision    20   0 2190848   1.9g  10280 S   0.0  12.3   1:34.71 TabNine

[Shougo]

It is normal memory usage.
Tabnine local eats lot of memory

[cstuder]

@drorweiss I've been using the 2.3.4 beta on Windows 10 & Visual Studio code for a week now and am happy to report that CPU usage is normal. I'm no longer encountering the problems I've reported in #148 .

[drorweiss]

Thanks, @cstuder
We're continuing to gradually release the correction to all users.
I'll add the comment about the fix in the original issue.

[NilsIrl]

Hi All, we have a fix in beta (version 2.3.4) for excessive CPU and memory consumption by TabNine.

Were compromises take taken to allow this? And has this been included in version 2.3.5 (I've enabled beta and for me the CPU usage is still very high even though I'm only using TabNine Cloud)?

[drorweiss]

Hi @NilsIrl

Were compromises take taken to allow this?

Not really. The only "compromise" is that occasionally, TabNine would stop itself for a very brief period to prevent consuming excessive resources.

And has this been included in version 2.3.5

Yes.

CPU usage is still very high even though I'm only using TabNine Cloud)

CPU usage can get high for short periods of time. Do you see high CPU usage for long periods of time?

[NilsIrl]

CPU usage can get high for short periods of time. Do you see high CPU usage for long periods of time?

Yes, generally after I'm typing it is still high.

Right now though I can't get it to go over 20%. What I'm going to assume is that yesterday for whatever reason it was still using TabNine local even though I had told it not to. And that with a system restart the setting somehow got applied.

So for me, assuming the current behavior doesn't change, I would consider it fixed.

[yeukfei02]

on macbook pro 16 inch intel, macos version 12.6.1..it seems the same issue
use a lot of CPU and memory...

I use vscode...

does it fixed?

[kyakaze]

I just come to say this. It happens again right now with the latest version.
I'm using Tabnine Pro with IntelliJ, MacOS Big Sur (11.6), 16GB RAM. It's eating up all my CPU, making the CPU to be throttled to 40-50% of its capacity.

Is there any solution for this?

[sarduri]

Hi
Every 5 seconds the CPU consumption of TabNine jumps up to 75% even though not working on VisualStudio and keeping the Windows just in the background.
Tabnine 3.79.0 Liunx Mint 21.3