Codeium throws errors on large files

[wonderbeel]

Hi, I am playing around with this plugin and found an issue: I have treesitter configured to disable text highlight in big files (in those cases it is too slow and makes the editor hang for a couple of seconds after each key stroke, falling back to the regex parser solves this issue) and it seems that doing this makes your plugin break with the following error

Error detected while processing TextChangedI Autocommands for "*":
Error executing lua callback: .../.local/share/nvim/lazy/plenary.nvim/lua/plenary/job.lua:405: Failed to spawn process: {
  _additional_on_exit_callbacks = {},
  _shutdown_check = <userdata 1>,
  _stderr_results = {},
  _stdout_results = {},
  _user_on_exit = <function 1>,
  args = { "-sSL", "-D", "/tmp/plenary_curl_2fe047dc.headers", "--compressed", "-X", "POST", "-H", "Content-Type: application/json", "--data-raw", "{\"editor_options\": {\"insert_spaces\": true, \"tab_size\": 2}, \"metadata\": {\"extension_name\": \"vim\", \"ide_name\": \"neovim\", \"request_id\": 3, \"api_key\": \"API_KEY_HERE\", \"extension_version\": \"1.1.32\", \"ide_version\": \"0.8.3\"}, \"document\": {\"editor_language\": \"sql\", \"language\": 41, \"line_ending\": \"\\n\", \"cursor_offset\": 1043599, \"text\": \"ENTIRE_FILE_HERE\"}

(api key and text have been cropped out just to be safe)

If I enable treesitter highlight again the error disappear but then the editor became unusable, it would be great to fix this issue.

Let me know if you need something else to help debug this issue :).

[jcdickinson]

I'm having trouble reproing this issue. How are you disabling TS? TSBufDisable?

[wonderbeel]

I am disabling it directly in the configuration like this (basically what treesitter suggests in their docs https://github.com/nvim-treesitter/nvim-treesitter/blob/master/doc/nvim-treesitter.txt#L117 )

      highlight = {
        enable = true,
        disable = function(_, buf)
          local max_filesize = 100 * 1024 -- 100 KB
          local ok, stats = pcall(vim.loop.fs_stat, vim.api.nvim_buf_get_name(buf))
          if ok and stats and stats.size > max_filesize then
            return true
          end
        end,
      }

Now that I reread the issue again it was slightly misleading, what I am actually doing is disabling the highlight module and fallback to the regex, not disabling the entire treesitter, sorry my bad.

[jemag]

Not sure if this is related, but also getting an error from the command window (q:), when editing text and completion is about to appear

Error detected while processing TextChangedI Autocommands for "*":
Error executing lua callback: Vim:E474: String "<89>
stack traceback:
	[C]: in function 'json_encode'
	...g/.local/share/nvim/lazy/codeium.nvim/lua/codeium/io.lua:407: in function 'post'
	.../.local/share/nvim/lazy/codeium.nvim/lua/codeium/api.lua:146: in function 'request'
	.../.local/share/nvim/lazy/codeium.nvim/lua/codeium/api.lua:287: in function 'request_completion'
	...ocal/share/nvim/lazy/codeium.nvim/lua/codeium/source.lua:103: in function 'complete'
	...jemag/.local/share/nvim/lazy/nvim-cmp/lua/cmp/source.lua:325: in function 'complete'
	...e/jemag/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:289: in function 'complete'
	...e/jemag/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:169: in function 'callback'
	...e/jemag/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:219: in function 'autoindent'
	...e/jemag/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:161: in function 'on_change'
	...e/jemag/.local/share/nvim/lazy/nvim-cmp/lua/cmp/init.lua:313: in function 'callback'
	...local/share/nvim/lazy/nvim-cmp/lua/cmp/utils/autocmd.lua:49: in function 'emit'
	...local/share/nvim/lazy/nvim-cmp/lua/cmp/utils/autocmd.lua:23: in function <...local/share/nvim/lazy/nvim-cmp/lua/cmp/utils/autocmd.lua:22>

---

Edit: Not related to this, created separate issue

[brunolpsousa]

I got the same error, but in my case the problem was not in the treesitter highlight, but in the big files themselves.

I was having this problem on a file with around 3500 lines and when I deleted around 1000 lines the problem was gone. On the other hand, I took a plain text file and copy/pasted until I got to about 5000 lines and the problem started.

I was also able to reproduce this in .zshrc and Lua files.

Edit: the problem is E2BIG: argument list too long

Full error

Error detected while processing TextChangedI Autocommands for "*":
Error executing lua callback: .../.local/share/nvim/lazy/plenary.nvim/lua/plenary/job.lua:405: Failed to spawn process: {
  _additional_on_exit_callbacks = {},
  _shutdown_check = <userdata 1>,
  _stderr_results = {},
  _stdout_results = {},
  _user_on_exit = <function 1>,
  args = { "-sSL", "-D", "/run/user/1000/plenary_curl_3ffe4d44.headers", "--compressed", "-X", "POST", "-H", "Content-Type: application/json", "--data-raw", "{\"editor_options\": {\"insert_spaces\": true, \"tab_size\": 2}, \"metadata\": {\"request_id\": 4, \"ide_name\": \"neovim\", \"extension_name\": \"vim\", \"api_key\": \"my_api_key\", \"extension_version\": \"1.2.85\", \"ide_version\": \"0.9.2\"}, \"document\": {\"editor_language\": \"zsh\", \"language\": 0, \"line_ending\": \"\\n\", \"cursor_offset\": 1288, \"text\": \"a lot of text...
  command = "curl",
  enable_handlers = true,
  enable_recording = true,
  interactive = true,
  pid = "E2BIG: argument list too long",
  stderr = <userdata 2>,
  stdin = <userdata 3>,
  stdout = <userdata 4>,
  user_data = {},
  <metatable> = <1>{
    __index = <table 1>,
    _create_uv_options = <function 2>,
    _execute = <function 3>,
    _pipes_are_closed = <function 4>,
    _prepare_pipes = <function 5>,
    _reset = <function 6>,
    _shutdown = <function 7>,
    _stop = <function 8>,
    add_on_exit_callback = <function 9>,
    after = <function 10>,
    after_failure = <function 11>,
    after_success = <function 12>,
    and_then = <function 13>,
    and_then_on_failure = <function 14>,
    and_then_on_failure_wrap = <function 15>,
    and_then_on_success = <function 16>,
    and_then_on_success_wrap = <function 17>,
    and_then_wrap = <function 18>,
    chain = <function 19>,
    chain_status = <function 20>,
    co_wait = <function 21>,
    is_job = <function 22>,
    join = <function 23>,
    new = <function 24>,
    pid = <function 25>,
    result = <function 26>,
    send = <function 27>,
    shutdown = <function 28>,
    start = <function 29>,
    stderr_result = <function 30>,
    sync = <function 31>,
    wait = <function 32>
  }
}
stack traceback:
        .../.local/share/nvim/lazy/plenary.nvim/lua/plenary/job.lua:405: in function '_execute'
        .../.local/share/nvim/lazy/plenary.nvim/lua/plenary/job.lua:448: in function 'start'
        ....local/share/nvim/lazy/plenary.nvim/lua/plenary/curl.lua:288: in function 'post'
        ...o/.local/share/nvim/lazy/codeium.nvim/lua/codeium/io.lua:452: in function 'post'
        .../.local/share/nvim/lazy/codeium.nvim/lua/codeium/api.lua:146: in function 'request'
        .../.local/share/nvim/lazy/codeium.nvim/lua/codeium/api.lua:285: in function 'request_completion'
        ...ocal/share/nvim/lazy/codeium.nvim/lua/codeium/source.lua:142: in function 'complete'
        ...bruno/.local/share/nvim/lazy/nvim-cmp/lua/cmp/source.lua:334: in function 'complete'
        ...e/bruno/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:299: in function 'complete'
        ...e/bruno/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:169: in function 'callback'
        ...e/bruno/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:229: in function 'autoindent'
        ...e/bruno/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:161: in function 'on_change'
        ...e/bruno/.local/share/nvim/lazy/nvim-cmp/lua/cmp/init.lua:338: in function 'callback'
        ...local/share/nvim/lazy/nvim-cmp/lua/cmp/utils/autocmd.lua:49: in function 'emit'
        ...local/share/nvim/lazy/nvim-cmp/lua/cmp/utils/autocmd.lua:23: in function <...local/share/nvim/lazy/nvim-cmp/lua/cmp/utils/autocmd.lua:22>
stack traceback:
        [C]: in function 'error'
        .../.local/share/nvim/lazy/plenary.nvim/lua/plenary/job.lua:405: in function '_execute'
        .../.local/share/nvim/lazy/plenary.nvim/lua/plenary/job.lua:448: in function 'start'
        ....local/share/nvim/lazy/plenary.nvim/lua/plenary/curl.lua:288: in function 'post'
        ...o/.local/share/nvim/lazy/codeium.nvim/lua/codeium/io.lua:452: in function 'post'
        .../.local/share/nvim/lazy/codeium.nvim/lua/codeium/api.lua:146: in function 'request'
        .../.local/share/nvim/lazy/codeium.nvim/lua/codeium/api.lua:285: in function 'request_completion'
        ...ocal/share/nvim/lazy/codeium.nvim/lua/codeium/source.lua:142: in function 'complete'
        ...bruno/.local/share/nvim/lazy/nvim-cmp/lua/cmp/source.lua:334: in function 'complete'
        ...e/bruno/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:299: in function 'complete'
        ...e/bruno/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:169: in function 'callback'
        ...e/bruno/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:229: in function 'autoindent'
        ...e/bruno/.local/share/nvim/lazy/nvim-cmp/lua/cmp/core.lua:161: in function 'on_change'
        ...e/bruno/.local/share/nvim/lazy/nvim-cmp/lua/cmp/init.lua:338: in function 'callback'
        ...local/share/nvim/lazy/nvim-cmp/lua/cmp/utils/autocmd.lua:49: in function 'emit'
        ...local/share/nvim/lazy/nvim-cmp/lua/cmp/utils/autocmd.lua:23: in function <...local/share/nvim/lazy/nvim-cmp/lua/cmp/utils/autocmd.lua:22>

[waynebowie99]

I'm getting this same error.

I think my issue is related to @brunolpsousa where I'm getting this on large C# files.

Files that don't work:

• 34KB with ~ 700 lines
• 31KB with ~ 600 lines
• 28KB with ~ 680 lines

Working files

• 29KB with ~ 550 lines

It looks like the issue is tied more to the number of lines than anything in my case. Once the file gets to be around 600 lines it starts to fail in C#.

I'm running on Windows. I can provide any logs as needed

[pqn]

The fix is we probably have to switch the curl to accept data from stdin instead of passing it as an argument.

[amoschristian]

Still happening in Apr 2024 :(

[mitya-murakhouski]

Still happening in Apr 2024 :(

Same thing for me. Getting this error at big yml files, than any completion call on any file raise this error

[Anthony-Fiddes]

Edit: Realizing I still experience the issue if I open the too big file along with a smaller one. The codeium source appears to send other buffers as additional context. That's a big enough issue to where I guess I just have to disable the extension for now.

I was experiencing this issue too. I ended up configuring nvim-cmp per buffer, and excluding codeium on files over 100KB. From above comments, that number should probably be lower, but this works for me at the moment. You can see the change in my config here.

I referred to the nvim-cmp documentation on this page: https://github.com/hrsh7th/nvim-cmp/wiki/Advanced-techniques#disabling-completion-in-certain-contexts-such-as-comments

This obviously doesn't fix the root issue, and it would be nice if codeium worked in large files, but it will keep your neovim usable.

[brunolpsousa]

I made a PR to fix this. Can anyone test it and provide feedback?

[Anthony-Fiddes]

Yup, I can confirm that it works. I just changed Exafunction to brunolpsousa in my config and reinstalled the plugin. Awesome stuff, hope it gets merged soon!

Edit: Let me know if you're looking for any specific feedback. I tried it out with a file over 1MB.

[CNich]

I'm getting this problem too. It doesn't seem to be for big files necessarily, but seems to come when I have many tabs open/leave neovim on for long periods of time. I switched to brunolpsousa and ran Lazy sync but still get this problem, am I doing something wrong?

[brunolpsousa]

I'm getting this problem too. It doesn't seem to be for big files necessarily, but seems to come when I have many tabs open/leave neovim on for long periods of time. I switched to brunolpsousa and ran Lazy sync but still get this problem, am I doing something wrong?

You don't need to use my branch because it has already been merged. Make sure your Lazy config is not locked to a specific commit or any other version control. If it still doesn't work, I think it would be better to open a new issue and provide logs.