Autocompletion very slow for conda environments

[OleBialas]

First, thank you for this project and all the extensive documentation!
I'm just getting started with LSPs, so this may be a dumb question:

Why is the autocompletion so slow when I edit a Python script inside a Conda environment?

When I edit files in my system's base environment, the LSP is very responsive, but with Conda it takes ~5-10 seconds until autocomplete suggestions pop up. This is my configuration:

local lsp = require('lsp-zero')

lsp.preset("recommended")

local lua_opts = lsp.nvim_lua_ls()
require('lspconfig').lua_ls.setup(lua_opts)

require('mason').setup({})
require('mason-lspconfig').setup({
  ensure_installed = {'pyright', 'texlab', 'matlab_ls', 'r_language_server'},
  handlers = {
    lsp.default_setup,
  }
})

lsp.on_attach(function(client, bufnr)
  lsp.default_keymaps({buffer = bufnr})
end)

I assume it somehow takes a lot of time to index all the files in the environment. Is there a way to speed this up?