perf(rag): defer provider config imports

[dexhunter]

Disclosure: this PR was prepared with a coding agent. GitHub does not allow external contributors to apply labels in this repository, so please add the required llm-generated label.

Summary

• defer RAG provider config imports until the RAG configuration surface is used
• keep crewai.rag.config = ... behavior and existing patchable client factory seams intact
• add regression coverage for top-level imports, RAG module assignment, provider types, and KnowledgeStorage construction

Why

import crewai currently loads the Qdrant and FastEmbed dependency chain even when a process never uses RAG, knowledge, or memory storage. This adds cold-start cost to CLI commands, short-lived workers, serverless deployments, and test processes.

The eager chain enters through two module-level import sites:

• crewai.rag.__init__ imports RagConfigType and set_rag_config for the package assignment hook.
• knowledge_storage.py imports provider config and factory helpers that are only used while initializing a storage client.

Import-time work has come up repeatedly in #5510, #4226, #3991, and the merged framework-overhead work in #5187.

Performance

Metric	Before	After	Change
cold import crewai	2070.977 ms	1409.077 ms	-31.96% (1.47x)

Measured on current main (860817c) with Python 3.13.13 on Linux after uv sync --all-groups --all-extras. Each value is the median of seven fresh subprocesses pinned to the same dedicated CPU. The patched import also verifies that qdrant_client is absent from sys.modules after import crewai.

Implementation

• Move type-only provider imports under TYPE_CHECKING.
• Import set_rag_config only when a non-module value is assigned to crewai.rag.config.
• Allow importlib to register the real crewai.rag.config submodule without routing that module object through the configuration setter.
• Keep module-level create_client and get_rag_client wrappers so existing tests and integrations can continue to patch those names.
• Import ChromaDBConfig only when KnowledgeStorage actually initializes an embedded client.

Validation

• uv run pytest lib/crewai/tests/test_imports.py lib/crewai/tests/rag lib/crewai/tests/knowledge lib/crewai/tests/utilities/test_planning_handler.py -q (245 passed, 1 skipped)
• uvx ruff check and uvx ruff format --check on all changed files
• uv run mypy on both changed source modules
• git diff --check
• public import surface and RAG assignment behavior preserved by regression tests

Autoresearch

Supplementary autoresearch: 20-step public dashboard.

[coderabbitai]

📝 Walkthrough

Walkthrough

Eager RAG provider and configuration imports are replaced with lazy imports in knowledge storage and crewai.rag. Client helpers and config assignment handling are updated, with regression tests covering import-time loading, annotations, provider types, and KnowledgeStorage instantiation.

Changes

Lazy RAG Import Loading

Layer / File(s)	Summary
Lazy client/config imports in knowledge storage lib/crewai/src/crewai/knowledge/storage/knowledge_storage.py	Adds on-demand RAG client helpers and locally imports ChromaDBConfig during client initialization.
Lazy config assignment in crewai.rag lib/crewai/src/crewai/rag/__init__.py	Defers configuration imports and routes config assignments through the registered module or lazily imported set_rag_config.
Regression tests for lazy import behavior lib/crewai/tests/test_imports.py	Tests deferred qdrant loading, RAG assignment behavior, runtime annotations, provider config types, and KnowledgeStorage instantiation.

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant RagModule as crewai.rag
  participant SetRagConfig as set_rag_config

  Caller->>RagModule: assign rag_mod.config
  alt registered config submodule
    RagModule->>RagModule: store config module
  else other config value
    RagModule->>SetRagConfig: import and call set_rag_config
  end

Loading

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title is concise and accurately summarizes the main change: deferring RAG provider config imports.
Description check	✅ Passed	The description clearly matches the changeset and explains the import deferral, behavior preservation, and added tests.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[dexhunter]

Rebased onto current main. Focused validation on the rebased head: 245 passed, 1 skipped across import, RAG, knowledge, and planning tests; Ruff check/format and git diff check passed.

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

lib/crewai/tests/test_imports.py (1)

49-54: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Prefer pytest.raises over try/except/else for exception assertions.

Using pytest.raises(AttributeError) is more idiomatic and concise. This is a minor readability improvement that can be deferred.

♻️ Optional refactor

-    # Non-config attributes are rejected.
-    try:
-        rag_mod.some_random_attribute = 1
-    except AttributeError:
-        pass
-    else:
-        raise AssertionError("crewai.rag accepted an arbitrary attribute")
+    # Non-config attributes are rejected.
+    import pytest
+
+    with pytest.raises(AttributeError):
+        rag_mod.some_random_attribute = 1

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/crewai/tests/test_imports.py` around lines 49 - 54, Replace the manual
try/except/else assertion in the import test with a
pytest.raises(AttributeError) context manager around the assignment to
rag_mod.some_random_attribute, preserving the existing validation message or
assertion as appropriate.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/crewai/tests/test_imports.py`:
- Around line 32-34: Add a finite timeout argument to the subprocess.run call in
the import test, and handle subprocess.TimeoutExpired so the test fails clearly
rather than hanging indefinitely. Keep the existing sys.executable invocation
and captured output behavior unchanged.

---

Nitpick comments:
In `@lib/crewai/tests/test_imports.py`:
- Around line 49-54: Replace the manual try/except/else assertion in the import
test with a pytest.raises(AttributeError) context manager around the assignment
to rag_mod.some_random_attribute, preserving the existing validation message or
assertion as appropriate.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: acbd50c8-80f1-41b6-b81a-e3009d6a3994

📥 Commits

Reviewing files that changed from the base of the PR and between 52d289a and 79f3157.

📒 Files selected for processing (3)

• lib/crewai/src/crewai/knowledge/storage/knowledge_storage.py
• lib/crewai/src/crewai/rag/__init__.py
• lib/crewai/tests/test_imports.py

🚧 Files skipped from review as they are similar to previous changes (2)

• lib/crewai/src/crewai/knowledge/storage/knowledge_storage.py
• lib/crewai/src/crewai/rag/init.py