Add production RAG embeddings and clear mypy debt

[2002yy]

Summary

• clear the existing mypy soft-check debt so python -m mypy --explicit-package-bases src is clean locally
• add configurable RAG embedding providers with a default local_hash path and an OpenAI-compatible provider for production retrieval
• wire Chroma upsert/query through the configured embedding provider, with fake-client contract tests
• update README/docs/env templates to distinguish default local-first RAG from explicit production embedding/Chroma configuration

Verification

• python -m pytest -q -> 277 passed
• python -m ruff check .
• python -m mypy --explicit-package-bases src
• detect-secrets scan --disable-plugin KeywordDetector --exclude-files '.*\.(pyc|jpg|png|zip)$' .github README.md docs src tests tools config templates roles changelog assets .env.example
• python tools\package_project_helper.py . NUL 0
• git diff --check

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 00607129b5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Batch OpenAI embedding inputs

When indexing a corpus that produces more than the embeddings endpoint allows in one request (for OpenAI, arrays are limited to 2048 inputs and 300k total tokens), this sends the entire texts list in a single API call via index_documents → Chroma upsert. In that common larger-index scenario the OpenAI-compatible provider rejects the request and the whole indexing operation fails, so this should split the input into bounded batches and concatenate the returned vectors in order.

Useful? React with 👍 / 👎.