Fix/db connection pool limit

[Prajna1999]

Summary

Target issue is #PLEASE_TYPE_ISSUE_NUMBER
Explain the motivation for making this change. What existing problem does the pull request solve?

Checklist

Before submitting a pull request, please ensure that you mark these task.

• Ran fastapi run --reload app/main.py or docker compose up in the repository root and test.
• If you've fixed a bug or added code that is tested and has test cases.

Notes

Please add here if any other information is required for the reviewer.

Summary by CodeRabbit

• Chores
  • Refactored internal architecture for job processing and callback handling to optimize system operations and improve code maintainability.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: dd77b6dd-1ba2-4d76-81a4-15cdef115a7e

📥 Commits

Reviewing files that changed from the base of the PR and between 60dd566 and f2a9d3a.

📒 Files selected for processing (3)

• backend/app/services/collections/providers/base.py
• backend/app/services/llm/chain/executor.py
• backend/app/services/llm/jobs.py

---

📝 Walkthrough

Walkthrough

Refactors DB session management and callback ordering in LLM job/chain execution; removes DocumentCrud dependency from collection provider BaseProvider.create() in favor of pre-fetched docs_batches: list[list[Document]].

Changes

Cohort / File(s)	Summary
Collection provider signature backend/app/services/collections/providers/base.py	Replaced document_crud: DocumentCrud parameter with docs_batches: list[list[Document]]; removed DocumentCrud import and updated docstring to require pre-fetched batches.
LLM execution & job lifecycle backend/app/services/llm/chain/executor.py, backend/app/services/llm/jobs.py	Moved send_callback(...) calls to run outside/before DB Session(engine) contexts; on success/failure callbacks are sent prior to opening DB sessions, and DB updates use scoped with Session(engine) blocks. Removed explicit db.flush() / db.commit() after job creation and simplified JobCrud(...).update(...) usage.

Sequence Diagram(s)

sequenceDiagram
    participant Executor
    participant CallbackSvc
    participant DB as "Database\n(Session/JobCrud)"
    Executor->>CallbackSvc: build callback_response
    Executor->>CallbackSvc: send_callback(callback_response)
    Note right of CallbackSvc: callback sent before DB work
    Executor->>DB: with Session(engine): update job status / persist results
    DB-->>Executor: update confirmation
    Executor-->>Client: return callback_response.model_dump()

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

I nibble at sessions, tidy and bright,
Callbacks hop forward, then DBs take flight,
Documents pre-gathered, no Crud in my paws,
Flushed out the fuss with minimal cause,
A rabbit's small cheer for cleaner code laws 🐰

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/db-connection-pool-limit

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!