Feat: Add final summary builder with structured output

[svenaric]

Overview: This PR introduces a new function to generate a structured final summary by combining narrative text and numeric scores.

Changes

• Implemented build_final_summary in backend/app/services/summary/summary_engine.py to merge NLG outputs with agent scores.
• The function produces a JSON object with overall_summary, scores, weaknesses, and strengths.
• Ensured consistent field ordering for improved readability of the final report.
• This enhancement is crucial for generating comprehensive and well-organized ChainReports.

Summary by CodeRabbit

Release Notes

• Refactor

  • Summary reports now feature a restructured format with improved organization. Metrics are labeled with human-friendly names, and strengths and weaknesses are explicitly categorized for better clarity and analysis. Scores are rounded to two decimal places for consistency.

• Tests

  • Updated test coverage to validate the new summary structure, scoring system, and weakness/strength categorization.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

The build_final_summary method in ReportSummaryEngine now returns a structured dictionary containing overall_summary text, transformed scores with human-friendly labels, weaknesses (scores < 5.0), and strengths (scores >= 7.0) instead of a plain string. Tests updated to validate the new dictionary structure and renamed score keys.

Changes

Cohort / File(s)	Summary
Return Type Refactoring backend/app/services/summary/report_summary_engine.py	Modified build_final_summary method signature to return Dict[str, Any] instead of str. Method now aggregates NLG outputs into overall_summary, maps scores to human-friendly labels, computes weaknesses (scores < 5.0) and strengths (scores >= 7.0), and returns structured dictionary with keys: overall_summary, scores, weaknesses, strengths. Scores rounded to two decimals.
Test Updates backend/app/services/summary/tests/test_report_summary_engine.py	Updated test expectations from string assertions to dictionary structure validation. Added code_audit to NLG outputs. Extended scores input with code_maturity and audit_confidence fields; adjusted sentiment_health from 8.2 to 4.2. Assertions now verify dictionary keys, overall_summary content consolidation, renamed score mappings (Tokenomics Strength, Sentiment Health, Code Maturity, Audit Confidence), and weaknesses/strengths categorization.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

• Pay attention to the dictionary structure returned by build_final_summary and ensure all callers are updated to handle the new Dict[str, Any] return type rather than a string
• Verify that score rounding to two decimals is applied correctly and that the threshold logic for weaknesses (< 5.0) and strengths (>= 7.0) is correct
• Check that the human-friendly score key mappings match expected conventions across the codebase

Possibly related PRs

• Feat: Add Summary Engine Base Class #57: Directly relates as the main PR changes ReportSummaryEngine.build_final_summary return type from str to Dict[str, Any], which impacts the base class interface contract
• feat: Implement scoring functions in summary engine #58: Both PRs modify ReportSummaryEngine; #58 adds scoring generation and string-based summary, while this PR transforms that output into a structured dictionary with renamed score keys

Suggested reviewers

• felixjordandev
• klingonaston

Poem

🐰 The summary now wears a structured vest,
With strengths and weaknesses expressed,
No longer strings, but dicts so neat,
With rounded scores—a feat complete!
Hop hop, the data's now refined,
Organized in a thoughtful mind! 📊

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: adding a new build_final_summary function that returns structured dictionary output instead of a simple string.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch feat/add-final-summary-builder

Tip

📝 Customizable high-level summaries are now available in beta!

You can now customize how CodeRabbit generates the high-level summary in your pull requests — including its content, structure, tone, and formatting.

• Provide your own instructions using the high_level_summary_instructions setting.
• Format the summary however you like (bullet lists, tables, multi-section layouts, contributor stats, etc.).
• Use high_level_summary_in_walkthrough to move the summary from the description to the walkthrough section.

Example instruction:

"Divide the high-level summary into five sections:

1. 📝 Description — Summarize the main change in 50–60 words, explaining what was done.
2. 📓 References — List relevant issues, discussions, documentation, or related PRs.
3. 📦 Dependencies & Requirements — Mention any new/updated dependencies, environment variable changes, or configuration updates.
4. 📊 Contributor Summary — Include a Markdown table showing contributions:
   | Contributor | Lines Added | Lines Removed | Files Changed |
5. ✔️ Additional Notes — Add any extra reviewer context.
   Keep each section concise (under 200 words) and use bullet or numbered lists for clarity."

Note: This feature is currently in beta for Pro-tier users, and pricing will be announced later.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (2)

backend/app/services/summary/report_summary_engine.py (2)

54-64: Consider extracting the repeated transformation logic.

The weaknesses and strengths computation logic is correct. However, the transformation score_name.replace('_', ' ').title() is repeated three times (lines 55, 61, and 68). Consider extracting it into a helper method for better maintainability.

Example refactor:

+    def _format_score_name(self, score_name: str) -> str:
+        return score_name.replace('_', ' ').title()
+
     def build_final_summary(self, nlg_outputs: Dict[str, str], scores: Dict[str, float]) -> Dict[str, Any]:
         overall_summary_parts = []
         for agent, output in nlg_outputs.items():
             overall_summary_parts.append(f"{agent.replace('_', ' ').title()} Insights: {output}")
         overall_summary = "\n\n".join(overall_summary_parts)
 
         weaknesses = [
-            score_name.replace('_', ' ').title()
+            self._format_score_name(score_name)
             for score_name, score_value in scores.items()
             if score_value < 5.0
         ]
 
         strengths = [
-            score_name.replace('_', ' ').title()
+            self._format_score_name(score_name)
             for score_name, score_value in scores.items()
             if score_value >= 7.0
         ]
 
         final_summary = {
             "overall_summary": overall_summary,
-            "scores": {score_name.replace('_', ' ').title(): round(score_value, 2) for score_name, score_value in scores.items()},
+            "scores": {self._format_score_name(score_name): round(score_value, 2) for score_name, score_value in scores.items()},
             "weaknesses": weaknesses,
             "strengths": strengths,
         }
         return final_summary

---

66-72: Well-structured output format.

The final_summary dictionary is well-organized with all required fields. The rounding of scores to 2 decimal places improves readability. The dictionary field order (overall_summary, scores, weaknesses, strengths) is consistent as mentioned in the PR objectives.

Note: The order of items within the scores dictionary depends on the insertion order of the input scores parameter. If a specific ordering is required (e.g., alphabetical or by importance), consider using sorted() on scores.items().

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 558d837 and 8e24f33.

⛔ Files ignored due to path filters (3)

• backend/app/services/summary/__pycache__/__init__.cpython-313.pyc is excluded by !**/*.pyc
• backend/app/services/summary/__pycache__/report_summary_engine.cpython-313.pyc is excluded by !**/*.pyc
• backend/app/services/summary/tests/__pycache__/test_report_summary_engine.cpython-313-pytest-8.4.2.pyc is excluded by !**/*.pyc

📒 Files selected for processing (2)

• backend/app/services/summary/report_summary_engine.py (1 hunks)
• backend/app/services/summary/tests/test_report_summary_engine.py (1 hunks)

🧰 Additional context used

🧬 Code graph analysis (1)

backend/app/services/summary/tests/test_report_summary_engine.py (1)

backend/app/services/summary/report_summary_engine.py (1)

• build_final_summary (48-72)

🔇 Additional comments (4)

backend/app/services/summary/report_summary_engine.py (1)

48-52: LGTM! Clear and effective implementation.

The overall_summary construction correctly aggregates agent insights with clear labeling and formatting. The use of dictionary iteration order is safe in Python 3.7+.

backend/app/services/summary/tests/test_report_summary_engine.py (3)

40-51: Excellent test data design.

The test input data is well-chosen to validate both weaknesses (scores < 5.0) and strengths (scores >= 7.0), ensuring comprehensive coverage of the new dictionary structure.

---

54-63: Thorough validation of dictionary structure and content.

The assertions correctly verify that the summary is a dictionary with all required keys, and that the overall_summary includes insights from all three agents. The use of substring matching is appropriate for this test.

---

65-81: Comprehensive validation of scores, weaknesses, and strengths.

The test thoroughly validates:

• Correct transformation of score keys to Title Case with spaces
• Accurate identification of weaknesses (scores < 5.0)
• Accurate identification of strengths (scores >= 7.0)
• Both presence and count of items in each category

This provides strong confidence in the implementation correctness.

[felixjordandev]

the new build_final_summary function really ties everything together, especially with the structured JSON output. Merging narrative and scores like this is gonna improve report clarity a lot. I'm merging this now.