Development Branch #220
Open
Development Branch #220
+8,066
−1,447
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Improve decontextualization by passing along previous results as part of the request.
More small changes to conversational history
- also provides better abstraction - adds a number of tests
fix item details UI
- Resolved conflict in code/retrieval/retriever.py - Kept dynamic imports with auto-installation from main - Fixed variable references (db_type instead of self.db_type)
misc fixes and UI for ensembles
- Added AccompanimentHandler for finding complementary items (wines, sides, sauces) - Added SubstitutionHandler for recipe ingredient substitutions - Updated tools.xml with handler definitions for new tools - Added UI support for substitution suggestions in managed-event-source.js - Integrated new tools with the general handler-based routing approach 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Added podcast_scraper.py (renamed from npr_podcast_scraper_complete.py) for scraping podcast data - Added process_npr_rss_by_org.py for processing NPR RSS feeds by organization - Cherry-picked only the NPR-specific files from the PR to avoid reverting other changes 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Added modular testing framework with base_test_runner.py - Added specific test runners for end-to-end, site retrieval, and query retrieval tests - Added comprehensive test runner scripts (run_all_tests.sh/.bat, run_tests_comprehensive.sh) - Updated run_tests.py to be a test dispatcher that routes to appropriate test runners - Added JSON test configuration files for different test types - Updated testing README with comprehensive documentation - Preserved all existing functionality while adding new testing capabilities 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Moved retrieve_and_rank_for_query out as standalone method - Replaced get_vector_db_client with search abstraction - Fixed handling of 4-tuple search results - Skip tool selection when generate_mode is summarize or generate - Applied logging improvements from PR #204 to decontextualize.py
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Updated documentation and release notes
…s.py - Kept the new _get_item_by_url() method from development (URL-based retrieval feature) - Preserved _send_no_items_found_message() method from both branches - Maintained all functionality from both branches
There was a problem hiding this comment.
Pull Request Overview
This pull request consolidates many changes by merging previously unmerged branches into the development branch and resolving conflicts. The changes include adjustments and new features for event streaming and chat interfaces in JavaScript, extensive documentation updates, improvements to the retrieval system configuration, and numerous additions to the testing and utility scripts.
Reviewed Changes
Copilot reviewed 62 out of 67 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| static/streaming.js | Update parsing of “details” property and change from a bulleted list to a table for array formatting. |
| static/managed-event-source.js | Introduce new message types and debug message collection, as well as enhancements to tool selection flow. |
| docs/* | New and revised documentation on tools, retrieval, headers, and configuration guidelines. |
| code/utils/json_utils.py | New JSON trimming and merging functions with recursive merge logic. |
| code/tools/* | Refactoring of Qdrant loading and other tool modules, including asynchronous adaptations and configuration adjustments. |
| code/testing/* | Numerous new test runners and scripts for site retrieval, query retrieval, end-to-end testing and comprehensive test suite improvements. |
Comments suppressed due to low confidence (1)
code/tools/db_load.py:922
- Ensure that the configuration migration from 'preferred_retrieval_endpoint' to 'write_endpoint' is clearly documented in the migration guide to help developers adjust their setups.
endpoint_name = database or CONFIG.write_endpoint
| elif val1 == val2: | ||
| merged[key] = val1 | ||
| else: | ||
| # Create array with both values |
There was a problem hiding this comment.
Consider adding a comment to document the rationale for merging non-identical scalar values into an array, ensuring that future maintainers understand this intentional behavior.
Suggested change
| # Create array with both values | |
| # Create array with both values to preserve both conflicting scalar values. | |
| # This ensures no data is lost during the merge process. |
Copilot uses AI. Check for mistakes.
| stats['failed_conversion'] += 1 | ||
|
|
||
| # Be respectful with requests | ||
| time.sleep(2) |
There was a problem hiding this comment.
[nitpick] Consider using an asynchronous sleep (e.g., await asyncio.sleep(2)) if the surrounding context is asynchronous to avoid blocking the event loop; if synchronous delay is intended, please document this decision.
Suggested change
| time.sleep(2) | |
| await asyncio.sleep(2) |
Copilot uses AI. Check for mistakes.
…variables in our .env.template
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pulled all the changes from earlier unmerged branches into this. Resolved conflicts. We can now do testing on this and merge this with main.