Add focused multimodal catalog agent example - #226
Merged
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
📝 WalkthroughWalkthroughChangesThe example adds a multimodal Shopify taxonomy agent. It loads and caches listings, ranks taxonomy candidates with text and image-plus-copy calls, verifies the candidate union, computes hierarchical metrics, and exposes evaluation and prediction commands. Catalog agent workflow
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant ShopifyDataset
participant catalog_agent
participant SIEClient
participant JSONOutput
ShopifyDataset->>catalog_agent: load CatalogListing
catalog_agent->>SIEClient: rank text and image-plus-copy candidates
SIEClient-->>catalog_agent: return ranking scores
catalog_agent->>SIEClient: verify candidate union
SIEClient-->>catalog_agent: return selected index and review flag
catalog_agent->>JSONOutput: write CatalogDecision and metrics
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Contributor
Author
|
@coderabbitai full review |
✅ Action performedFull review finished. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (2)
examples/taxonomy-classification/taxonomy_classification/catalog_agent.py (2)
103-147: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick winInclude the dataset revision in the image cache key.
The cache filename at Line 125 uses only
row_idx. IfDATASET_REVISIONchanges later, a stale image from the old revision is silently reused for the samerow_idx, desynchronizing cached bytes from the current row's title, description, and ground truth. Add the revision to the cache key.♻️ Proposed fix
- image_path = cache_dir / f"shopify-train-{row_idx}.image" + image_path = cache_dir / f"shopify-train-{DATASET_REVISION[:12]}-{row_idx}.image"🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@examples/taxonomy-classification/taxonomy_classification/catalog_agent.py` around lines 103 - 147, Update the image cache key in load_shopify_rows to include DATASET_REVISION alongside row_idx, ensuring revisions use distinct cache files while preserving the existing cache read and write behavior.
379-439: 🩺 Stability & Availability | 🔵 Trivial | 🏗️ Heavy liftPersist evaluation progress incrementally.
eval_mainonly writesargs.outputafter the full loop finishes (Line 434-438). A network error or model failure on a later row discards every earlier successful classification, and rerunning repeats potentially expensive, capacity-waited model calls (PROVISION_TIMEOUT_S = 900.0). Write a checkpoint after each row (or every N rows) so a restart can resume.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@examples/taxonomy-classification/taxonomy_classification/catalog_agent.py` around lines 379 - 439, The eval_main classification loop currently persists output only after all listings complete; update it to checkpoint results incrementally after each successful classify_listing call (or a small fixed batch), preserving completed decisions and enabling restart/resume without reprocessing them. Reuse the existing output construction and args.output writing flow, and ensure resumed runs skip already recorded row_idx values.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@examples/taxonomy-classification/taxonomy_classification/catalog_agent.py`:
- Around line 442-450: Update predict_main to pass args.limit to
load_shopify_rows instead of hardcoding limit=1, preserving the default of one
listing from parser.set_defaults(limit=1). Leave the existing
classify_listing(listings[0]) behavior unchanged.
- Around line 226-289: Update verify_candidates to reject an empty candidates
list before constructing candidate_lines or the verifier response schema,
raising a clear ValueError that identifies the missing candidates. Preserve the
existing schema and selection validation for non-empty candidate lists.
---
Nitpick comments:
In `@examples/taxonomy-classification/taxonomy_classification/catalog_agent.py`:
- Around line 103-147: Update the image cache key in load_shopify_rows to
include DATASET_REVISION alongside row_idx, ensuring revisions use distinct
cache files while preserving the existing cache read and write behavior.
- Around line 379-439: The eval_main classification loop currently persists
output only after all listings complete; update it to checkpoint results
incrementally after each successful classify_listing call (or a small fixed
batch), preserving completed decisions and enabling restart/resume without
reprocessing them. Reuse the existing output construction and args.output
writing flow, and ensure resumed runs skip already recorded row_idx values.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: f40bf948-2b8b-4daa-93f2-050ff733c8e5
⛔ Files ignored due to path filters (1)
examples/taxonomy-classification/uv.lockis excluded by!**/*.lock
📒 Files selected for processing (5)
examples/taxonomy-classification/README.mdexamples/taxonomy-classification/pyproject.tomlexamples/taxonomy-classification/results/catalog-agent-summary.jsonexamples/taxonomy-classification/taxonomy_classification/catalog_agent.pyexamples/taxonomy-classification/tests/test_catalog_agent.py
svonava
marked this pull request as ready for review
August 3, 2026 05:32
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
examples/taxonomy-classification/tests/test_catalog_agent.py (1)
13-13: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueDrop the redundant module alias.
import taxonomy_classification.catalog_agent as catalog_agentaliases the submodule to its own name. Usefrom taxonomy_classification import catalog_agentinstead.♻️ Proposed fix
-import taxonomy_classification.catalog_agent as catalog_agent +from taxonomy_classification import catalog_agent🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@examples/taxonomy-classification/tests/test_catalog_agent.py` at line 13, Update the catalog_agent import in test_catalog_agent.py to use the package-level from taxonomy_classification import catalog_agent form, removing the redundant self-alias while preserving all existing references.Source: Linters/SAST tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@examples/taxonomy-classification/taxonomy_classification/catalog_agent.py`:
- Around line 425-444: Update _write_evaluation_output so temporary_path is
assigned immediately after NamedTemporaryFile creates the file, before json.dump
and temporary.write execute. Keep the existing finally cleanup and atomic
replace behavior unchanged.
---
Nitpick comments:
In `@examples/taxonomy-classification/tests/test_catalog_agent.py`:
- Line 13: Update the catalog_agent import in test_catalog_agent.py to use the
package-level from taxonomy_classification import catalog_agent form, removing
the redundant self-alias while preserving all existing references.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 67140d46-2165-44f9-9c66-70d34a5ddba6
📒 Files selected for processing (2)
examples/taxonomy-classification/taxonomy_classification/catalog_agent.pyexamples/taxonomy-classification/tests/test_catalog_agent.py
svonava
added
the
coderabbit-direct
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Measured result
The current recorded run moves exact-path matches from 42/100 for copy alone to 54/100 for the agent. Macro hierarchical F1 moves from 0.6710 to 0.7446. These values came from real model outputs.
Validation
Before merge
Summary by CodeRabbit
New Features
Documentation
Tests