ReSum Inference Framework - #180
Conversation
WxxShirley
commented
Sep 29, 2025
WxxShirley
commented
- ReSum Inference
- ReAct Inference
- Corresponding README for quick start
There was a problem hiding this comment.
Pull Request Overview
This PR introduces the ReSum inference framework for web agents, which enables unlimited exploration through periodic conversation summarization instead of appending all interaction history. The framework provides both ReSum and ReAct inference methods with corresponding shell scripts for quick deployment.
Key changes:
- Implementation of ReSum inference paradigm with conversation summarization capabilities
- Complete web agent toolkit including search and visit tools with API integrations
- Evaluation framework with multi-language judge prompts and statistical analysis
Reviewed Changes
Copilot reviewed 12 out of 15 changed files in this pull request and generated 11 comments.
Show a summary per file
| File | Description |
|---|---|
| tool_visit.py | Implements webpage visiting and content summarization functionality |
| tool_search.py | Provides batched Google search capabilities with concurrent execution |
| summary_utils.py | Contains ReSum server communication and conversation summarization logic |
| run_resum.sh | Shell script for ReSum inference deployment with model server management |
| run_react.sh | Shell script for ReAct inference deployment with simplified configuration |
| react_agent.py | Multi-turn ReAct agent with ReSum integration and token management |
| prompt.py | System prompts and templates for various inference components |
| main.py | Main inference orchestration with concurrent processing |
| judge_prompt.py | Evaluation prompts for different datasets and languages |
| evaluate.py | Comprehensive evaluation framework with statistical analysis |
| example.jsonl | Sample evaluation data to prevent test data leakage |
| README.md | Updated documentation with quick start guide and setup instructions |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| SUMMARY_MODEL_NAME = os.environ.get("SUMMARY_MODEL_NAME", "") | ||
|
|
||
|
|
||
| @staticmethod |
There was a problem hiding this comment.
The @staticmethod decorator is placed outside of any class definition, making it invalid. This function should either be moved inside the Visit class or the decorator should be removed.
| @staticmethod |
| try: | ||
| url = params["url"] | ||
| goal = params["goal"] | ||
| except: |
There was a problem hiding this comment.
Using bare except: clause is discouraged as it catches all exceptions including system exits. Use except Exception: instead to catch only regular exceptions.
| except: | |
| except Exception: |
| if content: | ||
| try: | ||
| json.loads(content) | ||
| except: |
There was a problem hiding this comment.
Using bare except: clause is discouraged. Specify the exception type, likely except json.JSONDecodeError: for JSON parsing errors.
| except: | |
| except json.JSONDecodeError: |
| try: | ||
| raw = json.loads(raw) | ||
| break | ||
| except: |
There was a problem hiding this comment.
Using bare except: clause is discouraged. Specify the exception type, likely except json.JSONDecodeError: for JSON parsing errors.
| except: | |
| except json.JSONDecodeError: |
| from concurrent.futures import ThreadPoolExecutor | ||
| from typing import List, Union | ||
| import requests | ||
| from qwen_agent.tools.base import BaseTool, register_tool |
There was a problem hiding this comment.
The import from qwen_agent.tools.base import BaseTool, register_tool is duplicated on lines 1 and 6. Remove the duplicate import.
| from qwen_agent.tools.base import BaseTool, register_tool |
| assert GOOGLE_SEARCH_KEY is not None, "Please set the GOOGLE_SEARCH_KEY environment variable." | ||
| try: | ||
| query = params["query"] | ||
| except: |
There was a problem hiding this comment.
Using bare except: clause is discouraged. Use except Exception: or more specific exception types instead.
| except: | |
| except (KeyError, TypeError): |
| content = re.sub(pattern, '', content, flags=re.DOTALL).strip() | ||
| try: | ||
| content = content.split("<summary>")[1].split("</summary>")[0] | ||
| except: |
There was a problem hiding this comment.
Using bare except: clause is discouraged. Specify the exception type, likely except (IndexError, ValueError): for string splitting operations.
| except: | |
| except (IndexError, ValueError): |
| {"role": "user", "content": new_observation} | ||
| ] | ||
| full_trajectory.append({"role": "user", "content": new_observation}) | ||
| token_count = self.count_tokens(messages) |
There was a problem hiding this comment.
The variable parse_retry_times is initialized to 2 and compared against 3, but the loop condition suggests it should start at 0. The initialization and loop logic appear inconsistent.
| # TODO: Replace with your own DashScope API key | ||
| import dashscope | ||
| dashscope.api_key = 'YOUR_DASHSCOPE_API_KEY' |
There was a problem hiding this comment.
API key is hardcoded as a placeholder string. This should be loaded from environment variables or configuration files to avoid accidentally committing real API keys.
| # TODO: Replace with your own DashScope API key | |
| import dashscope | |
| dashscope.api_key = 'YOUR_DASHSCOPE_API_KEY' | |
| # Load DashScope API key from environment variable for security | |
| import dashscope | |
| dashscope_api_key = os.environ.get('DASHSCOPE_API_KEY') | |
| if not dashscope_api_key: | |
| raise RuntimeError("DASHSCOPE_API_KEY environment variable not set. Please set it to your DashScope API key.") | |
| dashscope.api_key = dashscope_api_key |
| answer_lengths, traj_lengths = [], [] | ||
|
|
||
| try: | ||
| tokenizer = AutoTokenizer.from_pretrained("/path/to/your/Qwen2.5-72B-Instruct") |
There was a problem hiding this comment.
The tokenizer path is hardcoded with a placeholder value. This should be configurable through environment variables or command-line arguments.
