Skip to content

chore: sort trace by loop_id #977

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Jul 7, 2025
Merged

chore: sort trace by loop_id #977

merged 10 commits into from
Jul 7, 2025

Conversation

qew21
Copy link
Collaborator

@qew21 qew21 commented Jun 19, 2025

Description

Motivation and Context

How Has This Been Tested?

  • If you are adding a new feature, test on your own test scripts.

Screenshots of Test Results (if appropriate):

  1. Your own tests:

Types of changes

  • Fix bugs
  • Add new feature
  • Update documentation

📚 Documentation preview 📚: https://RDAgent--977.org.readthedocs.build/en/977/

@qew21 qew21 marked this pull request as ready for review June 19, 2025 10:53
@you-n-g you-n-g requested a review from Copilot June 19, 2025 10:57
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors how SOTA experiment statistics are determined by sorting trace files by loop ID, propagates the chosen loop ID into downstream summary logic, and improves error logging with full tracebacks.

  • Updated get_sota_exp_stat to accept an optional sota_loop_id and iterate through sorted feedback traces to find the decision point.
  • Enhanced error handlers in mle_summary.py to include full stack traces and sorted messages by loop ID in summarize_folder.
  • Propagated sota_loop_id into summary and added a new sota_exp_stat_new field.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
rdagent/log/ui/utils.py Changed get_sota_exp_stat signature, replaced max-based selection with sorting by loop ID
rdagent/log/ui/utils.py Updated summary builder to pass through sota_loop_id when calling get_sota_exp_stat
rdagent/log/mle_summary.py Added import traceback; improved exception logging; sorted log messages by loop ID
Comments suppressed due to low confidence (3)

rdagent/log/ui/utils.py:276

  • [nitpick] The field name sota_exp_stat_new may be confusing alongside the existing sota_exp_stat. Consider renaming or documenting it more clearly (e.g., sota_exp_stat_resolved) to improve maintainability.
            v["sota_exp_stat_new"] = get_sota_exp_stat(Path(lf) / k, v.get("sota_loop_id"))

rdagent/log/ui/utils.py:146

  • New behavior branches depending on sota_loop_id and sorted feedback traces. Please add unit tests for both flows: when sota_loop_id is provided and when it must be deduced from trace files.
def get_sota_exp_stat(log_path: Path, sota_loop_id: int | None = None):

rdagent/log/mle_summary.py:44

  • Using print for exception output can make log aggregation harder. Consider using a logger (e.g., logger.exception(...)) to automatically include the traceback in structured logs.
                    print(f"Error in {log_trace_path}: {e}", traceback.format_exc())

qew21 and others added 3 commits June 19, 2025 19:28
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@qew21 qew21 merged commit 3eb0445 into main Jul 7, 2025
9 of 10 checks passed
@qew21 qew21 deleted the ui-stat-update branch July 7, 2025 09:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants