Skip to content

L/small changes 12#386

Merged
lorenss-m merged 12 commits into
mainfrom
l/small-changes-12
Apr 1, 2026
Merged

L/small changes 12#386
lorenss-m merged 12 commits into
mainfrom
l/small-changes-12

Conversation

@lorenss-m
Copy link
Copy Markdown
Contributor

@lorenss-m lorenss-m commented Apr 1, 2026

Note

Medium Risk
Moderate risk: changes core evaluation helpers to be async (including subprocess execution) and extends CLI task discovery/import behavior, which could affect existing integrations and task loading in real projects.

Overview
Adds a significantly expanded hud.native evaluation toolkit: graders are now fully async, BashGrader uses async subprocesses with timeout_seconds, and a new Grade.gather() runs multiple graders/subscores in parallel. This also introduces LLMJudgeGrader (rubric-based LLM judging) plus built-in answer comparison/normalization helpers, and exports them via hud.native.

Improves CLI task collection to better match real project layouts: supports importing a directory as a package (running its own discovery), recursively finds **/task.py, and adds project-root sys.path injection to fix cross-module imports; includes extensive new tests for these edge cases.

Updates agent console output to show a tool discovery table at init and a structured per-step tool call/result summary at INFO, and refreshes docs for the new graders API plus minor doc link/typo fixes.

Written by Cursor Bugbot for commit 2d5b407. This will update automatically on new commits. Configure here.

@mintlify
Copy link
Copy Markdown

mintlify Bot commented Apr 1, 2026

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
hud 🟢 Ready View Preview Apr 1, 2026, 1:05 AM

Comment thread hud/cli/utils/collect.py
Comment thread hud/native/graders.py Outdated
Comment thread hud/native/graders.py
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Comment thread hud/native/__init__.py Outdated
Comment thread hud/native/graders.py Outdated
@lorenss-m lorenss-m merged commit c5f1c23 into main Apr 1, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant