Skip to content

fix: add Phractal eval verification metadata#106

Merged
fazxes merged 3 commits intomainfrom
fix/phractal-eval-verifier
Apr 6, 2026
Merged

fix: add Phractal eval verification metadata#106
fazxes merged 3 commits intomainfrom
fix/phractal-eval-verifier

Conversation

@fazxes
Copy link
Copy Markdown
Member

@fazxes fazxes commented Apr 6, 2026

Summary

  • add repo-specific evaluation target metadata so Phractal clones infer a real verification command automatically
  • document and ship the new eval-target module through the installer, ops docs, tracker, and handoff updates
  • add regression coverage for the verifier inference path and the documented metadata contract

Test plan

  • bash scripts/validate-docs.sh
  • make check
  • PYTHONPATH=/Users/no9labs/Developer/Recursive/Nightshift python3 -c "from pathlib import Path; from nightshift.config import merge_config, infer_verify_command; repo=Path('/tmp/nightshift-phractal-inspect'); print(infer_verify_command(repo, merge_config(repo)))"

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f88fbdf53e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread nightshift/eval_targets.py Outdated
if config_path is None:
return None

parser = configparser.ConfigParser()
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Disable interpolation when reading origin URL

Using configparser.ConfigParser() with default interpolation makes parser.get(..., "url") raise InterpolationSyntaxError for valid remotes containing % escapes (for example, URL-encoded paths like %20). In that scenario infer_verify_command() can now throw instead of falling back to normal verifier inference, causing startup/verification inference failures on affected repos. Use interpolation=None (or RawConfigParser) and/or catch configparser.Error around the get() call.

Useful? React with 👍 / 👎.

Comment thread docs/tasks/0099.md Outdated
Comment on lines +1 to +3
---
status: pending

## status: done
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Restore valid frontmatter for completed task 0099

The task frontmatter is malformed (## status: done) and no closing --- delimiter is present, so queue tooling will treat 0099.md as invalid metadata rather than a cleanly completed task. This prevents normal archival logic (which looks for ^status: done) and keeps the queue/reporting in an inconsistent state. Replace the heading-like key with valid YAML (status: done) and close the frontmatter block.

Useful? React with 👍 / 👎.

@fazxes fazxes merged commit e2d235c into main Apr 6, 2026
@fazxes fazxes deleted the fix/phractal-eval-verifier branch April 6, 2026 04:04
fazxes added a commit that referenced this pull request Apr 6, 2026
…le tasks

Pentest findings (both confirmed by source inspection):
- #142 (urgent): $after_task shell injection in run_evaluation() — agent-
  controlled text interpolated into Python -c string; fix via env var
- #143 (urgent): PR title injected raw into builder and pentest prompts —
  sanitize or wrap in XML data label before injection

Closed:
- #77 (wontfix): code-reviewer.md no longer exists; system unified into
  review.md in PR #107; the described issue is absent from the codebase
- #129 (wontfix/superseded): absorbed into #106 (lower-numbered, picked
  first); #106 updated to include the 50-task hard cap explicitly

Queue: 53 pending (same count; +2 urgent, -2 wontfix)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant