Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 9 additions & 8 deletions .github/scripts/classify-agent-authorship.rb
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,9 @@

required_patterns = {
"co_author" => /^Co-Authored-By:\s*Maestro\s+<maestro@evalops\.dev>\s*$/i,
"version" => /^Maestro-Version:\s*\S.+$/i,
"prompt_id" => /^Maestro-Prompt-Id:\s*\S.+$/i,
"approvals_id" => /^Maestro-Approvals-Id:\s*\S.+$/i,
"version" => /^Maestro-Version:\s*\S.*$/i,
"prompt_id" => /^Maestro-Prompt-Id:\s*\S.*$/i,
"approvals_id" => /^Maestro-Approvals-Id:\s*\S.*$/i,
}

marker_pattern = /
Expand All @@ -38,14 +38,14 @@
/ix

agent_commits = 0
human_commits = 0
untrailered_commits = 0
incomplete_commits = 0

messages.each do |message|
has_marker = message.lines.any? { |line| line.match?(marker_pattern) }

unless has_marker
human_commits += 1
untrailered_commits += 1
next
end

Expand All @@ -57,19 +57,20 @@
end

label =
if agent_commits.positive? && human_commits.positive?
if agent_commits.positive? && untrailered_commits.positive?
"mixed-authorship"
elsif agent_commits.positive?
"agent-authored"
else
"human-authored"
"agent-assisted"
end

outputs = {
"label" => label,
"total_commits" => messages.length,
"agent_commits" => agent_commits,
"human_commits" => human_commits,
"untrailered_commits" => untrailered_commits,
"human_commits" => untrailered_commits,
"incomplete_agent_commits" => incomplete_commits,
}

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "Agent authorship labels",
"description": "Apply agent-authored, human-authored, or mixed-authorship labels to pull requests based on Maestro commit trailers.",
"description": "Apply agent-authored, agent-assisted, or mixed-authorship labels to pull requests based on Maestro commit trailers.",
"iconName": "octicon tag",
"categories": [
"Automation",
Expand Down
27 changes: 19 additions & 8 deletions .github/workflows/agent-authorship-label.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,9 +82,9 @@ jobs:
fi
}

ensure_label "agent-authored" "6f42c1" "All PR commits carry Maestro authorship trailers"
ensure_label "human-authored" "0e8a16" "No PR commits carry Maestro authorship trailers"
ensure_label "mixed-authorship" "fbca04" "Some PR commits carry Maestro authorship trailers"
ensure_label "agent-authored" "6f42c1" "All PR commits carry explicit Maestro authorship trailers"
ensure_label "agent-assisted" "1d76db" "PR commits are assumed LLM-assisted but do not carry explicit Maestro trailers"
ensure_label "mixed-authorship" "fbca04" "Some PR commits carry explicit Maestro trailers and some are untrailered"

- name: Apply authorship label
shell: bash
Expand All @@ -95,13 +95,24 @@ jobs:
run: |
set -euo pipefail

for label in agent-authored human-authored mixed-authorship; do
gh api --method DELETE \
"repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/labels/${label}" >/dev/null 2>&1 || true
current_labels="$(gh api "repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/labels" --jq '.[].name')"

has_label() {
local label="$1"
grep -Fxq "${label}" <<<"${current_labels}"
}

for label in agent-authored agent-assisted mixed-authorship human-authored; do
if [ "${label}" != "${AUTHORSHIP_LABEL}" ] && has_label "${label}"; then
gh api --method DELETE \
"repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/labels/${label}" >/dev/null
fi
Comment thread
haasonsaas marked this conversation as resolved.
done

gh api --method POST "repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/labels" \
-f "labels[]=${AUTHORSHIP_LABEL}" >/dev/null
if ! has_label "${AUTHORSHIP_LABEL}"; then
gh api --method POST "repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/labels" \
-f "labels[]=${AUTHORSHIP_LABEL}" >/dev/null
fi

echo "Applied ${AUTHORSHIP_LABEL} to #${PR_NUMBER}."

Expand Down
10 changes: 6 additions & 4 deletions profile/AGENT_AUTHORSHIP.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,12 +42,12 @@ each PR:

| Label | Meaning |
|---|---|
| `agent-authored` | Every commit in the PR carries Maestro authorship metadata. |
| `human-authored` | No commit in the PR carries Maestro authorship metadata. |
| `mixed-authorship` | Some commits carry Maestro metadata and some do not. |
| `agent-authored` | Every commit in the PR carries explicit Maestro authorship metadata. |
| `agent-assisted` | No commit in the PR carries explicit Maestro metadata; EvalOps treats untrailered PR code as LLM-assisted by default. |
| `mixed-authorship` | Some commits carry explicit Maestro metadata and some are untrailered. |

The labels are a GitHub UI affordance. The commit trailers remain the source of
truth because they travel with the git history.
truth for explicit Maestro provenance because they travel with the git history.

## Reusable Workflow

Expand All @@ -73,6 +73,8 @@ jobs:

The workflow creates the three labels if they are missing, removes stale
authorship labels, and applies the label that matches the current PR commit set.
It only mutates labels when the desired label set changed, so repeated
`synchronize` events do not remove and re-add the same label.

## Audit Indexing

Expand Down
Loading