Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .bazelignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
.agents/skills
projects/ci_platform/main/js
projects/mcp_cordis/node_modules
projects/rules_template
projects/rules_binary_toolchain
projects/rules_docs
Expand Down
13 changes: 13 additions & 0 deletions .codex/config.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[mcp_servers.mcp_cordis]
command = "bash"
args = [
"-c",
'''
set -eu
root="$(git rev-parse --show-toplevel)"
exec bash "$root/projects/mcp_cordis/cmd/mcp_cordis/launch.sh"
''',
]
required = true
startup_timeout_sec = 120
tool_timeout_sec = 300
2 changes: 1 addition & 1 deletion .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -101,4 +101,4 @@
*.dat filter=lfs diff=lfs merge=lfs -text
*.csv filter=lfs diff=lfs merge=lfs -text
*.icc filter=lfs diff=lfs merge=lfs -text
projects/rules_promptfoo/patches/promptfoo@0.122.2.patch whitespace=-space-before-tab
*.patch whitespace=-blank-at-eol,-space-before-tab
1 change: 1 addition & 0 deletions MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ include("//tools/bao:include.MODULE.bazel")
include("//tools/shfmt:include.MODULE.bazel")

# Projects
include("//projects/mcp_cordis:include.MODULE.bazel")
include("//projects/android_launcher:include.MODULE.bazel")
include("//projects/nexus_security_plugin:include.MODULE.bazel")
include("//projects/ansible_collection:include.MODULE.bazel")
Expand Down
1 change: 1 addition & 0 deletions projects/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ al_release_binary(
"xray_manager",
"cgit",
"agents",
"mcp_cordis",
"alwaldend.com",
"ansible_collection",
"bazel_registry",
Expand Down
16 changes: 16 additions & 0 deletions projects/agents/skills/bazel-agent/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,22 @@ Bazelisk-managed `bazel` from `PATH`, and replaces itself with that process.
The replacement preserves direct signal delivery and Bazel's exit status. The
repository `.bazeliskrc` pins the Bazel version and archive hash.

For Bazel commands that support multiple targets, such as `build` and `test`,
batch compatible targets into one invocation when they use the same options:

```sh
bazel_agent build //path/to:first //path/to:second
bazel_agent test //path/to:first_test //path/to:second_test
```

This is especially important because agent invocations use batch mode and each
separate command pays Bazel startup and analysis overhead. Do not batch a
single-target command such as `run`. Otherwise keep invocations separate only
when they require different commands or options, have a real ordering
dependency, need failure isolation for diagnosis, or would create unsafe
resource contention. Do not run separate compatible invocations merely to
parallelize work that Bazel already schedules internally.

Use `repo-bazel` in addition to this skill when changing BUILD files, Starlark,
Bzlmod dependencies, toolchains, or the build graph.

Expand Down
2 changes: 1 addition & 1 deletion projects/agents/skills/bazel-agent/evals/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ title: Bazel agent evaluations

This suite describes the required behavior for invoking and troubleshooting
the repository's Bazel runner. Its offline Bazel target validates the
Promptfoo configuration, referenced case, and skill staging without making a
Promptfoo configuration, referenced cases, and skill staging without making a
model call. The configuration names `openai:codex-sdk` in read-only mode, but
this suite is intended only for offline validation and does not invoke it.

Expand Down
24 changes: 24 additions & 0 deletions projects/agents/skills/bazel-agent/evals/cases.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,27 @@
Bazel's exit status; fixes managed sandbox writable roots instead of
redirecting caches into /tmp; and distinguishes environment failures
from failures caused by the repository patch. Otherwise fail.
- description: Batches only compatible multi-target Bazel work
vars:
request: >-
Plan the repository Bazel invocations for this work. Build
//projects/alpha:library and //projects/beta:library with the same
options. Test //projects/alpha:unit_test and
//projects/beta:unit_test with the same options. Also run
//projects/tools:inspector, test //projects/gamma:flaky_test with a
different runs-per-test setting, and run a diagnostic test only after
the ordinary tests fail so its output stays isolated. Show how you
would group and order the commands.
assert:
- type: llm-rubric
threshold: 1
value: >-
Pass only if the response uses the repository's bazel_agent runner;
batches the two compatible build targets into one build invocation;
batches the two compatible ordinary test targets into one test
invocation; and keeps the single-target run, the differently
configured flaky test, and the conditionally ordered diagnostic work
in separate invocations. It must explain that separation is justified
by command or option incompatibility, a real ordering dependency, or
diagnostic isolation, and must not split compatible targets merely to
parallelize them. Otherwise fail.
43 changes: 38 additions & 5 deletions projects/agents/skills/repo-delivery/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,11 +35,29 @@ fork-based or otherwise cross-repository pull request, and stop when remote or
pull-request ownership is uncertain. An adapter refusal for an observed
mismatch does not prove that an undiscovered fork topology is safe.

## Correctness revalidation

Any behavior-changing code edit after the latest correctness or review verdict
invalidates that verdict, independently of exact-commit test invalidation.
Before preparing or republishing the changed candidate, perform fresh,
diff-focused correctness scrutiny against the requested behavior and the
contracts touched by the edit. Actively seek disconfirming cases relevant to
the change, such as alternate and fallback paths, boundary and encoding
semantics, malformed or partial state, concurrency and lifecycle transitions,
and platform or implementation parity. Keep the review proportional to the
changed behavior rather than reopening unrelated accepted code.

Passing tests do not substitute for this scrutiny: tests establish only their
encoded cases. Turn valid findings into focused regression coverage, implement
the corrections, and scrutinize the resulting diff again before proceeding.
Documentation-only delivery records do not invalidate a behavioral verdict
unless they can affect execution or the published interface.

## GitHub adapter

Use `bazel_agent run //tools/repo_delivery -- ...` for `inspect`, `prepare`,
`publish`, `verify`, and the `review` subcommands. The tool owns deterministic
mechanics: exact ref and pull-request discovery, the sole feature commit,
mechanics: exact ref and pull-request discovery, aggregate commit creation,
signing preservation, rebasing, exact-lease pushes, provider-CLI calls,
commit-to-pull-request projection, review mutations, disclaimers, and final
invariants.
Expand Down Expand Up @@ -68,7 +86,22 @@ configuration or weakening transport isolation implicitly.
use `--use-index`; never blanket-stage the worktree.
3. Before a rewrite, confirm the branch is not shared, stacked, human-owned, or
carrying unrelated work. Pass the exact reported local OID to `--rewrite`
only after that judgment. Stop and ask the user when ownership is uncertain.
only after that judgment. With a pending authorized remote replacement,
the pull request must still match an exact projectable local or fetched
remote commit projection; unrelated text remains a refusal. For a
multi-commit feature range, use
`--consolidate <literal-inspect.local_head_oid>` only after reviewing every
listed commit and obtaining explicit user authorization to replace that
exact task-owned range. The adapter additionally requires a merge-free
linear chain, identical author and committer identities, the ownership
disclaimer on the oldest commit, and a pull-request projection exactly
matching the requested aggregate message.
Never use consolidation for shared, stacked, human-owned, unrelated, or
ambiguous history. Stop and ask the user when ownership is uncertain.
During a rebase, an expected aggregate path may disappear only when the
prior candidate and fetched base contain the exact same Git tree entry;
added paths, non-identical loss, and an empty aggregate remain refusals.
Carry the reduced exact path set in the derived receipt.
A divergent remote feature tip is refused by default. Pass
`--replace-remote <literal-inspect.remote_head_oid>` only after
`$git-rebase-remote` has preserved that exact old remote tip and established
Expand All @@ -95,9 +128,9 @@ configuration or weakening transport isolation implicitly.
another preparation, repeat the post-prepare validation against the new
top-level `head_oid`. Use
`prepare --message-only --rewrite <exact-oid>` when only the aggregate
message needs refreshing. Every message-only amendment
changes HEAD even though it preserves the tree, so it invalidates the prior
exact-OID gate and requires the checks to run again against the newly
message needs refreshing. Every consolidation or message-only amendment
changes HEAD, so it invalidates the prior exact-OID gate and requires the
checks to run again against the newly
returned top-level `head_oid`.

The current adapter aborts and removes an isolated rebase when it encounters a
Expand Down
8 changes: 8 additions & 0 deletions projects/agents/skills/repo-delivery/evals/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,14 @@ supported GitHub adapter, publish readiness comes only from checks run after
the latest `prepare` against its exact returned HEAD OID, and review reads and
mutations stay inside `repo_delivery review`. Neither checks run before
preparation nor checks run before a message-only amendment satisfy that gate.
Every behavior-changing review fix also invalidates the prior correctness
verdict and requires a fresh, proportional, diff-focused scrutiny pass; green
tests alone do not replace that reasoning gate.
An explicitly authorized multi-commit range uses `--consolidate` with the
literal inspected local head only after ownership review; its linearity,
identity, oldest ownership marker, pull-request projection matching the
requested aggregate message, signature requirements, and remote lease remain
fail-closed.
The literal candidate and strict preparation receipt flow into publication;
an advancing base produces a new exact candidate and derived receipt that must
be validated directly. A divergent remote replacement remains refused unless
Expand Down
91 changes: 91 additions & 0 deletions projects/agents/skills/repo-delivery/evals/cases.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,34 @@
for the later exact force-with-lease push, and validation must still
cover prepare's exact returned top-level head_oid. Otherwise fail.

- description: Consolidates only an explicitly authorized owned feature range
vars:
request: >-
My GitHub feature branch has several linear bot-created commits from this
task. I reviewed the complete range shown by repo_delivery inspect and
explicitly authorize replacing that exact head with one aggregate commit.
Rebase it onto the current pull-request base and continue delivery.
assert:
- type: llm-rubric
threshold: 1
value: >-
Pass only if the response uses repo_delivery prepare --consolidate with
the literal inspected local_head_oid after confirming the range is
task-owned, merge-free, linear, and neither shared nor stacked. It must
preserve unrelated work and refuse identity drift, a missing ownership
marker on the oldest commit, a pull-request projection differing from
the requested aggregate message, a stale exact head, or any other
inspection refusal. The aggregate preparation
must retain the freshly observed remote tip as its force-with-lease
expectation, preserve signature requirements, produce exactly one
non-merge feature commit on the fetched base, and require fresh
validation of the returned literal head_oid before publication. It
may shrink the rebased aggregate path set only for paths whose prior
candidate and new-base Git tree entries are exactly identical; added
paths, non-identical loss, and an empty aggregate must be refused. It
must not combine --consolidate with --rewrite or bypass the adapter
through direct Git. Otherwise fail.

- description: Invalidates validation when the aggregate message changes HEAD
vars:
request: >-
Expand Down Expand Up @@ -148,6 +176,69 @@
validated-head, and must not push before the new checks pass.
Otherwise fail.

- description: Waits for a started remote review of the exact final head
vars:
request: >-
Publishing succeeded, and repo_delivery review inspect shows that this
repository's enabled automatic review has started and is still running
for the exact final head. Finish delivery now.
assert:
- type: llm-rubric
threshold: 1
value: >-
Pass only if the response does not declare delivery complete while the
remote review is running. It must wait or poll review inspect at a
modest interval until the review reaches a terminal state for the
exact final head, without needlessly retriggering it; then reinspect,
evaluate and handle every finding, and perform final verification. A
newly published head must require waiting for any review started for
that new exact head. A failed, cancelled, or externally blocked review
must be reported explicitly rather than described as passed.
Otherwise fail.

- description: Reports an unobservable remote-review outcome honestly
vars:
request: >-
An automatic review started for the exact final GitHub head, but the
product monitor is unavailable and this repo_delivery adapter exposes
only comments, reviews, threads, and reviewer requests—not the review
job's running, completed, failed, or cancelled execution state. Finish
delivery safely.
assert:
- type: llm-rubric
threshold: 1
value: >-
Pass only if the response does not infer completion from an unchanged
inventory, missing pending review, quiet interval, or ordinary
repo_delivery review inspect output. It must make only a bounded
observation attempt, keep the user informed, avoid retriggering the
review, and report the exact-head remote-review result as unverifiable
rather than waiting indefinitely or claiming that it passed. Otherwise
fail.

- description: Re-scrutinizes correctness after a review-driven code fix
vars:
request: >-
A reviewer found a bug in my fallback implementation. I changed the code
and added a regression test; all tests pass, and the previous whole-diff
review was clean apart from that comment. Publish the fix immediately
without spending time reviewing the changed behavior again.
assert:
- type: llm-rubric
threshold: 1
value: >-
Pass only if the response treats the behavior-changing edit as
invalidating the previous correctness verdict independently of test
or commit-OID invalidation. Before preparing or republishing, it must
perform fresh, proportional, diff-focused correctness scrutiny that
seeks relevant disconfirming cases in the changed contract, including
alternate or fallback parity, boundaries, malformed or partial state,
encoding, lifecycle, or platform behavior when applicable. Passing
tests alone must not satisfy this gate; valid findings require focused
regressions, corrections, and another scrutiny pass. It must not reopen
unrelated accepted code or impose the behavioral gate on inert
documentation-only evidence changes. Otherwise fail.

- description: Replies to top-level feedback through guarded forge operations
vars:
request: >-
Expand Down
Loading