Skip to content

Add update-androidsdk-packages skill and skill-runner workflow - #12374

Open
jonathanpeppers wants to merge 10 commits into
mainfrom
jonathanpeppers-add-update-androidsdk-packages-skill
Open

Add update-androidsdk-packages skill and skill-runner workflow#12374
jonathanpeppers wants to merge 10 commits into
mainfrom
jonathanpeppers-add-update-androidsdk-packages-skill

Conversation

@jonathanpeppers

Copy link
Copy Markdown
Member

Why

src/androidsdk/androidsdk.csproj pins Android SDK package versions/revisions/URLs/SHA-256 hashes in Configuration.props and src/androidsdk/androidsdk.targets. Google republishes tool revisions on its own cadence (see PR #12371 for the last manual refresh), and keeping these two files in sync is a repetitive, error-prone task: it requires querying Google's manifest, distinguishing stable from preview releases, and recomputing SHA-256 hashes per host/arch since Google only publishes SHA-1.

What this adds

update-androidsdk-packages Copilot skill (.github/skills/update-androidsdk-packages/)

Automates the refresh, matching the pattern from PR #12371. It enforces two hard rules:

  • Never touch the Android NDK (_XAAndroidNdk*/XAAndroidNdkHash*) - out of scope, separate release cadence.
  • Never add a new Android platform API level to _PlatformPackage - only refresh revisions/hashes of API levels already in the catalog. If a newer stable platform level exists upstream than what's in the catalog (e.g. platform 37.1 while the catalog tops out at 37.0), the skill must always surface that fact in its summary, even if nothing else changed.

Bundled resources:

  • scripts/fetch_repo_package.cs, scripts/sha256_of_url.cs - C# file-based dotnet run apps (matching the ci-status skill's convention) that query Google's repository2-3.xml manifest and compute authoritative SHA-256 hashes by downloading archives to a scratch temp file. Preview/alpha/beta/RC/canary releases are filtered out using a word-boundary regex plus the <revision><preview> element, since channelRef alone isn't a reliable signal.
  • references/package-catalog.md - maps each package family to its manifest path and Configuration.props/androidsdk.targets properties.
  • evals/evals.json - realistic prompts covering normal updates and both hard-rule exclusions.

Validated by building Xamarin.Android.Tools.BootstrapTasks.csproj (required for androidsdk.csproj to evaluate), live-testing both scripts against Google's real manifest, and a rubber-duck review that found and fixed several bugs (false-positive preview matches inside substrings, relative archive URLs not resolved against the manifest base URI, preview revisions colliding in the sort key, and more).

skill-runner agentic workflow (.github/workflows/skill-runner.md)

A gh-aw workflow, modeled on nightly-fix-finder, that runs repository Copilot skills unattended on a weekly schedule or on demand:

  • workflow_dispatch exposes a skill dropdown (currently just update-androidsdk-packages, validated against an ELIGIBLE_SKILLS allowlist). When no skill is explicitly selected, one is picked at random among eligible skills.
  • The prompt is skill-agnostic: it loads whichever SKILL.md was selected and follows it verbatim.
  • Every run reports its outcome - including no-ops, a newly published platform level found upstream, and any errors - on a self-deduplicating tracking issue. A PR is opened only when validated changes are produced, restricted to Configuration.props and src/androidsdk/androidsdk.targets.
  • An "Adding a New Skill" section documents how to wire in additional skills later.

Review notes

  • No functional/build code is touched by this PR - it's entirely new skill/workflow tooling.
  • skill-runner.lock.yml is generated via gh aw compile; don't hand-edit it.
  • The workflow's PAT-pool secret usage mirrors the already-approved pattern in nightly-fix-finder.md.

jonathanpeppers and others added 2 commits August 13, 2026 10:01
…nner workflow

Add a new project-local Copilot skill, update-androidsdk-packages, that
automates refreshing the stable Android SDK package pins (versions,
revisions, URLs, SHA-256 hashes) that src/androidsdk/androidsdk.csproj
consumes from Configuration.props and src/androidsdk/androidsdk.targets,
modeled on the update pattern in PR #12371.

The skill enforces two hard rules:
1. Never touch the Android NDK (_XAAndroidNdk*/XAAndroidNdkHash*) --
   it has its own release cadence and is out of scope here.
2. Never add a new Android platform API level to _PlatformPackage --
   only refresh the revision/archive/hash of API levels already in the
   catalog. If Google has published a newer stable platform level than
   the highest one already present (e.g. platform 37.1 while the
   catalog tops out at 37.0), do not add it, but always surface that
   fact in the final summary every run, regardless of whether the
   request otherwise mentioned platform levels.

Bundled resources:
- scripts/fetch_repo_package.cs, scripts/sha256_of_url.cs -- C#
  file-based `dotnet run` apps (matching the ci-status skill's
  ci_failures.cs convention) that query Google's repository2-3.xml SDK
  manifest and compute authoritative SHA-256 hashes by downloading
  archives into a scratch temp file, since Google's manifests only
  publish SHA-1. Preview/alpha/beta/RC/canary releases are filtered
  with a word-boundary regex plus a <revision><preview> check, since
  channelRef alone is not a reliable stable/preview signal.
- references/package-catalog.md -- mapping from each package family to
  its manifest path and Configuration.props/androidsdk.targets
  properties.
- evals/evals.json -- realistic prompts covering normal updates and
  both hard-rule exclusions.

Both scripts were reviewed via a rubber-duck pass that found and fixed:
false-positive preview matches inside substrings like "sources", a
missing <revision><preview> preview signal, channelRef being read from
the wrong XML location, relative archive URLs not resolved against the
manifest base URI, a Uri constructor call outside its try block, and a
cleanup File.Delete that could mask the real success/failure outcome.

Validated by building
build-tools/Xamarin.Android.Tools.BootstrapTasks/Xamarin.Android.Tools.BootstrapTasks.csproj
(required for androidsdk.csproj to evaluate) followed by
src/androidsdk/androidsdk.targets evaluation, live-testing both C#
scripts against Google's real manifest, and confirming a clean diff.

Also add a new gh-aw agentic workflow, skill-runner, that runs
repository Copilot skills unattended on a weekly schedule or on demand:

- workflow_dispatch exposes a `skill` dropdown (currently just
  update-androidsdk-packages) so a caller can pick which skill runs.
  When no skill is explicitly selected (scheduled runs, or dispatch
  left blank), a bootstrap step picks uniformly at random among the
  eligible skills -- the same pattern nightly-fix-finder uses for its
  scan scripts -- and records the choice for the prompt to read.
- The prompt is skill-agnostic: it loads whichever SKILL.md was
  selected and follows it verbatim, consulting a "Known Skills" table
  only for workflow-level context (always-report rules, prerequisite
  bootstraps) that supplements rather than overrides the skill file.
- Every run reports its outcome -- including pure no-ops, a newly
  published platform level found upstream, and any errors -- on a
  self-deduplicating tracking issue (close-older-issues, 30-day
  expiry). A PR is opened only when validated changes are produced,
  restricted to Configuration.props and
  src/androidsdk/androidsdk.targets.
- An "Adding a New Skill" section documents the steps to wire in
  additional skills: dropdown + ELIGIBLE_SKILLS array entry, Known
  Skills table row, optional prerequisite-bootstrap elif branch,
  allowed-files update, and recompiling with `gh aw compile`.

Compiled cleanly with `gh aw compile --approve` (the PAT-pool secrets
this workflow references are the same ones already approved for
nightly-fix-finder).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…unner

Address three issues found in a rubber-duck review of the previous
squashed commit:

1. fetch_repo_package.cs: revision sort/display key ignored
   <revision><preview>, so preview builds with different preview
   numbers (e.g. 36.0.0-rc1 vs 36.0.0-rc2) collapsed to the same sort
   key as each other and as the stable release, making "sorted newest
   first" unreliable under --all. The key is now a 4-tuple including
   preview number, with stable releases (preview == 0) always sorting
   above previews of the same major.minor.micro -- verified live
   against Google's manifest that build-tools 37.0.0 now sorts above
   37.0.0-rc2 above 37.0.0-rc1.

2. skill-runner.md: an explicit workflow_dispatch `skill` input was
   accepted as long as `.github/skills/<input>/SKILL.md` existed on
   disk, bypassing the documented ELIGIBLE_SKILLS allowlist entirely.
   The bootstrap step now validates the input is an exact member of
   ELIGIBLE_SKILLS before using it, and fails fast if that array is
   ever empty instead of dividing by zero in `RANDOM % COUNT`.

3. SKILL.md carved out a rare exception allowing package.xml.in to
   change alongside Configuration.props/androidsdk.targets, but the
   skill-runner workflow's create-pull-request.allowed-files never
   included it -- a validated run could hit that rare path and then be
   unable to open its PR. Removed the exception: this skill's scope is
   package pins only, and automated runs must stay within the two
   files the workflow is authorized to touch.

Recompiled skill-runner.lock.yml with `gh aw compile --approve`.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings August 13, 2026 15:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds repository automation to keep Android SDK package pins (revisions/URLs/SHA-256) in sync with Google’s stable SDK manifest by introducing a dedicated Copilot skill and an unattended “skill runner” gh-aw workflow.

Changes:

  • Introduces the update-androidsdk-packages skill with detailed, hard-rule-guided procedure and bundled helper scripts for manifest querying + SHA-256 computation.
  • Adds a scheduled/on-demand skill-runner gh-aw workflow (plus generated lock file) to execute eligible skills, open PRs for validated changes, and report outcomes via a tracking issue.

Reviewed changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
.github/workflows/skill-runner.md New gh-aw workflow definition to select/run an eligible skill and bootstrap prerequisites.
.github/workflows/skill-runner.lock.yml Generated compiled workflow output for skill-runner.md.
.github/skills/update-androidsdk-packages/SKILL.md Skill playbook defining scope, hard rules, workflow, validation, and reporting requirements.
.github/skills/update-androidsdk-packages/scripts/sha256_of_url.cs Helper script to download an archive and print SHA-256 for pinning.
.github/skills/update-androidsdk-packages/scripts/fetch_repo_package.cs Helper script to query Google’s manifest for package revisions/archives and flag preview-looking entries.
.github/skills/update-androidsdk-packages/references/package-catalog.md Reference mapping of package families to manifest paths and repo pin locations.
.github/skills/update-androidsdk-packages/evals/evals.json Skill evaluation prompts/expectations to validate behavior and hard-rule exclusions.

Comment thread .github/skills/update-androidsdk-packages/scripts/sha256_of_url.cs Outdated
Replace `sha256.Hash!` with an explicit null-check that throws
InvalidOperationException, per repo C# guidance banning the
null-forgiving operator. Verified the script still hashes correctly
against a real archive (build-tools_r37_linux.zip).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@jonathanpeppers

Copy link
Copy Markdown
Member Author

/review

@github-actions

github-actions Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Android PR Reviewer completed successfully!

Generated by Android PR Reviewer for #12374

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Needs Changes

Findings: 0 errors · 1 warning · 0 suggestions

The skill/workflow separation, allowlisted output files, NDK/platform guardrails, and relative-manifest URL handling are well designed. The archive hashing path should verify Google's manifest-provided SHA-1 and size before treating the downloaded bytes as authoritative and emitting the SHA-256 pin.

CI is still in progress: four checks plus CLA have passed, three platform builds are running, and the aggregate check is queued; no failures are currently reported.

Generated by Android PR Reviewer for #12374 · gpt56 · 129.2 AIC · ⌖ 8.85 AIC · ⊞ 25.6K
Comment /review to run again

Comment thread .github/skills/update-androidsdk-packages/scripts/sha256_of_url.cs
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 49de4c35-38d9-4c5a-a41b-b6b017260349
@jonathanpeppers jonathanpeppers added the ready-to-review This PR is ready to review/merge, I think any CI failures are just flaky (ignorable). label Aug 13, 2026
@jonathanpeppers

Copy link
Copy Markdown
Member Author

@dalexsoto review

@dalexsoto dalexsoto left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Five workflow blockers remain:

  1. Manual dispatch can run from an arbitrary branch; unpinned checkouts execute that branch’s build/skill content and can open output from it. Require main, pin both checkouts, and set the PR base explicitly.

  2. The skill installs sources into sources/android-37.0, while consumers resolve sources/android-37; distinguish archive version 37.0 from the integer destination.

  3. Platform search results sort only by package revision, not API/extension level, so older platforms can be reported as newest. Sort platform paths by API and extension.

  4. Weekly runs do not detect an existing open update PR, so unchanged main can produce duplicate PRs. Add stable-branch/open-PR deduplication.

  5. cancel-in-progress: true can cancel an earlier run before its unconditional report (or during output processing). Serialize without cancellation or otherwise guarantee final reporting.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 49de4c35-38d9-4c5a-a41b-b6b017260349
@jonathanpeppers

Copy link
Copy Markdown
Member Author

Addressed all five blockers from review 4931001292 in 1a3f6a1: workflows now check out and target main with queued non-cancelling concurrency; source destinations use integer API directories; platform discovery sorts by API/extension; and existing open skill-runner PRs are detected before creating another. The workflow was recompiled and the platform helper was verified against the live manifest.

@dalexsoto dalexsoto left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two workflow blockers remain:

  1. Manual dispatch still does not reject non-main refs. The selected ref controls the workflow/activation content, an activation checkout remains unpinned, and generated PR-context handling can replace the main checkout with branch content. Reject non-main dispatches before activation and disable PR-context checkout for this workflow.

  2. Open-PR deduplication is only an agent prompt using a default-limited gh pr list and an “equivalent” judgment. It can miss older PRs or allow multiple skill-runner updates. Use a deterministic paginated/search-filtered key and enforce it immediately before PR creation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 49de4c35-38d9-4c5a-a41b-b6b017260349
@jonathanpeppers

Copy link
Copy Markdown
Member Author

Addressed both workflow blockers: manual dispatches are rejected unless the ref is main, and the dedupe check is now a deterministic exact-title search keyed to the selected skill before PR creation.

@dalexsoto dalexsoto left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Three workflow blockers remain:

  1. The non-main dispatch rejection runs only after pre_activation, PAT-pool selection, and activation have already processed branch-controlled workflow content. Enforce main at the earliest job/environment conditions before activation.

  2. Generated aw_context handling can still replace the pinned main checkout with a PR branch on manual dispatch. Reject PR ambient context or disable that generated checkout.

  3. PR deduplication remains prompt-only and bounded to gh pr list --limit 100; it is neither exhaustive nor enforced in the trusted creation path. Add deterministic exhaustive uniqueness enforcement immediately before PR creation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 49de4c35-38d9-4c5a-a41b-b6b017260349
@jonathanpeppers

Copy link
Copy Markdown
Member Author

Fixed the remaining blockers by rejecting non-main workflow_dispatch runs before activation, forbidding PR aw_context on manual dispatch, and enforcing the exact-match open-PR dedupe gate before any pull request creation.

@dalexsoto dalexsoto left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two workflow blockers remain:

  1. The main guard is only on activation; pre_activation and pat_pool still run first without the non-main workflow_dispatch condition, so branch-controlled compiled workflow content reaches the protected environment and PAT-selection path before rejection. Apply the main-ref gate to the earliest jobs (including pre_activation/pat_pool), not only their dependent activation job.

  2. Open-PR deduplication is still agent-prompt logic and its prescribed command is bounded by gh pr list --limit 100; “repeat/paginate” does not make that command exhaustive, and nothing in the trusted safe-output creation path enforces the exact workflow/skill key. Add deterministic exhaustive uniqueness enforcement immediately before PR creation outside the agent’s discretionary prompt execution.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 49de4c35-38d9-4c5a-a41b-b6b017260349
@jonathanpeppers

Copy link
Copy Markdown
Member Author

You were right that the previous change only restated the controls in the agent prompt and gated activation too late. Fixed in 354bd42: the ref/PR-context checks are now injected into pre_activation via on.steps, and an exact GitHub search filter prevents any run when an open [skill-runner] update PR for update-androidsdk-packages already exists. The compiled lock file was regenerated.

@dalexsoto dalexsoto left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Four workflow blockers remain:

  1. The non-main/PR-context checks are steps inside pre_activation, so the job enters the protected copilot-pat-pool environment and runs setup, membership, and search actions before rejecting branch-controlled dispatches. Put these conditions on the job itself before environment allocation (skill-runner.lock.yml:1712-1776).

  2. PR deduplication is neither exact nor enforced at creation: the early title search omits base:main, safe outputs only require the generic [skill-runner] prefix, and the trusted creation handler does not recheck the workflow/skill key. Enforce the exact key against main immediately before creation (skill-runner.lock.yml:1752-1764, 1885-1903).

  3. A skip-query match disables activation, which skips the agent and conclusion; the required no-op tracking report and platform-catalog status are never emitted. Route duplicate matches through the mandatory reporting path (skill-runner.lock.yml:1721-1764, 1127-1145).

  4. The skill requires both androidsdk.csproj validations with --no-restore, but a fresh runner only restores the unrelated BootstrapTasks graph. androidsdk.csproj/openjdk.csproj therefore have no assets file and validation cannot run. Restore that project first or remove --no-restore from the initial validation (SKILL.md:135-151).

jonathanpeppers and others added 2 commits August 14, 2026 09:17
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 49de4c35-38d9-4c5a-a41b-b6b017260349
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 49de4c35-38d9-4c5a-a41b-b6b017260349
@jonathanpeppers

Copy link
Copy Markdown
Member Author

Addressed the four latest blockers in 43b5204 and 1e17348: added a custom workflow_guard job required before pre_activation so invalid dispatches never enter the PAT environment; replaced skip-if-match with a base=main, paginated exact-title check that emits a safe noop before agent/PR creation and preserves conclusion reporting; and added dotnet restore src/androidsdk/androidsdk.csproj before --no-restore validation. The compiled lock file was regenerated.

@dalexsoto dalexsoto left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The trusted duplicate guard still only appends a noop; the bootstrap and agent continue, and safe-output processing can still create a pull request because there is no exact-key recheck or suppression before create_pull_request. Stop the PR-producing path when a duplicate matches, or enforce the exhaustive exact title/base check inside the trusted creation handler (skill-runner.lock.yml:456-475, 1853-1889).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-review This PR is ready to review/merge, I think any CI failures are just flaky (ignorable).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants