Skip to content

feat: resolve upload_artifact temporary IDs to artifact URLs in safe output bodies#26108

Merged
pelikhan merged 4 commits intomainfrom
copilot/validate-temporary-ids-urls
Apr 14, 2026
Merged

feat: resolve upload_artifact temporary IDs to artifact URLs in safe output bodies#26108
pelikhan merged 4 commits intomainfrom
copilot/validate-temporary-ids-urls

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 13, 2026

upload_artifact had no way to declare a stable temporary ID, and even when a tmpId was generated, it was never registered in the handler manager — so #aw_ID references in subsequent create_discussion/create_issue/add_comment bodies were never replaced with the real artifact URL, causing broken images.

Changes

Schema (safe_outputs_tools.json)

  • Added optional temporary_id field to upload_artifact with a pattern constraint (^aw_[A-Za-z0-9_]{3,12}$) consistent with other tools, so agents can declare the ID they'll embed in downstream bodies (e.g. ![chart](#aw_chart1))

upload_artifact.cjs

  • Added resolveTemporaryArtifactId(message): uses message.temporary_id when present and valid, falls back to random generation; emits a warning and falls back on invalid format
  • Handler result now includes temporaryId (alongside tmpId) for consistency with other safe output handlers
  • Duplicate temporary_id detection: warns with a user-friendly message and keeps the first resolver mapping when the same ID appears on multiple uploads

temporary_id.cjs

  • Added replaceArtifactUrlReferences(text, artifactUrlMap): replaces #aw_ID with the corresponding artifact URL (strips the # prefix so the result is a valid URL); mirrors replaceTemporaryIdReferences by scanning for malformed #aw_... tokens via TEMPORARY_ID_CANDIDATE_PATTERN and emitting warnings for references that won't be resolved
  • Updated hasUnresolvedTemporaryIds(text, tempIdMap, artifactUrlMap?): treats an ID as resolved if present in either the issue-number map or the new artifact URL map

safe_output_handler_manager.cjs

  • Maintains artifactUrlMap: Map<string, string> alongside temporaryIdMap
  • After each successful upload_artifact, registers tmpId → artifactUrl; duplicate IDs keep the first registered URL with a warning
  • Pre-processes each message body with replaceArtifactUrlReferences before dispatch, so handlers receive bodies with real URLs already in place
  • Passes artifactUrlMap through hasUnresolvedTemporaryIds checks and processSyntheticUpdates so the synthetic-update path also resolves artifact references

safe_outputs_action_outputs.cjs

  • Added upload_artifact case to emit two new named step outputs for the first successful upload:
    • upload_artifact_tmp_id — the generated or agent-declared temporary ID
    • upload_artifact_url — the artifact download URL

Example

Agent output:

[
  { "type": "upload_artifact", "path": "chart.png", "temporary_id": "aw_chart1" },
  { "type": "create_discussion", "title": "Results", "body": "![chart](#aw_chart1)" }
]

After upload, the discussion body is rewritten to:

![chart](https://github.com/owner/repo/actions/runs/12345/artifacts/42)

When skip-archive: true, the URL points directly to the raw file, making it suitable for inline images.

Copilot AI and others added 2 commits April 13, 2026 23:15
- Add `temporary_id` field to `upload_artifact` tool schema so agents can
  declare an artifact ID upfront, enabling topological sort ordering and
  `#aw_ID` reference embedding in subsequent message bodies
- Update `upload_artifact.cjs` to use `message.temporary_id` when provided
  (via `resolveTemporaryArtifactId`), falling back to random generation
- Add `replaceArtifactUrlReferences(text, artifactUrlMap)` to `temporary_id.cjs`
  for replacing `#aw_ID` references with actual artifact download URLs
- Update `hasUnresolvedTemporaryIds` to accept optional `artifactUrlMap` so
  artifact URL references count as resolved
- Update `safe_output_handler_manager.cjs`:
  - Track `artifactUrlMap: Map<string, string>` alongside `temporaryIdMap`
  - After successful `upload_artifact`, register `tmpId → artifactUrl`
  - Pre-process message bodies to replace artifact URL refs before handler calls
  - Pass `artifactUrlMap` to all `hasUnresolvedTemporaryIds` checks
  - Update `processSyntheticUpdates` to apply artifact URL replacement
  - Include `artifactUrlMap` in processing result for synthetic updates
- Add tests for new `resolveTemporaryArtifactId`, `replaceArtifactUrlReferences`,
  and updated `hasUnresolvedTemporaryIds` functionality

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/9e280abe-a3c6-4060-a0ec-8fbb2812d072

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot the safe output upload-artifact should return the generated or provided temporary id as well

@pelikhan pelikhan marked this pull request as ready for review April 13, 2026 23:53
Copilot AI review requested due to automatic review settings April 13, 2026 23:53
…action outputs

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/21c1222b-06ce-47c5-aa80-05d54d1b0e2a

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 13, 2026

@copilot the safe output upload-artifact should return the generated or provided temporary id as well

Done in d57abae. The upload_artifact handler now returns temporaryId in its result object (alongside the existing tmpId), and emitSafeOutputActionOutputs has been updated to emit two new named step outputs for the first successful artifact upload:

  • upload_artifact_tmp_id — the generated or agent-declared temporary ID
  • upload_artifact_url — the artifact download URL

Copilot AI requested a review from pelikhan April 13, 2026 23:57
@github-actions github-actions bot mentioned this pull request Apr 13, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds first-class support for stable upload_artifact temporary IDs and resolves #aw_ID references in downstream “safe output” message bodies to the final artifact download URL, preventing broken inline images/links.

Changes:

  • Extend the upload_artifact tool schema(s) with an optional temporary_id field for stable cross-message references.
  • Teach upload_artifact to honor a valid declared temporary_id (normalize/validate; warn+fallback on invalid).
  • Track tmpId → artifactUrl in the handler manager and rewrite message bodies (and synthetic updates) by replacing #aw_ID with the resolved artifact URL.
Show a summary per file
File Description
pkg/workflow/js/safe_outputs_tools.json Adds upload_artifact.temporary_id to the workflow-side tool schema.
actions/setup/js/safe_outputs_tools.json Adds upload_artifact.temporary_id to the action runtime tool schema.
actions/setup/js/upload_artifact.cjs Resolves a stable tmpId from message.temporary_id when valid; falls back to generated IDs otherwise.
actions/setup/js/upload_artifact.test.cjs Adds coverage for declared/normalized/invalid temporary_id behavior and resolver output.
actions/setup/js/temporary_id.cjs Adds artifact URL replacement and expands unresolved-ID checks to include artifact URL mappings.
actions/setup/js/temporary_id.test.cjs Adds tests for artifact URL replacement and unresolved-ID logic across both maps.
actions/setup/js/safe_output_handler_manager.cjs Maintains an artifactUrlMap, pre-rewrites bodies before dispatch, and resolves artifact refs in synthetic updates.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 9/9 changed files
  • Comments generated: 5

"additionalProperties": false
},
"temporary_id": {
"type": "string",
Copy link

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

upload_artifact.inputSchema.temporary_id is missing a pattern constraint even though other temporary_id fields in this file include one (e.g. create_issue and add_comment). Adding a pattern here would provide consistent schema-level validation and clearer feedback to callers about the accepted format.

Suggested change
"type": "string",
"type": "string",
"pattern": "^aw_[A-Za-z0-9_]{3,12}$",

Copilot uses AI. Check for mistakes.
"additionalProperties": false
},
"temporary_id": {
"type": "string",
Copy link

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

upload_artifact.inputSchema.temporary_id is missing a pattern constraint even though other temporary_id fields in this schema include one. Adding a pattern here would make validation consistent and prevent invalid IDs from making it to runtime only to be replaced with a random ID.

Suggested change
"type": "string",
"type": "string",
"pattern": "^aw_[A-Za-z0-9_]{3,12}$",

Copilot uses AI. Check for mistakes.
Comment on lines +436 to +447
function replaceArtifactUrlReferences(text, artifactUrlMap) {
if (!artifactUrlMap || artifactUrlMap.size === 0) {
return text;
}
return text.replace(TEMPORARY_ID_PATTERN, (match, tempId) => {
const url = artifactUrlMap.get(normalizeTemporaryId(tempId));
if (url !== undefined) {
// Replace #aw_XXXX with the URL directly (no '#' prefix)
return url;
}
return match;
});
Copy link

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replaceArtifactUrlReferences currently does not emit any warning for malformed #aw_... tokens that won't match TEMPORARY_ID_PATTERN (e.g. #aw_bad-id). replaceTemporaryIdReferences in the same module does a candidate scan via TEMPORARY_ID_CANDIDATE_PATTERN to warn about these cases; consider mirroring that logic here so artifact references fail loudly/diagnosably rather than silently remaining broken.

Copilot uses AI. Check for mistakes.
const artifactName = deriveArtifactName(message, i);
const tmpId = generateTemporaryArtifactId();
const tmpId = resolveTemporaryArtifactId(message);
resolver[tmpId] = artifactName;
Copy link

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolver[tmpId] = artifactName will overwrite an existing entry if the same temporary_id is declared on multiple upload_artifact messages. That can break downstream resolution (and will disagree with the topological sort behavior that treats only the first duplicate temporary_id as the provider). Consider detecting duplicates here and either fail validation or warn and keep the first mapping.

Suggested change
resolver[tmpId] = artifactName;
if (Object.prototype.hasOwnProperty.call(resolver, tmpId)) {
core.warning(
`upload_artifact: duplicate temporary_id "${tmpId}" for artifact "${artifactName}" in slot ${i}; keeping existing resolver mapping to "${resolver[tmpId]}".`
);
} else {
resolver[tmpId] = artifactName;
}

Copilot uses AI. Check for mistakes.
Comment on lines +580 to +581
artifactUrlMap.set(normalizedTmpId, result.artifactUrl);
core.info(`Registered artifact URL: ${result.tmpId} -> ${result.artifactUrl}`);
Copy link

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When registering tmpId → artifactUrl, artifactUrlMap.set(...) will overwrite an existing mapping if the same temporary_id is reused in multiple upload_artifact messages. Since the topological sorter explicitly warns that only the first occurrence of a duplicate temporary_id is used as the dependency provider, overwriting here can cause later replacements of #aw_ID to point at the wrong artifact URL. Consider treating duplicate tmpIds as an error or (at minimum) warn and keep the first URL (only set when the key is not already present).

Suggested change
artifactUrlMap.set(normalizedTmpId, result.artifactUrl);
core.info(`Registered artifact URL: ${result.tmpId} -> ${result.artifactUrl}`);
if (!artifactUrlMap.has(normalizedTmpId)) {
artifactUrlMap.set(normalizedTmpId, result.artifactUrl);
core.info(`Registered artifact URL: ${result.tmpId} -> ${result.artifactUrl}`);
} else {
const existingArtifactUrl = artifactUrlMap.get(normalizedTmpId);
core.warning(`Duplicate artifact temporary ID '${result.tmpId}' encountered; keeping existing artifact URL '${existingArtifactUrl}' and ignoring later URL '${result.artifactUrl}'`);
}

Copilot uses AI. Check for mistakes.
@github-actions
Copy link
Copy Markdown
Contributor

🧪 Test Quality Sentinel Report

Test Quality Score: 80/100

Excellent test quality

Metric Value
New/modified tests analyzed 15
✅ Design tests (behavioral contracts) 15 (100%)
⚠️ Implementation tests (low value) 0 (0%)
Tests with error/edge cases 10 (67%)
Duplicate test clusters 0
Test inflation detected ⚠️ Yes — temporary_id.test.cjs (2.09:1 ratio, marginal)
🚨 Coding-guideline violations None

Test Classification Details

All 15 tests (click to expand)
Test File Classification Issues Detected
should treat ID as resolved when present in artifactUrlMap temporary_id.test.cjs ✅ Design None — covers new third-param contract
should still detect unresolved ID when not in either map temporary_id.test.cjs ✅ Design None — good negative/mixed case
should return false when all IDs are covered across both maps temporary_id.test.cjs ✅ Design None — verifies combined-map resolution
should replace #aw_ID with the artifact URL temporary_id.test.cjs ✅ Design None — happy path for new function
should be case-insensitive for the temporary ID temporary_id.test.cjs ✅ Design None — edge case for case normalization
should leave non-artifact temporary IDs unchanged temporary_id.test.cjs ✅ Design None — boundary: mixed ID types
should handle multiple artifact references in the same text temporary_id.test.cjs ✅ Design None — multi-value case
should return text unchanged when artifactUrlMap is empty temporary_id.test.cjs ✅ Design None — empty input edge case
should return text unchanged when artifactUrlMap is null/undefined temporary_id.test.cjs ✅ Design None — null-safety edge case
uses declared temporary_id from message when valid upload_artifact.test.cjs ✅ Design None — core contract with output assertion
normalises declared temporary_id to lowercase upload_artifact.test.cjs ✅ Design None — normalization contract
strips leading '#' from declared temporary_id upload_artifact.test.cjs ✅ Design None — normalization contract
generates a random ID when temporary_id is not provided upload_artifact.test.cjs ✅ Design None — default behavior
generates a random ID and emits warning when temporary_id format is invalid upload_artifact.test.cjs ✅ Design None — error path with warning assertion
uses declared temporary_id in resolver file upload_artifact.test.cjs ✅ Design None — side-effect assertion on disk state

Observations

⚠️ Marginal Test Inflation — temporary_id.test.cjs

Inflation ratio: 71 lines added in test vs. 34 lines added in production (2.09:1, threshold 2:1).

This is borderline and not a quality concern — replaceArtifactUrlReferences is an entirely new function requiring dedicated coverage for 6 distinct scenarios. The ratio is explained by the new function needing thorough standalone testing. No action required, but noted for transparency.

✅ Mocking Approach

The upload_artifact.test.cjs tests mock mockCore (GitHub Actions core object) and the file system — both are external I/O dependencies and legitimate vitest mock targets. No internal business-logic functions are mocked without behavioral output assertions.

✅ Null/Undefined Safety

The should return text unchanged when artifactUrlMap is null/undefined test covers both null and undefined in a single it() block — a good defensive test for the new API parameter.

💡 Minor Suggestion

The tests in replaceArtifactUrlReferences could benefit from one additional edge case: what happens when the same temporary ID appears multiple times in the text? The current tests cover two different IDs together, but not repeated references to the same ID. This is low-priority since the regex used likely handles it correctly, but it would close a small contract gap.


Language Support

Tests analyzed:

  • 🐹 Go (*_test.go): 0 tests (no Go test files changed)
  • 🟨 JavaScript (*.test.cjs): 15 tests (vitest) across 2 files

Verdict

Check passed. 0% of new tests are implementation tests (threshold: 30%). All 15 new tests verify observable behavioral contracts — output values, normalization behavior, null-safety, error paths, and side-effects on the resolver file. No coding-guideline violations detected.


📖 Understanding Test Classifications

Design Tests (High Value) verify what the system does:

  • Assert on observable outputs, return values, or state changes
  • Cover error paths and boundary conditions
  • Would catch a behavioral regression if deleted
  • Remain valid even after internal refactoring

Implementation Tests (Low Value) verify how the system does it:

  • Assert on internal function calls (mocking internals)
  • Only test the happy path with typical inputs
  • Break during legitimate refactoring even when behavior is correct
  • Give false assurance: they pass even when the system is wrong

Goal: Shift toward tests that describe the system's behavioral contract — the promises it makes to its users and collaborators.

🧪 Test quality analysis by Test Quality Sentinel · ● 576.3K ·

Copy link
Copy Markdown
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: 80/100. Test quality is excellent — 0% of new tests are implementation tests (threshold: 30%). All 15 new JavaScript tests verify observable behavioral contracts covering the new replaceArtifactUrlReferences function and the extended hasUnresolvedTemporaryIds signature. No coding-guideline violations detected.

@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot review all comments

… malformed ref warnings

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/30754141-3e79-4bcd-a5ba-264de1a860c1

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 14, 2026

@copilot review all comments

Addressed all review comments in fb06ae1:

  1. Both safe_outputs_tools.json files — added "pattern": "^aw_[A-Za-z0-9_]{3,12}$" to the upload_artifact temporary_id field for consistent schema-level validation.
  2. temporary_id.cjsreplaceArtifactUrlReferences now scans for malformed #aw_... tokens via TEMPORARY_ID_CANDIDATE_PATTERN (matching replaceTemporaryIdReferences) and emits a warning explaining the reference won't be replaced.
  3. upload_artifact.cjs — duplicate temporary_id detection: warns with a user-friendly message and keeps the first resolver mapping.
  4. safe_output_handler_manager.cjs — duplicate tmpId in artifactUrlMap: keeps the first URL, warns without exposing full URLs in the log.

@pelikhan pelikhan merged commit ec6f8b8 into main Apr 14, 2026
@pelikhan pelikhan deleted the copilot/validate-temporary-ids-urls branch April 14, 2026 00:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants