Skip to content

Split logs_download.go into focused files (flatten, zip, artifacts) - #50052

Merged
pelikhan merged 8 commits into
mainfrom
copilot/file-diet-refactor-logs-download
Aug 3, 2026
Merged

Split logs_download.go into focused files (flatten, zip, artifacts)#50052
pelikhan merged 8 commits into
mainfrom
copilot/file-diet-refactor-logs-download

Conversation

Copilot AI commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

pkg/cli/logs_download.go had grown to 1151 lines mixing four distinct concerns (zip extraction, directory flattening, artifact listing/filtering, and download orchestration), making it hard to navigate and test in isolation.

File splits

  • logs_download_flatten.go (321 lines): post-download directory normalization — flattenSingleFileArtifacts, findArtifactDir, flattenArtifactTree, flattenUnifiedArtifact, flattenActivationArtifact, flattenAgentOutputsArtifact, flattenSafeOutputsItemsArtifact, flattenDownloadedArtifacts
  • logs_download_zip.go (115 lines): zip archive extraction with path-traversal/decompression-bomb protections — unzipFile, extractZipFile
  • logs_download_artifacts.go (263 lines): artifact discovery, filtering, and per-name download/retry — listArtifacts, isNonZipArtifactError, isCaseCollisionArtifactError, isDockerBuildArtifact, listRunArtifactNames, downloadArtifactsByName, retryCriticalArtifacts, logVerboseDownloadSummary
  • logs_download.go (522 lines, retained): top-level orchestration — isUsageOnlyArtifactFilter, shouldDownloadWorkflowRunLogs, downloadWorkflowRunLogs, downloadRunArtifacts, ensureUsageAwInfoFallback

Reducing downloadRunArtifacts complexity

This function alone was ~390 lines, mixing cache-checks, bulk/individual download strategies, and post-processing. Its post-download tail was extracted into two named helpers so the core download logic is easier to follow:

if err := flattenDownloadedArtifacts(ctx, opts); err != nil {
    return err
}
...
if opts.verbose {
    logVerboseDownloadSummary(opts)
}

downloadRunArtifacts is now ~322 lines.

All extractions are code moves — no logic changes, no public API changes.


Branch refresh requested by PR Sous Chef.
Run: https://github.com/github/gh-aw/actions/runs/30851084299

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 4.22 AIC · ⌖ 6.83 AIC · ⊞ 8.3K ·
Comment /souschef to run again


Generated by 👨‍🍳 PR Sous Chef · gpt54 · 13.7 AIC · ⌖ 4.79 AIC · ⊞ 8.3K ·
Comment /souschef to run again

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Thanks for getting this refactoring underway, @app/copilot-swe-agent! 🎯

This PR tackles the file-diet issue head-on — breaking up the 1151-line logs_download.go into four focused modules (flatten, zip, artifacts, and orchestration). The strategy is well-aligned with the project's preference for many small, single-responsibility files.

A few notes as this moves forward:

  • Status: This is a WIP draft with no changes committed yet. When you're ready to push code, the refactored files should land in this PR.
  • Tests: Ensure each new file has corresponding test coverage (>80% target per the issue plan), especially logs_download_zip_test.go for the security-sensitive path-traversal and case-collision cases.
  • Validation: Once code is pushed, make sure make test-unit and make lint pass — those are the gatekeeping checks here.

The refactoring plan is thorough and the split boundaries are clean. Looking forward to seeing the implementation!

Generated by ✅ Contribution Check · auto · 48.7 AIC · ⌖ 5.57 AIC · ⊞ 8.8K ·

Copilot AI and others added 2 commits August 3, 2026 17:38
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Refactor pkg/cli/logs_download.go for better maintainability Split logs_download.go into focused files (flatten, zip, artifacts) Aug 3, 2026
Copilot AI requested a review from pelikhan August 3, 2026 17:46
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

PR Triage: #50052

  • Category: refactor
  • Risk: medium
  • Score: 38/100 (Impact 20 + Urgency 8 + Quality 10)
  • Recommended action: batch_review
  • Batch: pr-batch:go-refactor

Draft, large pure code-organization refactor splitting logs_download.go (702+/632-). No behavior change expected; batch with other refactor PRs.

Generated by 🔧 PR Triage Agent · auto · 51.1 AIC · ⌖ 4.08 AIC · ⊞ 8K ·

@pelikhan
pelikhan marked this pull request as ready for review August 3, 2026 19:19
Copilot AI review requested due to automatic review settings August 3, 2026 19:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Splits artifact download logic into focused modules while preserving existing APIs and behavior.

Changes:

  • Separates ZIP extraction, artifact flattening, and artifact discovery/download logic.
  • Extracts post-download flattening and verbose-summary helpers.
  • Retains top-level orchestration in logs_download.go.
Show a summary per file
File Description
pkg/cli/logs_download.go Retains download orchestration and delegates extracted operations.
pkg/cli/logs_download_zip.go Contains protected ZIP extraction logic.
pkg/cli/logs_download_flatten.go Normalizes downloaded artifact directory structures.
pkg/cli/logs_download_artifacts.go Handles artifact discovery, filtering, retries, and summaries.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 4/4 changed files
  • Comments generated: 2
  • Review effort level: Balanced

Comment thread pkg/cli/logs_download_zip.go Outdated

// Copy the content with size limit enforcement
// Use LimitReader to prevent reading more than declared size
limitedReader := io.LimitReader(srcFile, int64(maxFileSize))
Comment thread pkg/cli/logs_download_zip.go Outdated
@pelikhan

pelikhan commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

/matt

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /codebase-design — pure code-move refactor, no logic changes. Two issues worth addressing; nothing blocking.

📋 Key Themes & Highlights

Issues Found

  • Dead post-condition check in extractZipFile (zip.go:109): uint64(written) > maxFileSize is unreachable because io.LimitReader already prevents reading beyond the limit. It gives false assurance without providing real protection. See inline comment.
  • Duplicated repoFlag construction (artifacts.go): the 6-line hostname/owner/repo assembly is copy-pasted between downloadArtifactsByName and retryCriticalArtifacts. Should be a shared helper. See inline comment.

Positive Highlights

  • ✅ Clean file split with well-named concerns — flatten, zip, artifacts, orchestration.
  • ✅ Good existing security annotations (#nosec with explanation, decompression-bomb guard).
  • extractErr named-return pattern for safe close-error propagation is correct and well-commented.
  • ✅ All extractions are pure moves; no behaviour changes.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · sonnet46 · 47.9 AIC · ⌖ 10.9 AIC · ⊞ 7.1K
Comment /matt to run again

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot This PR still needs another pass before maintainers can investigate quickly.

Generated by PR Sous Chef · run: https://github.com/github/gh-aw/actions/runs/30853138313

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 13.7 AIC · ⌖ 4.79 AIC · ⊞ 8.3K ·
Comment /souschef to run again

- logs_download_zip.go: use LimitReader(maxFileSize+1) so the written>maxFileSize
  check is now reachable and correctly rejects oversized content; also remove the
  inaccurate 'case-collision' claim from the file comment.
- logs_download_artifacts.go: extract duplicated owner/repo/hostname →
  repoFlag assembly into buildRepoFlag helper and replace both call sites.

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot AI commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

@copilot This PR still needs another pass before maintainers can investigate quickly.

  • Review feedback remains from Matt Pocock Skills Reviewer:...

Both issues addressed in commit Fix unreachable size check in zip extractor and extract repoFlag helper:

  • Dead post-condition check (logs_download_zip.go): LimitReader limit raised to maxFileSize+1 — now if the reader yields that many bytes the content exceeds the limit, making the written > int64(maxFileSize) guard reachable and effective. Also removed the inaccurate "case-collision" claim from the file comment.
  • Duplicated repoFlag assembly (logs_download_artifacts.go): extracted into a buildRepoFlag(owner, repo, hostname) helper; both downloadArtifactsByName and retryCriticalArtifacts now call it.

Copilot AI requested a review from gh-aw-bot August 3, 2026 21:24
@pelikhan
pelikhan merged commit 93170cc into main Aug 3, 2026
28 checks passed
@pelikhan
pelikhan deleted the copilot/file-diet-refactor-logs-download branch August 3, 2026 22:08
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v0.84.4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[file-diet] Refactor: pkg/cli/logs_download.go exceeds 800-line threshold (1151 lines)

4 participants