Skip to content

Optimize normalizeJar memory footprint and preserve entry metadata - #2164

Merged
Goooler merged 10 commits into
mainfrom
g/20260809/stop-normalizing
Aug 9, 2026
Merged

Optimize normalizeJar memory footprint and preserve entry metadata#2164
Goooler merged 10 commits into
mainfrom
g/20260809/stop-normalizing

Conversation

@Goooler

@Goooler Goooler commented Aug 9, 2026

Copy link
Copy Markdown
Member

No description provided.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves the post-R8 jar “normalization” step so Shadow can rewrite R8’s output jar while preserving important entry metadata and avoiding loading entire entry contents into memory.

Changes:

  • Rework R8Minimizer.normalizeJar to stream entry bytes from the input zip instead of materializing them, and preserve each entry’s Unix mode.
  • Ensure META-INF/MANIFEST.MF is written as the first file entry in the normalized jar.
  • Add a focused regression test validating manifest ordering and executable permissions preservation.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
src/test/kotlin/com/github/jengelman/gradle/plugins/shadow/internal/R8MinimizerTest.kt Adds coverage for manifest-first ordering and Unix mode preservation during normalization.
src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/internal/Zip.kt Adds UnixMode.raw(...) to allow writing preserved Unix modes verbatim.
src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/internal/R8Minimizer.kt Streams jar normalization, preserves Unix permissions, and forces manifest to be first file entry.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Suppressed comments (2)

src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/tasks/ShadowCopyAction.kt:139

  • ShadowCopyAction still accepts an encoding parameter, but the StreamAction no longer applies it to the provided ZipOutputStream. This makes the encoding argument ineffective for callers that provide a custom zosProvider, which is a behavioral/API regression.
  private inner class StreamAction(private val zipOutStr: ZipOutputStream) :
    CopyActionProcessingStreamAction {
    init {
      logger.info("Relocator count: {}.", relocators.size)
    }

src/test/kotlin/com/github/jengelman/gradle/plugins/shadow/internal/R8MinimizerTest.kt:71

  • The test will currently fail with an NPE if the entry is missing, which obscures the cause. Adding an explicit null check gives a clearer failure message.
    ZipFile(outputJar).use { zipFile ->
      // Executable unix mode must be preserved
      val scriptEntry = zipFile.getEntry("bin/script.sh")
      assertThat(scriptEntry.unixMode).isEqualTo(expectedExecutableMode)
    }

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@Goooler
Goooler force-pushed the g/20260809/stop-normalizing branch from 7a8bec9 to 040b005 Compare August 9, 2026 14:16
@Goooler
Goooler force-pushed the g/20260809/stop-normalizing branch from 040b005 to ae7b42b Compare August 9, 2026 14:19
@Goooler
Goooler merged commit 1b993b1 into main Aug 9, 2026
6 checks passed
@Goooler
Goooler deleted the g/20260809/stop-normalizing branch August 9, 2026 14:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants