Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

License plugin reading incorrect artifact file might cause OOM #106

Closed
blindpirate opened this issue Oct 23, 2019 · 2 comments
Closed

License plugin reading incorrect artifact file might cause OOM #106

blindpirate opened this issue Oct 23, 2019 · 2 comments

Comments

@blindpirate
Copy link

blindpirate commented Oct 23, 2019

Context

Today I'm investigating an OOM after upgrade to Android Gradle Plugin 3.6.0-beta01, and finally found that it was caused by OssLicensesPlugin.

DependencyTask generates a dependencies JSON file from all dependency artifact files by ResolvedArtifact.file.getAbsolutePath(), then LicensesTask search for POM files based on the assumption of Maven repo layout.

However, sometimes ResolvedArtifact.file.getAbsolutePath() might return the transformed artifact file path, like ~/.gradle/caches/transforms-2/files-2.1/c592d6fca3e3510c0238d97e044b0eab/jetified-kotlin-stdlib-1.3.40.jar. In this case, the POM-searching-code will have serious performance issue - it's searching the transform directory ~/.gradle/caches/transforms-2, which has nearly 30000 files in my computer.

        FileTree tree = project.fileTree(
            dir: artifactFile.parentFile.parentFile, <--- This is not the Maven pom directory
            include: ["**/${pomFileName}", pomFileName])

Since Gradle has internal cache for file tree search patterns, there will be tens of thousands of useless cache entries, which might result in OOM easily.

image

To Reproduce

Run ./gradlew clean assemble in https://github.com/gradle/santa-tracker-android/tree/agp-3.6.0 branch you can see the OOM at the end of build.

On Gradle's side, we'll decrease the internal cache size, but I think we'd also better exclude tranformed dependencies because there's no POM at all.

blindpirate added a commit to gradle/santa-tracker-android that referenced this issue Oct 28, 2019
See google/play-services-plugins#106

It turns out that there's a performance issue in
Google's licenses plugin. Remove it until fixed.
blindpirate added a commit to blindpirate/play-services-plugins that referenced this issue Oct 31, 2019
Fixes google#106

Previously license plugin search pom files in file system, which is
inefficient and incorrect for artifact transform directory.

This PR fixes this issue by using Gradle artifact query API. It also
refactors some code.
@blindpirate
Copy link
Author

Seems to be closed by #104

@jkasnicki
Copy link
Collaborator

Released in com.google.android.gms:oss-licenses-plugin:0.10.1

See https://developers.google.com/android/guides/releases#january_13_2020

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants