Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gradle crashes if GRADLE_RO_DEP_CACHE is set and it cannot create modules-2 directory within it #12293

Closed
maartenh opened this issue Feb 20, 2020 · 11 comments · Fixed by #12408
Closed
Assignees
Milestone

Comments

@maartenh
Copy link

@maartenh maartenh commented Feb 20, 2020

Expected Behavior

If the GRADLE_RO_DEP_CACHE is set, the build should never fail because it cannot be written to.

Current Behavior

When I create a directory for the shared read-only cache, and make it available as an empty read-only directory, with no modules-2 subdirectory present, the build crashes when it tries to create the modules-2 subdirectory.
Please find the full failure report including stacktrace in the attached file.

Context

I'm using Jenkins 2 with builds occurring in ephemeral containers. Using directories mapped from the host server into the container for the read-only dependency cache.
The goal is to minimise the required downloads and the build time increases caused by those downloads.
My current workaround for this issue is the pre-create the modules-2 directory, but that will cause a failure if Gradle ever moves to a modules-3 version.

Steps to Reproduce

Please run the following script to reproduce:

mkdir gradle-shared-cache-test
cd gradle-shared-cache-test
mkdir shared-cache
chmod 444 shared-cache
export GRADLE_RO_DEP_CACHE="`pwd`/shared-cache"
gradle init --stacktrace --info

Your Environment

Gradle 6.2
Running on Linux in docker container, or MacOS 10.15.3 locally.

@maartenh
Copy link
Author

@maartenh maartenh commented Feb 20, 2020

@melix
Copy link
Contributor

@melix melix commented Feb 20, 2020

Thanks for the report. Maybe it's unclear but the shared read-only directory should always be pre-populated. You mustn't have one build with write access to it and others with read-access: all builds must have read-only access so don't let any Gradle build write directly to it.

@maartenh
Copy link
Author

@maartenh maartenh commented Feb 20, 2020

Thanks for the report. Maybe it's unclear but the shared read-only directory should always be pre-populated. You mustn't have one build with write access to it and others with read-access: all builds must have read-only access so don't let any Gradle build write directly to it.

Thank you. The requirement for having it pre-populated was not clear to me from the documentation.
And indeed, I do give all Gradle builds read-only access to it. The intend (once I see actual build time improvements from it) is to have only the master branch build update the cache after the Gradle build is finished.

@ljacomet ljacomet added this to the 6.2.1 milestone Feb 20, 2020
@ljacomet
Copy link
Member

@ljacomet ljacomet commented Feb 24, 2020

Explicit notes have been added to the documentation to clarify the need of having a populated read-only cache.

@ChristianCiach
Copy link

@ChristianCiach ChristianCiach commented Feb 25, 2020

I ran into the same issue and I still think it's bad that Gradle crashes if the directory is empty.

This directory is always read-only. This is even reflected by the property name. So I don't see why Gradle should ever even attempt to write to it (by attempting to create directories). I think Gradle should just behave as if the property hasn't been set (maybe logging a warning or info) if the directory is empty, doesn't exist at all or cannot be used for any other reason.

My use case: I tried to define "GRADLE_RO_DEP_CACHE" at the Jenkins system configuration, making it available to all builds. This works fine for all builds running directly on the host, but unfortunately, this immediately caused all builds that are executed inside a docker container to fail. I really think Gradle should just swallow a non-existing (sub)directory in this case.

@ljacomet
Copy link
Member

@ljacomet ljacomet commented Feb 25, 2020

That's a fair point. Let us work on fixing that for the next release.

@ljacomet ljacomet reopened this Feb 25, 2020
@ljacomet ljacomet removed this from the 6.2.1 milestone Feb 25, 2020
@ljacomet ljacomet added this to the 6.3 RC1 milestone Feb 25, 2020
@melix
Copy link
Contributor

@melix melix commented Feb 25, 2020

-1 to silently failing. This indicates a mis-configuration. It's important to fix the configuration problem. We can avoid crashing, but we shouldn't silence problems.

@ChristianCiach
Copy link

@ChristianCiach ChristianCiach commented Feb 25, 2020

-1 to silently failing. This indicates a mis-configuration.

Of course I agree with you! This was badly worded on my part. By "swallowing" I just meant "not crashing". A log message is perfectly adequate for this situation.

@prakashul
Copy link

@prakashul prakashul commented Feb 27, 2020

Hi folks,
I am facing a related issue. I have pre-populated the to be read-only dependency directory. I made it read-only and set variables GRADLE_HOME (separate writable local cache dir) and GRADLE_RO_DEP_CACHE (for read-only cache).
Turns out Gradle tries to rewrite modules-2 dir in GRADLE_RO_DEP_CACHE and ignores and/or overrides the GRADLE_HOME variable. Thus the process crashes with exceptions of failed to write to the read-only directory. The concept of local cache (writable) and a read-only cache is not honoured.

Gradle version used: 6.2.1

@ljacomet
Copy link
Member

@ljacomet ljacomet commented Feb 27, 2020

Hey @prakashul ,

Could you share more specific information about what you tried exactly?
Things like:

  1. What did you copy?
  2. Where did you copy it?
  3. Which value did you put for GRADLE_RO_DEP_CACHE?

For example, this would be a valid setup:

  1. Copy /users/foo/.gradle/caches/modules-2
  2. Into /shared/gradle-cache/ so I have /shared/gradle-cache/modules-2
  3. Add GRADLE_RO_DEP_CACHE=/shared/gradle-cache/ to system env

@prakashul
Copy link

@prakashul prakashul commented Mar 2, 2020

Hi @ljacomet

Thanks for clearing that up, the solution worked. By your example, i had copied /users/foo/.gradle/caches to a new location.
Copying /users/foo/.gradle/caches/modules-2 and setting the env var to <CACHE_DIR> solved the problem.

Thanks :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants