airbyte-ci: simplify gradle task execution #34560
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Ignored Deployment
|
postamar
force-pushed
the
postamar/airbyte-ci-simplify-gradle-steps
branch
from
7404d04
to
c954d28
Compare
|
@alafanechere I tested that this works manually and I couldn't find any downside to this change. |
| # Copy the local maven repository and force evaluation of `with_whole_git_repo` container. | ||
| .with_directory(self.LOCAL_MAVEN_REPOSITORY_PATH, await with_whole_git_repo.directory(self.LOCAL_MAVEN_REPOSITORY_PATH)) | ||
| # Copy the gradle home directory and force evaluation of `with_whole_git_repo` container. | ||
| .with_directory(self.GRADLE_HOME_PATH, await with_whole_git_repo.directory(self.GRADLE_HOME_PATH)) |
There was a problem hiding this comment.
Just to be clear:
The .gradle folder from the host will not be mounted as it's excluded when calling get_repo_dir.
I think your intent is that self._get_gradle_command("help", *warm_dependency_cache_args) will seed .gradle. Which works.
Can we make sure that .gradle is empty before calling f"(rsync -a --stats --mkpath {self.GRADLE_DEP_CACHE_PATH}/ {self.GRADLE_HOME_PATH} || true)", , (with a without_directory call), it'll guarantee that any refactoring an the get_repo_dir method will not alter this behavior and lead to corrupted .gradle folder mixing host file and container files.
Suspect IssuesThis pull request was deployed and Sentry observed the following issues:
Did you find this useful? React with a 👍 or 👎 |
The gradle task execution framework we built in airbyte-ci includes a
:airbyte-cdk:java:airbyte-cdk:publishSnapshotIfNeededtask which never actually did anything. It was there in anticipation of the java CDK workflow making use of SNAPSHOT releases. This never materialized, and considering where @stephane-airbyte and I are taking the CDK release workflow, this won't materialize for a while at least. We can therefore remove anything to do with the local maven repository, aka/root/.m2. This simplifies and speeds up gradle in dagger/airbyte-ci.Relates to airbytehq/airbyte-internal-issues#2475 .