Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Java] Enable Java Build Caching #39707

Closed
clayburn opened this issue Jan 19, 2024 · 0 comments · Fixed by #39708
Closed

[Java] Enable Java Build Caching #39707

clayburn opened this issue Jan 19, 2024 · 0 comments · Fixed by #39708

Comments

@clayburn
Copy link
Contributor

Describe the enhancement requested

This enhancement was discussed at the Dec 2023 Infra Roundtable. Arrow was identified as a project that would be interested in build caching.

The proposed enhancement would connect the Java/Maven portion of the Arrow build to the Develocity instance at https://ge.apache.org. As a result of this, local build caching would become available to the Java/Maven portion of the Arrow build.

Component(s)

Java

@kou kou changed the title Enable Java Build Caching [Java] Enable Java Build Caching Jan 19, 2024
lidavidm pushed a commit that referenced this issue Mar 22, 2024
### Rationale for this change

This change has two main benefits:

#### Enabling local build caching

Enabling local build cache can speed up builds that occur on the same machine by skipping the execution of certain deterministic goals, such as Java compilation, when no change has occurred. 

In the future, [https://ge.apache.org](ge.apache.org) will support remote build caching as well so that results can be shared between builds (e.g. ephemeral CI build agents).

#### Enabling build scans

This change enables the publishing of build scans of the Apache Arrow project to the Develocity instance at [ge.apache.org](https://ge.apache.org/), hosted by the Apache Software Foundation and run in partnership between the ASF and Gradle. This Develocity instance has all features and extensions enabled and is freely available for use by the Apache Arrow project and all other Apache projects. Currently, Maven-built projects such as [Pulsar](https://ge.apache.org/scans?search.buildToolType=maven&search.rootProjectNames=*Pulsar*&search.timeZoneId=America%2FChicago), [IoTDB](https://ge.apache.org/scans?search.buildToolType=maven&search.rootProjectNames=*iotdb*&search.timeZoneId=America%2FChicago), [Ozone](https://ge.apache.org/scans?search.buildToolType=maven&search.rootProjectNames=*ozone*&search.timeZoneId=America%2FChicago), and others are using this instance.

On this Develocity instance, Apache Arrow will have access not only to all of the published build scans but other aggregate data features such as:

- Dashboards to view all historical build scans, along with performance trends over time
- Build failure analytics for enhanced investigation and diagnosis of build failures
- Test failure analytics to better understand trends and causes around slow, failing, and flaky tests

### What changes are included in this PR?

- Adds the [Develocity Maven Extension](https://docs.gradle.com/enterprise/maven-extension/) to enable local caching
- Publishes build scans to [ge.apache.org](https://ge.apache.org) from CI builds and authenticated local builds
- Adds the [Common Custom User Data Maven Extension](https://github.com/gradle/common-custom-user-data-maven-extension) to enhance build scans with more metadata

### Are these changes tested?

No (these changes are for the build and do not affect the code)

### Are there any user-facing changes?

No
* Closes: #39707

Authored-by: Clay Johnson <cjohnson@gradle.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
@lidavidm lidavidm added this to the 16.0.0 milestone Mar 22, 2024
lriggs pushed a commit to lriggs/arrow that referenced this issue Mar 22, 2024
…pache#39708)

### Rationale for this change

This change has two main benefits:

#### Enabling local build caching

Enabling local build cache can speed up builds that occur on the same machine by skipping the execution of certain deterministic goals, such as Java compilation, when no change has occurred. 

In the future, [https://ge.apache.org](ge.apache.org) will support remote build caching as well so that results can be shared between builds (e.g. ephemeral CI build agents).

#### Enabling build scans

This change enables the publishing of build scans of the Apache Arrow project to the Develocity instance at [ge.apache.org](https://ge.apache.org/), hosted by the Apache Software Foundation and run in partnership between the ASF and Gradle. This Develocity instance has all features and extensions enabled and is freely available for use by the Apache Arrow project and all other Apache projects. Currently, Maven-built projects such as [Pulsar](https://ge.apache.org/scans?search.buildToolType=maven&search.rootProjectNames=*Pulsar*&search.timeZoneId=America%2FChicago), [IoTDB](https://ge.apache.org/scans?search.buildToolType=maven&search.rootProjectNames=*iotdb*&search.timeZoneId=America%2FChicago), [Ozone](https://ge.apache.org/scans?search.buildToolType=maven&search.rootProjectNames=*ozone*&search.timeZoneId=America%2FChicago), and others are using this instance.

On this Develocity instance, Apache Arrow will have access not only to all of the published build scans but other aggregate data features such as:

- Dashboards to view all historical build scans, along with performance trends over time
- Build failure analytics for enhanced investigation and diagnosis of build failures
- Test failure analytics to better understand trends and causes around slow, failing, and flaky tests

### What changes are included in this PR?

- Adds the [Develocity Maven Extension](https://docs.gradle.com/enterprise/maven-extension/) to enable local caching
- Publishes build scans to [ge.apache.org](https://ge.apache.org) from CI builds and authenticated local builds
- Adds the [Common Custom User Data Maven Extension](https://github.com/gradle/common-custom-user-data-maven-extension) to enhance build scans with more metadata

### Are these changes tested?

No (these changes are for the build and do not affect the code)

### Are there any user-facing changes?

No
* Closes: apache#39707

Authored-by: Clay Johnson <cjohnson@gradle.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants