Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spike: capture latency of build cache HTTP requests #29146

Closed
wants to merge 1 commit into from

Conversation

lptr
Copy link
Member

@lptr lptr commented May 14, 2024

Capture the latency of HTTP requests to the build cache as a new build operation.

Findings:

  • It is possible to capture the processing of the request in a build operation in HttpBuildCacheService, which uses Apache HttpClient 4.x.
  • The response include a BasicHttpEntity; this has an 8 KB buffer, which will be read as part of the captured build operation. For small cache artifacts this means that the entire artifact's download time would be part of the measured latency.

If we want to use this approach, we need to introduce a new build operation type in :enterprise-operations. We'll need to reuse this same type in the Develocity HTTP cache client, too, and the scans plugin needs to handle this build operation going forward.

Reviewing cheatsheet

Before merging the PR, comments starting with

  • ❌ ❓must be fixed
  • 🤔 💅 should be fixed
  • 💭 may be fixed
  • 🎉 celebrate happy things

@lptr lptr added a:chore Minor issue without significant impact in:build-cache labels May 14, 2024
@lptr lptr added this to the 8.9 RC1 milestone May 14, 2024
@lptr lptr self-assigned this May 14, 2024
@lptr lptr force-pushed the lptr/build-cache/capture-latency branch from dd888db to 2895a8c Compare May 14, 2024 19:29
@lptr lptr force-pushed the lptr/build-cache/capture-latency branch from 1b142b2 to 5827114 Compare May 15, 2024 09:24
@lptr
Copy link
Member Author

lptr commented May 15, 2024

@bot-gradle test QFL please.

@bot-gradle
Copy link
Collaborator

I've triggered the following builds for you. Click here to see all build failures.

@lptr lptr changed the title Capture latency of build cache HTTP requests Spike: capture latency of build cache HTTP requests May 15, 2024
public BuildOperationDescriptor.Builder description() {
return BuildOperationDescriptor.displayName("HTTP " + request.getMethod() + " request for cache key " + key.getHashCode());
}
});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What kind of build operation is emitted in this case? Does it correlate somehow with load/store operation that's also emitted in this context

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now this is just an anonymous operation, but as mentioned in the description, we'll need to create a BuildOperationType to be able to properly identify it in the scan plugin.

This build operation is nested within the relevant remote load / remote store build operation.

@lptr
Copy link
Member Author

lptr commented May 27, 2024

Closing this as the experiment is now concluded.

@lptr lptr closed this May 27, 2024
@github-actions github-actions bot removed this from the 8.9 RC1 milestone May 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:chore Minor issue without significant impact in:build-cache
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants