Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add lifecycle event provider for the start of execution phase to FlowProviders #29064

Open
joshfriend opened this issue May 8, 2024 · 3 comments
Labels
a:feature A new functionality in:flow-actions FlowAction, FlowProvider, FlowScope, dataflow

Comments

@joshfriend
Copy link

Expected Behavior

I'd like to be able to run a dataflow action at the start of the execution phase. Intended use would be to analyze the user's build configuration and environment and report any errors by failing the build early and reporting to a telemetry endpoint. This cannot be done during configuration phase because I don't want environment variables, properties or shell commands collected/run by this analysis to be captured by the configuration cache.

Current Behavior (optional)

FlowProviders currently only provides getBuildWorkResult() which causes a dataflow action to execute at the end of a build. If no flow provider is given, the docs say that the dataflow action will be executed at an undefined point:

⚠️ If you’re not using a lifecycle event provider as an input to the dataflow action, then the exact timing when the action is executed is not defined and may change in the next version of Gradle.

Current implementation causes dataflow actions to be executed at the end of a build when no lifecycle event provider is used.

Context

We had previously implemented this as a regular task, but it is difficult to make the task always run for every build command. Adding the environment scan task as a prerequisite for all other tasks in every project eagerly creates all tasks during configuration phase and bloats the cache and slows configuration down. It is also incompatible with isolated projects.

@joshfriend joshfriend added a:feature A new functionality to-triage labels May 8, 2024
@ljacomet ljacomet added 👋 team-triage Issues that need to be triaged by a specific team in:build-operations in:configuration-cache Configuration Caching and removed to-triage in:build-operations labels May 14, 2024
@mlopatkin
Copy link
Member

This feature request is in the backlog of the relevant team and is prioritized by them.


This makes a lot of sense. There are two possible ways to implement this: execute immediately (at configuration time if not running from the configuration cache), or postpone until the execution phase actually starts. For the latter, we may expose some information about the configuration, like build finished provider does for the build result.

@mlopatkin mlopatkin removed the 👋 team-triage Issues that need to be triaged by a specific team label May 21, 2024
@bamboo bamboo added in:flow-actions FlowAction, FlowProvider, FlowScope, dataflow and removed in:configuration-cache Configuration Caching labels May 27, 2024
@alllex
Copy link
Member

alllex commented Jun 6, 2024

With Isolated Projects, the boundary between the "configuration phase" and "execution phase" can become blurry. At some point, a task in a project can start executing before all projects have been configured because the isolation is guaranteed.

How would such a change affect build configuration analysis?
More generally, do you expect to analyze each project in the build before it starts executing tasks or do you want to analyze "build level" information?

@joshfriend
Copy link
Author

More generally, do you expect to analyze each project in the build before it starts executing tasks or do you want to analyze "build level" information?

We only want to analyze build level info. This has included info about how many commits behind the main branch a user is working, or running launchctl limit maxfiles to see if developers have applied the workarounds we have for macOS file descriptor limit issues. If we run these during settings or configuration phase, they are captured by configuration cache and we see a much higher rate of CC invalidation (mostly from the git info changing). The previous implementation of making a task to collect/send this info and having all other tasks depend on it just bloated the configuration cache by eagerly configuring all tasks and causing cross-project configuration.

FlowProviders seems to be the only current solution to these issues but it can only run actions at the very end of the build. If it had an option to run at the start of a build, that would be awesome! My request for an after configuration hook was simply to be able to avoid capturing irrelevant info in the configuration cache.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:feature A new functionality in:flow-actions FlowAction, FlowProvider, FlowScope, dataflow
Projects
None yet
Development

No branches or pull requests

5 participants