Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build console with make bundle #6031

Merged
merged 1 commit into from
Feb 24, 2024

Conversation

chuckwagoncomputing
Copy link
Member

flock waits for a lock on a file; once it's unlocked, it locks it and holds the lock until the command is completed. This is necessary because parallel Gradle builds interfere with each other.

Green custom: https://github.com/chuckwagoncomputing/fw-custom-example/actions/runs/8020089442

@rusefillc
Copy link
Contributor

@chuckwagoncomputing do we expect this to be happy on Windows?

@rusefillc
Copy link
Contributor

This is necessary because parallel Gradle builds interfere with each other.

shall we disable parallel gradle? can you elaborate on the problem statement what is interfering with what how and when?

@chuckwagoncomputing
Copy link
Member Author

When you try to run a Gradle task when another one is already running, they fail in all sorts of ways. Making each Gradle worker do its work sequentially wouldn't solve the problem because you would still have multiple Gradle workers running at the same time. I have solved this by using flock to make sure only one is running at any time.

@rusefillc
Copy link
Contributor

@chuckwagoncomputing please ELI5 for me. Who is invoking multiple gradle instances in parallel?

@chuckwagoncomputing
Copy link
Member Author

If you pass the -j flag to GNU Make with a number, it will execute that many recipes in parallel. If multiple recipes call Gradle, it is likely that those recipes will be run in parallel. We call make -j$(nproc), where nproc returns the number of logical processors in your computer. For any modern computer, this is more than one. The end result is that we have multiple instances of Gradle running in parallel. However, I have solved this problem using flock.

@rusefillc
Copy link
Contributor

@rusefillc
Copy link
Contributor

I am concerned about long term maintainability of this complex bespoke solution

At least it should be documented as a code comment right next to magic

Ideally we need a much more official way of doing same. Not sure of " --no-daemon" is relevant but hopefully gradle has something official remedy for that?

random https://discuss.gradle.org/t/using-gradle-build-cache-in-parallel-ci-cd-pipeline-runs/44614

@mck1117
Copy link
Member

mck1117 commented Feb 24, 2024

hopefully gradle has something official remedy for that?

The official guidance is "please for the love of god don't call gradle from make"

@rusefillc
Copy link
Contributor

@chuckwagoncomputing FYI pick 2e6f218

@chuckwagoncomputing
Copy link
Member Author

do we expect this to be happy on Windows?

I expect it to work with Cygwin or WSL, if that's what you mean.

The official guidance is "please for the love of god don't call gradle from make"

@mck1117 Do you have a source, or can you explain why not?

@rusefillc rusefillc merged commit a9c94fc into rusefi:master Feb 24, 2024
65 checks passed
rusefillc pushed a commit that referenced this pull request Feb 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants