New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SOLR-16641 - Generate gradle.properties from gradlew #1320
Merged
Merged
Changes from 10 commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
e9aeeac
SOLR-16641 - Generate gradle.properties from template
colvinco b08c8a0
SOLR-16641 - Do generation from a java class
colvinco 4b82af5
Update ignores to not exclude buildSrc
colvinco ae62cb2
Fix GENERATOR_STATUS
colvinco a888803
Fix up formatting and move localSettings help into the properties file
colvinco 5a901b5
Tweak template comment
colvinco d9d6e83
Tweak template comment and bring back a simple helpLocalSettings
colvinco f84d494
Use string replacement instead
colvinco 4a1f767
Remove toolchain properties from template
colvinco 1d76b05
Exit if file exists
colvinco 89fdbe4
Update CHANGES.txt
colvinco File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
72 changes: 72 additions & 0 deletions
72
buildSrc/src/main/java/org/apache/lucene/gradle/GradlePropertiesGenerator.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,72 @@ | ||
/* | ||
* Licensed to the Apache Software Foundation (ASF) under one or more | ||
* contributor license agreements. See the NOTICE file distributed with | ||
* this work for additional information regarding copyright ownership. | ||
* The ASF licenses this file to You under the Apache License, Version 2.0 | ||
* (the "License"); you may not use this file except in compliance with | ||
* the License. You may obtain a copy of the License at | ||
* | ||
* http://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, software | ||
* distributed under the License is distributed on an "AS IS" BASIS, | ||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
* See the License for the specific language governing permissions and | ||
* limitations under the License. | ||
*/ | ||
package org.apache.lucene.gradle; | ||
|
||
import java.io.IOException; | ||
import java.nio.charset.StandardCharsets; | ||
import java.nio.file.Files; | ||
import java.nio.file.Path; | ||
import java.nio.file.Paths; | ||
import java.nio.file.StandardOpenOption; | ||
import java.util.Map; | ||
|
||
/** | ||
* Standalone class that generates a populated gradle.properties from a template. | ||
* | ||
* <p>Has no dependencies outside of standard java libraries | ||
*/ | ||
public class GradlePropertiesGenerator { | ||
public static void main(String[] args) { | ||
if (args.length != 2) { | ||
System.err.println("Usage: java GradlePropertiesGenerator.java <source> <destination>"); | ||
System.exit(2); | ||
} | ||
|
||
try { | ||
new GradlePropertiesGenerator().run(Paths.get(args[0]), Paths.get(args[1])); | ||
} catch (Exception e) { | ||
System.err.println("ERROR: " + e.getMessage()); | ||
System.exit(3); | ||
} | ||
} | ||
|
||
public void run(Path source, Path destination) throws IOException { | ||
if (!Files.exists(source)) { | ||
throw new IOException("template file not found: " + source); | ||
} | ||
if (Files.exists(destination)) { | ||
System.out.println(destination + " already exists, skipping generation."); | ||
janhoy marked this conversation as resolved.
Show resolved
Hide resolved
|
||
return; | ||
} | ||
|
||
// Approximate a common-sense default for running gradle/tests with parallel | ||
// workers: half the count of available cpus but not more than 12. | ||
var cpus = Runtime.getRuntime().availableProcessors(); | ||
var maxWorkers = (int) Math.max(1d, Math.min(cpus * 0.5d, 12)); | ||
var testsJvms = (int) Math.max(1d, Math.min(cpus * 0.5d, 12)); | ||
|
||
var replacements = Map.of("@MAX_WORKERS@", maxWorkers, "@TEST_JVMS@", testsJvms); | ||
|
||
System.out.println("Generating gradle.properties"); | ||
String fileContent = Files.readString(source, StandardCharsets.UTF_8); | ||
for (var entry : replacements.entrySet()) { | ||
fileContent = fileContent.replace(entry.getKey(), String.valueOf(entry.getValue())); | ||
} | ||
Files.writeString( | ||
destination, fileContent, StandardCharsets.UTF_8, StandardOpenOption.CREATE_NEW); | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,102 @@ | ||
############################# | ||
# Local developer settings # | ||
############################# | ||
# | ||
# The first invocation of any task in Solr gradlew will generate and save this project-local 'gradle.properties' file. | ||
# This file contains the defaults you may (but don't have to) tweak for your particular hardware (or taste). Note there | ||
# are certain settings in that file that may be _required_ at runtime for certain plugins (an example is the spotless/ | ||
# google java format plugin, which requires adding custom exports to JVM modules). gradlew only generates this file | ||
# if it's not already present (it never overwrites the defaults) -- occasionally you may have to manually delete (or move) | ||
# this file and regenerate from scratch. | ||
# | ||
# This is an overview of some of these settings. | ||
# | ||
############### | ||
# Parallelism # | ||
############### | ||
# | ||
# Gradle build can run tasks in parallel but by default it consumes all CPU cores which | ||
# is too optimistic a default for Solr tests. You can disable the parallelism | ||
# entirely or assign it a 'low' priority with these properties: | ||
# | ||
# org.gradle.parallel=[true, false] | ||
# org.gradle.priority=[normal, low] | ||
# | ||
# The default level of parallelism is computed based on the number of cores on | ||
# your machine (on the first run of gradle build). By default these are fairly conservative | ||
# settings (half the number of cores for workers, for example): | ||
# | ||
# org.gradle.workers.max=[X] | ||
# tests.jvms=[N <= X] | ||
# | ||
# The number of test JVMs can be lower than the number of workers: this just means | ||
# that two projects can run tests in parallel to saturate all the workers. The I/O and memory | ||
# bandwidth limits will kick in quickly so even if you have a very beefy machine bumping | ||
# it too high may not help. | ||
# | ||
# You can always override these settings locally using command line as well: | ||
# gradlew -Ptests.jvms=N --max-workers=X | ||
# | ||
############# | ||
# Test JVMS # | ||
############# | ||
# | ||
# Test JVMs have their own set of arguments which can be customized. These are configured | ||
# separately from the gradle workers, for example: | ||
# | ||
# tests.jvms=3 | ||
# tests.heapsize=512m | ||
# tests.minheapsize=512m | ||
# tests.jvmargs=-XX:+UseParallelGC -XX:TieredStopAtLevel=1 -XX:ActiveProcessorCount=1 | ||
# | ||
################# | ||
# Gradle Daemon # | ||
################# | ||
# | ||
# The gradle daemon is a background process that keeps an evaluated copy of the project | ||
# structure, some caches, etc. It speeds up repeated builds quite a bit but if you don't | ||
# like the idea of having a (sizeable) background process running in the background, | ||
# disable it. | ||
# | ||
# org.gradle.daemon=[true, false] | ||
# org.gradle.jvmargs=... | ||
############################################################################################# | ||
|
||
# UTF-8 as standard file encoding | ||
systemProp.file.encoding=UTF-8 | ||
|
||
# Set up gradle JVM defaults. | ||
# | ||
# We also open up internal compiler modules for spotless/ google java format. | ||
org.gradle.jvmargs=-Xmx1g -XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 \ | ||
--add-exports jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED \ | ||
--add-exports jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED \ | ||
--add-exports jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED \ | ||
--add-exports jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED \ | ||
--add-exports jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED | ||
|
||
# Run at normal priority, in parallel | ||
org.gradle.parallel=true | ||
org.gradle.priority=normal | ||
|
||
# This setting enables local task output caches. This will speed up | ||
# your local builds in most cases but will also consume disk space in your | ||
# gradle home. See SOLR-15603 for details. | ||
# org.gradle.caching=true | ||
|
||
# Silence gradle warnings. We'll deal with them when we upgrade the wrapper. | ||
org.gradle.warning.mode=none | ||
|
||
# You may disable the background daemon if it consumes too much memory. | ||
org.gradle.daemon=true | ||
# timeout after 15 mins of inactivity. | ||
org.gradle.daemon.idletimeout=900000 | ||
|
||
# Maximum number of parallel gradle workers. | ||
org.gradle.workers.max=@MAX_WORKERS@ | ||
|
||
# Maximum number of test JVMs forked per test task. | ||
tests.jvms=@TEST_JVMS@ | ||
|
||
# Disable auto JVM provisioning (we don't use toolchains yet but want no surprises). | ||
org.gradle.java.installations.auto-download=false |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just because this was based on a Lucene class doesn't mean this is a Lucene class. Should probably be packaged under o.a.solr.gradle?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1 -- analogous to this namespace issue. Not sure whether there are practical implications to the naming in this case, but regardless, solr package makes sense here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought about it too but thought we could leave it for another day. Although it would be natural to change when we're touching this area.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I only put it there because it's adjacent to the https://github.com/apache/solr/blob/1d76b053c4a6fb742a97f4250b332d755680d00e/buildSrc/src/main/java/org/apache/lucene/gradle/WrapperDownloader.java that's also being used in gradlew. In actual fact, neither of them belong in buildSrc, because they aren't used by the gradle scripts themselves. I just didn't want to do something different.
I could move them both to https://github.com/apache/solr/tree/1d76b053c4a6fb742a97f4250b332d755680d00e/gradle
Either into a new directory within it, or put the WrapperDownloader in https://github.com/apache/solr/tree/1d76b053c4a6fb742a97f4250b332d755680d00e/gradle/wrapper and the new generator in https://github.com/apache/solr/tree/1d76b053c4a6fb742a97f4250b332d755680d00e/gradle/generation?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove the package completely. As this is a command line class only and it is not even compiled, theres no need to have a package.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the reason it was originally placed there was so that it lived together with other "sources" - so that any validation checks, formatting, etc. would apply. I don't think we actually do it to buildSrc (haven't checked) but we could, in which case it'd be easier to maintain it as part of buildSrc than in some ad-hoc location.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we spin that refactoring out in a separate PR and merge this one, now that it is stable?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would keep the lucene namespace as at moment this makes merging cross source trees easier (I would like to port this over to Lucene, so cherry picking this commit works from there if the path names are similar). At moment I try to port most lucene and solr gradle build imüprovements to the other project, so aligned path names makes this easier.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does someone want to merge it then? :)