Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for offline and local builds #237

Closed
wants to merge 6 commits into from
Closed
Changes from all commits
Commits
File filter...
Filter file types
Jump to…
Jump to file or symbol
Failed to load files and symbols.

Always

Just for now

@@ -39,14 +39,14 @@ external Git source code hosting providers are available:
## Building

[JDK 12](http://jdk.java.net/12/) or later and [Gradle](https://gradle.org/)
5.2.1 or later is required for building. To build the project on macOS or
GNU/Linux, just run the following command from the source tree root:
5.6.2 or later is required for building. To build the project on macOS or
GNU/Linux x64, just run the following command from the source tree root:

```bash
$ sh gradlew
```

To build the project on Windows, run the following command from the source tree root:
To build the project on Windows x64, run the following command from the source tree root:

```bat
> gradlew
@@ -55,6 +55,42 @@ To build the project on Windows, run the following command from the source tree
The extracted jlinked image will end up in the `build` directory in the source
tree root.

### Other operating systems and CPU architectures

If you want to build on an operating system other than GNU/Linux, macOS or
Windows _or_ if you want to build on a CPU architecture other than x64, then
ensure that you have JDK 12 or later installed locally. You can then run the
following command from the source tree root:

```bash
$ sh gradlew
```

The extracted jlinked image will end up in the `build` directory in the source
tree root.

### Offline builds

If you don't want the build to automatically download any dependencies, then
you must ensure that you have installed the following software locally:

- JDK 12 or later
- Gradle 5.6.2 or later

To create a build then run the command:

```bash
$ gradle offline
```

_Please note_ that the above command does _not_ make use of `gradlew` to avoid
downloading Gradle.

The extracted jlinked image will end up in the `build` directory in the source
tree root.

### Cross-linking

It is also supported to cross-jlink jimages to GNU/Linux, macOS and/or Windows from
any of the aforementioned operating systems. To build all applicable jimages
(including the server-side tooling), run the following command from the
@@ -64,7 +64,7 @@ dependencies {
}

images {
linux {
linux_x64 {
modules = ['jdk.crypto.ec',
'org.openjdk.skara.bots.pr',
'org.openjdk.skara.bots.hgbridge',
@@ -104,19 +104,82 @@ reproduce {
dockerfile = 'test.dockerfile'
}

def getOS() {
def os = System.getProperty('os.name').toLowerCase()
if (os.startsWith('linux')) {
return 'linux'
}
if (os.startsWith('mac')) {
return 'macos'
}
if (os.startsWith('win')) {
return 'windows'
}
if (os.startsWith('sunos')) {
return 'solaris'
}
throw new GradleException("Unexpected operating system: " + os)
}

def getCPU() {
def cpu = System.getProperty('os.arch').toLowerCase()
This conversation was marked as resolved by edvbld

This comment has been minimized.

Copy link
@JornVernee

JornVernee Nov 5, 2019

I think this is "x86_64" on Mac for 64 bit platforms (ran into that for Panama as well). So, I think you need to check for cpu.startsWith('amd64') || cpu.startsWith('x86_64') below.

This comment has been minimized.

Copy link
@edvbld

edvbld Nov 6, 2019

Author Member

Fixed, also added a check for "x64".

if (cpu.startsWith('amd64') || cpu.startsWith('x86_64') || cpu.startsWith('x64')) {
return 'x64'
}
if (cpu.startsWith('x86') || cpu.startsWith('i386')) {
return 'x86'
}
if (cpu.startsWith('sparc')) {
return 'sparc'
}
if (cpu.startsWith('ppc')) {
return 'ppc'
}
if (cpu.startsWith('arm')) {
return 'arm'
}
This conversation was marked as resolved by edvbld

This comment has been minimized.

Copy link
@nick-arm

nick-arm Nov 7, 2019

Contributor

64-bit and 32-bit Arm have different os.arch values. So we need to add:

if (cpu.startsWith('aarch64')) {
    return 'aarch64'
}

This comment has been minimized.

Copy link
@edvbld

edvbld Nov 7, 2019

Author Member

Ah, right, thanks. Fixed!

if (cpu.startsWith('aarch64')) {
return 'aarch64';
}
throw new GradleException("Unexpected CPU: " + cpu)
}

task local(type: Copy) {
doFirst {
delete project.buildDir
}
def os = System.getProperty('os.name').toLowerCase()
def osName = os.startsWith('win') ? 'Windows' :
os.startsWith('mac') ? 'Macos' : 'Linux'

dependsOn ':cli:image' + osName
def os = getOS()
def cpu = getCPU()

if (os in ['linux', 'macos', 'windows'] && cpu == 'x64') {
def target = os.substring(0, 1).toUpperCase() + os.substring(1) +
cpu.substring(0, 1).toUpperCase() + cpu.substring(1)
dependsOn ':cli:image' + target
} else {
dependsOn ':cli:imageLocal'
}

from zipTree(file(project.rootDir.toString() +
'/cli/build/distributions/cli' +
'-' + project.version + '-' +
os + '-' + cpu + '.zip'))
into project.buildDir
}

task offline(type: Copy) {
doFirst {
delete project.buildDir
}

def os = getOS()
def cpu = getCPU()

dependsOn ':cli:imageLocal'
from zipTree(file(project.rootDir.toString() +
'/cli/build/distributions/cli' +
'-' + project.version + '-' +
osName.toLowerCase() + '.zip'))
os + '-' + cpu + '.zip'))
into project.buildDir
}

@@ -29,9 +29,52 @@
import org.gradle.api.artifacts.UnknownConfigurationException;

import java.util.ArrayList;
import java.util.HashSet;
import java.io.File;

public class ImagesPlugin implements Plugin<Project> {
private static String getOS() {
var p = System.getProperty("os.name").toLowerCase();
if (p.startsWith("win")) {
return "windows";
}
if (p.startsWith("mac")) {
return "macos";
}
if (p.startsWith("linux")) {
return "linux";
}
if (p.startsWith("sunos")) {
return "solaris";
}

throw new RuntimeException("Unknown operating system: " + System.getProperty("os.name"));
}

private static String getCPU() {
var p = System.getProperty("os.arch").toLowerCase();
if (p.startsWith("amd64") || p.startsWith("x86_64") || p.startsWith("x64")) {
return "x64";
}
if (p.startsWith("x86") || p.startsWith("i386")) {
return "x86";
}
if (p.startsWith("sparc")) {
return "sparc";
}
if (p.startsWith("ppc")) {
return "ppc";
}
if (p.startsWith("arm")) {
return "arm";
}
This conversation was marked as resolved by edvbld

This comment has been minimized.

Copy link
@nick-arm

nick-arm Nov 7, 2019

Contributor

Also need to check for aarch64 here:

if (p.startsWith("aarch64")) {
    return "aarch64";
}

This comment has been minimized.

Copy link
@edvbld

edvbld Nov 7, 2019

Author Member

Thanks, fixed this one as well!

if (p.startsWith("aarch64")) {
return "aarch64";
}

throw new RuntimeException("Unknown CPU: " + System.getProperty("os.arch"));
}

@Override
public void apply(Project project) {
NamedDomainObjectContainer<ImageEnvironment> imageEnvironmentContainer =
@@ -49,15 +92,23 @@ public ImageEnvironment create(String name) {

imageEnvironmentContainer.all(new Action<ImageEnvironment>() {
public void execute(ImageEnvironment env) {
var name = env.getName();
var subName = name.substring(0, 1).toUpperCase() + name.substring(1);
var parts = env.getName().split("_");;
var isLocal = parts.length == 1 && parts[0].equals("local");
var os = isLocal ? getOS() : parts[0];
var cpu = isLocal ? getCPU() : parts[1];
var osAndCpuPascalCased =
os.substring(0, 1).toUpperCase() + os.substring(1) +
cpu.substring(0, 1).toUpperCase() + cpu.substring(1);
var subName = isLocal ? "Local" : osAndCpuPascalCased;

var downloadTaskName = "download" + subName + "JDK";
project.getTasks().register(downloadTaskName, DownloadJDKTask.class, (task) -> {
task.getUrl().set(env.getUrl());
task.getSha256().set(env.getSha256());
task.getToDir().set(rootDir.resolve(".jdk"));
});
if (!isLocal) {
project.getTasks().register(downloadTaskName, DownloadJDKTask.class, (task) -> {
task.getUrl().set(env.getUrl());
task.getSha256().set(env.getSha256());
task.getToDir().set(rootDir.resolve(".jdk"));
});
}

var linkTaskName = "link" + subName;
project.getTasks().register(linkTaskName, LinkTask.class, (task) -> {
@@ -75,10 +126,15 @@ public void execute(ImageEnvironment env) {
// ignored
}

task.dependsOn(projectPath + ":" + downloadTaskName);
if (!isLocal) {
task.dependsOn(projectPath + ":" + downloadTaskName);
task.getUrl().set(env.getUrl());
} else {
task.getUrl().set("local");
}
task.getToDir().set(buildDir.resolve("images"));
task.getUrl().set(env.getUrl());
task.getOS().set(name);
task.getOS().set(os);
task.getCPU().set(cpu);
task.getLaunchers().set(env.getLaunchers());
task.getModules().set(env.getModules());
});
@@ -88,7 +144,8 @@ public void execute(ImageEnvironment env) {
task.getLaunchers().set(env.getLaunchers());
task.getOptions().set(env.getOptions());
task.getToDir().set(buildDir.resolve("launchers"));
task.getOS().set(name);
task.getOS().set(os);
task.getCPU().set(cpu);
});

var zipTaskName = "bundleZip" + subName;
@@ -99,7 +156,7 @@ public void execute(ImageEnvironment env) {
task.setPreserveFileTimestamps(false);
task.setReproducibleFileOrder(true);
task.getArchiveBaseName().set(project.getName());
task.getArchiveClassifier().set(name);
task.getArchiveClassifier().set(os + "-" + cpu);
task.getArchiveExtension().set("zip");

if (env.getMan().isPresent()) {
@@ -109,10 +166,11 @@ public void execute(ImageEnvironment env) {
});
}

task.from(buildDir.resolve("images").resolve(name), (s) -> {
var subdir = os + "-" + cpu;
task.from(buildDir.resolve("images").resolve(subdir), (s) -> {
s.into("image");
});
task.from(buildDir.resolve("launchers").resolve(name), (s) -> {
task.from(buildDir.resolve("launchers").resolve(subdir), (s) -> {
s.into("bin");
});
});
@@ -125,7 +183,7 @@ public void execute(ImageEnvironment env) {
task.setPreserveFileTimestamps(false);
task.setReproducibleFileOrder(true);
task.getArchiveBaseName().set(project.getName());
task.getArchiveClassifier().set(name);
task.getArchiveClassifier().set(os + "-" + cpu);
task.getArchiveExtension().set("tar.gz");
task.setCompression(Compression.GZIP);

@@ -136,10 +194,11 @@ public void execute(ImageEnvironment env) {
});
}

task.from(buildDir.resolve("images").resolve(name), (s) -> {
var subdir = os + "-" + cpu;
task.from(buildDir.resolve("images").resolve(subdir), (s) -> {
s.into("image");
});
task.from(buildDir.resolve("launchers").resolve(name), (s) -> {
task.from(buildDir.resolve("launchers").resolve(subdir), (s) -> {
s.into("bin");
});
});
@@ -37,13 +37,15 @@
public class LaunchersTask extends DefaultTask {
private Property<Path> toDir;
private Property<String> os;
private Property<String> cpu;
private MapProperty<String, String> launchers;
private ListProperty<String> options;

@Inject
public LaunchersTask(ObjectFactory factory) {
toDir = factory.property(Path.class);
os = factory.property(String.class);
cpu = factory.property(String.class);
launchers = factory.mapProperty(String.class, String.class);
options = factory.listProperty(String.class);
}
@@ -63,6 +65,11 @@ public LaunchersTask(ObjectFactory factory) {
return os;
}

@Input
Property<String> getCPU() {
return cpu;
}

@Input
MapProperty<String, String> getLaunchers() {
return launchers;
@@ -77,7 +84,7 @@ private static void clearDirectory(Path directory) throws IOException {

@TaskAction
void generate() throws IOException {
var dest = toDir.get().resolve(os.get());
var dest = toDir.get().resolve(os.get() + "-" + cpu.get());
if (Files.isDirectory(dest)) {
clearDirectory(dest);
}
ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.