Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release prql-java #781

Merged
merged 11 commits into from
Jul 12, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 29 additions & 11 deletions .github/workflows/release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -83,14 +83,32 @@ jobs:
working-directory: prql-js/
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
# - name: Bump npm version
# run: npm version patch
# working-directory: prql-js/
#
# - name: Commit version bump
# run: |
# git config user.name github-actions
# git config user.email github-actions@github.com
# git add prql-js/package.json
# git commit -m "Bump NPM version"
# git push
# - name: Bump npm version
# run: npm version patch
# working-directory: prql-js/
#
# - name: Commit version bump
# run: |
# git config user.name github-actions
# git config user.email github-actions@github.com
# git add prql-js/package.json
# git commit -m "Bump NPM version"
# git push

publish-prql-java:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3
- name: Install Java and Maven
uses: actions/setup-java@v1
with:
java-version: 8
- name: Release Maven package
uses: samuelmeuli/action-maven-publish@v1
with:
gpg_private_key: ${{ secrets.gpg_private_key }}
gpg_passphrase: ${{ secrets.gpg_passphrase }}
nexus_username: ${{ secrets.nexus_username }}
nexus_password: ${{ secrets.nexus_password }}
directory: prql-java/java/
2 changes: 0 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,3 @@ dist
.idea

target/

**/libprql_java*
4 changes: 4 additions & 0 deletions prql-java/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
target/

**/libprql_java*
**/prql_java.dll
54 changes: 54 additions & 0 deletions prql-java/DEVELOPMENT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# development description for prql-java module

---

## Implementation

We implement rust bindings to java with [jni](https://docs.oracle.com/javase/8/docs/technotes/guides/jni/).

First, define a native method -- `public static native String toSql(String query)` for PrqlCompiler, `toJson` is same.

And then implement it in rust with this [crate](https://docs.rs/jni/latest/jni/).

## Build

For ease of use to users, we need pre-build dynamic libs for different platforms. This process is combined into the build of java module.

We use [maven](https://maven.apache.org/) to build the java lib. To add the rust cross compilation into the maven build process, we add the following xml segment to the `pom.xml`:

```xml
<plugin>
<artifactId>exec-maven-plugin</artifactId>
<groupId>org.codehaus.mojo</groupId>
<version>1.6.0</version>
<executions>
<execution>
<id>Build for release</id>
<phase>generate-resources</phase>
<goals>
<goal>exec</goal>
</goals>
<configuration>
<executable>../cross.sh</executable>
<arguments>
<argument>${project.basedir}/../</argument>
</arguments>
</configuration>
</execution>
</executions>
</plugin>
```

When we build, it will execute the `cross.sh` script to get all the rust cdylibs. This process is time-consuming.

As to cross compilation toolchains, we use [cross](https://github.com/cross-rs/cross).

## Publish(for maintainer)

To publish the java lib to maven public repo,
project maintainer need first register a project in the maven nexus repo, by the doc:
https://central.sonatype.org/publish/publish-guide/.

And then, we can release our artifact in the `release` workflow.
The action we used is [action-maven-publish](https://github.com/marketplace/actions/action-maven-publish).
Project maintainer has to configure some personal information, those used in the first step, by the action's doc, such as `nexus_username`, `nexus_password`, `gpg_private_key`, `gpg_passphrase`.
27 changes: 27 additions & 0 deletions prql-java/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# prql-java

`prql-java` offers rust bindings to the `prql-compiler` rust library. It
exposes a java native method `public static native String toSql(String query)`.

## Installation

```xml
<dependency>
<groupId>org.prqllang</groupId>
<artifactId>prql-java</artifactId>
<version>${PRQL_VERSION}</version>
</dependency>
```

## Usage

```java
import org.prqllang.prql4j.PrqlCompiler;

class Main {
public static void main(String[] args) {
String sql = PrqlCompiler.toSql("from table");
System.out.println(sql);
}
}
```
61 changes: 61 additions & 0 deletions prql-java/cross.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
#!/bin/bash

PRQL_JAVA_MODULE=$1
echo PRQL_JAVA_MODULE="${PRQL_JAVA_MODULE}"
CONTEXT_PATH=$(pwd)
echo CONTEXT_PATH="${CONTEXT_PATH}"
cd "${PRQL_JAVA_MODULE}" || exit 1

# install cross
cargo install cross

# x86_64-unknown-linux-gnu
echo "compiling for x86_64-unknown-linux-gnu"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My guess is that this file could move to a workflow eventually; but to the extent you've been iterating on it locally, it looks good, and we can move to a workflow later.

rustup target add x86_64-unknown-linux-gnu
cross build --release --target x86_64-unknown-linux-gnu
ls -la ../target/x86_64-unknown-linux-gnu/release
cp -f ../target/x86_64-unknown-linux-gnu/release/libprql_java.so java/src/main/resources/libprql_java-linux64.so

## x86_64-unknown-linux-musl
#echo "compiling for x86_64-unknown-linux-musl"
#rustup target add x86_64-unknown-linux-musl
#cross build --release --target x86_64-unknown-linux-musl
#ls -la ../target/x86_64-unknown-linux-musl/release
#cp ../target/x86_64-unknown-linux-musl/release/libprql_java.so java/src/main/resources/libprql_java-linux64-musl.so

## x86_64-apple-darwin
#echo "compiling for x86_64-apple-darwin"
#rustup target add x86_64-apple-darwin
#cross build --release --target x86_64-apple-darwin
#ls -la ../target/x86_64-apple-darwin/release
#cp ../target/x86_64-apple-darwin/release/libprql_java.dylib java/src/main/resources/libprql_java-osx-x86_64.dylib

# x86_64-pc-windows-gnu
echo "compiling for x86_64-pc-windows-gnu"
rustup target add x86_64-pc-windows-gnu
cross build --release --target x86_64-pc-windows-gnu
ls -la ../target/x86_64-pc-windows-gnu/release
cp -f ../target/x86_64-pc-windows-gnu/release/prql_java.dll java/src/main/resources/libprql_java-win64.dll

# aarch64-unknown-linux-gnu
echo "compiling for aarch64-unknown-linux-gnu"
rustup target add aarch64-unknown-linux-gnu
cross build --release --target aarch64-unknown-linux-gnu
ls -la ../target/x86_64-unknown-linux-gnu/release
cp -f ../target/x86_64-unknown-linux-gnu/release/libprql_java.so java/src/main/resources/libprql_java-linux-aarch64.so

# aarch64-unknown-linux-musl
#echo "compiling for aarch64-unknown-linux-musl"
#rustup target add aarch64-unknown-linux-musl
#cross build --release --target aarch64-unknown-linux-musl
#ls -la ../target/aarch64-unknown-linux-musl/release
#cp -f ../target/aarch64-unknown-linux-musl/release/libprql_java.so java/src/main/resources/libprql_java-linux-aarch64-musl.so

## aarch64-apple-darwin
#echo "compiling for aarch64-apple-darwin"
#rustup target add aarch64-apple-darwin
#cross build --release --target aarch64-apple-darwin
#ls -la ../target/x86_64-apple-darwin/release
#cp -f ../target/x86_64-apple-darwin/release/libprql_java.dylib java/src/main/resources/libprql_java-osx-arm64.dylib

cd "${CONTEXT_PATH}" || exit 1
4 changes: 2 additions & 2 deletions prql-java/java/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ if [ "$KERNEL_NAME" = 'Linux' ]; then
else
target='libprql_java-linux32.so'
fi
cp -f ../target/release/libprql_java.so java/src/main/resources/${target}
cp -f ../target/release/libprql_java.so java/src/test/resources/${target}
elif [ "$KERNEL_NAME" = 'Darwin' ]; then
if [ "$ARCH" = 'arm64' ] || [ "$ARCH" = 'aarch64' ]; then
target='libprql_java-osx-arm64.dylib'
Expand All @@ -37,7 +37,7 @@ elif [ "$KERNEL_NAME" = 'Darwin' ]; then
echo [ERROR] have not support $ARCH:$$KERNEL_NAME yet
exit 1
fi
cp -f ../target/release/libprql_java.dylib java/src/main/resources/${target}
cp -f ../target/release/libprql_java.dylib java/src/test/resources/${target}
fi

ls -la ./java/src/main/resources
Loading