Skip to content
This repository has been archived by the owner on Mar 27, 2024. It is now read-only.

Migration of BATS to ICE #478

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
98 changes: 98 additions & 0 deletions org.eclipse.ice.bats/.gitignore
@@ -0,0 +1,98 @@

# Created by https://www.toptal.com/developers/gitignore/api/java,eclipse
# Edit at https://www.toptal.com/developers/gitignore?templates=java,eclipse

### Eclipse ###
.metadata
bin/
tmp/
*.tmp
*.bak
*.swp
*~.nib
local.properties
.settings/
.loadpath
.recommenders

# External tool builders
.externalToolBuilders/

# Locally stored "Eclipse launch configurations"
*.launch

# PyDev specific (Python IDE for Eclipse)
*.pydevproject

# CDT-specific (C/C++ Development Tooling)
.cproject

# CDT- autotools
.autotools

# Java annotation processor (APT)
.factorypath

# PDT-specific (PHP Development Tools)
.buildpath

# sbteclipse plugin
.target

# Tern plugin
.tern-project

# TeXlipse plugin
.texlipse

# STS (Spring Tool Suite)
.springBeans

# Code Recommenders
.recommenders/

# Annotation Processing
.apt_generated/
.apt_generated_test/

# Scala IDE specific (Scala & Java development for Eclipse)
.cache-main
.scala_dependencies
.worksheet

# Uncomment this line if you wish to ignore the project description file.
# Typically, this file would be tracked if it contains build/dependency configurations:
#.project

### Eclipse Patch ###
# Spring Boot Tooling
.sts4-cache/

### Java ###
# Compiled class file
*.class

# Log file
*.log

# BlueJ files
*.ctxt

# Mobile Tools for Java (J2ME)
.mtj.tmp/

# Package Files #
*.jar
*.war
*.nar
*.ear
*.zip
*.tar.gz
*.rar

# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml
hs_err_pid*

# End of https://www.toptal.com/developers/gitignore/api/java,eclipse

target
122 changes: 122 additions & 0 deletions org.eclipse.ice.bats/README.md
@@ -0,0 +1,122 @@
# BATS

BATS, the Basic Artifact Tracking System (BATS), is a simple data management service for managing scientific data.

BATS is a standalone maven package that can be used within or outside of ICE.

BATS leverages [Apache Jena](https://jena.apache.org/index.html) to create [RDF Models](https://jena.apache.org/tutorials/rdf_api.html) and connects to [Apache Jena Fuseki](https://jena.apache.org/documentation/fuseki2/index.html) to publish these models.



## Build Instructions

### Prerequisites

BATS requires a full installation of Docker for building, executing, and storing images.


### Using Maven

Building the package can be performed like most other maven packages, where the end result is a jar file that can be included as a dependency
Due to the integration tests requiring a [Apache Jene Fuseki](https://jena.apache.org/documentation/fuseki2/index.html) server,
we need to first build the image using the Dockerfile located in `src/main/docker/Dockerfile`.
A test Fuseki container will be started and stopped during the tests using the [fabric8io docker-maven-plugin](https://dmp.fabric8.io/)
(GitHub repo link [here](https://github.com/fabric8io/docker-maven-plugin)

```
$ mvn clean docker:build install
```

This installs the jar file to the local repository in `~/.m2`. It is also possible to build the package without installing by running

```
$ mvn clean docker:build verify
```

In both cases one can skip the tests by including `-DskipTests` in your build.

## BATS API

Below are examples of a few general use cases for the BATS API

### Upload a new DataSet

One can create a connection to a Fuseki server to upload a [Dataset](https://jena.apache.org/documentation/javadoc/arq/org/apache/jena/query/Dataset.html), or a collection of named graphs (called [Models](https://jena.apache.org/documentation/javadoc/jena/org/apache/jena/rdf/model/Model.html) in Apache Jena).

Here we upload a new, empty Dataset called `my-new-dataset`
to our Fuseki server `http://my-fuseki-server.org:3030`.

The upload takes place when we issue the `.create()` method
for "creating" the data on the server.

```
import org.eclipse.ice.bats.DataSet;

DataSet dataset = new DataSet();
dataset.setName("my-new-dataset");
dataset.setHost("http://my-fuseki-server.org");
dataset.setPort(3030);
dataset.create();
```

### Upload a new RDF Model to a DataSet

We can also add a RDF [Model](https://jena.apache.org/documentation/javadoc/jena/org/apache/jena/rdf/model/Model.html) in Apache Jena,
to the Dataset on the Fuseki server as follows:

```
import org.eclipse.ice.bats.DataSet;
import org.apache.jena.rdf.model.Model;
import org.apache.jena.rdf.model.ModelFactory;

DataSet dataset = new DataSet();
dataset.setName("my-new-dataset");
dataset.setHost("http://my-fuseki-server.org");
dataset.setPort(3030);

Model model = ModelFactory.createDefaultModel();
dataset.updateModel("my-model", model);
```

### Pull a RDF Model from a DataSet

Following the previous example for uploading a RDF Model,
we can pull that same model using the following:

```
import org.eclipse.ice.bats.DataSet;
import org.apache.jena.rdf.model.Model;

DataSet dataset = new DataSet();
dataset.setName("my-new-dataset");
dataset.setHost("http://my-fuseki-server.org");
dataset.setPort(3030);

Model model = dataset.getModel("my-model");
```

### Get "raw" Jena Dataset

Sometimes we require the "raw" Jena dataset.
We can use the following to pull this dataset and check if it is "null"

```
import org.eclipse.ice.bats.DataSet;
import org.apache.jena.query.Dataset;

DataSet dataset = new DataSet();
dataset.setName("my-new-dataset");
dataset.setHost("http://my-fuseki-server.org");
dataset.setPort(3030);

Dataset rawDataset = dataset.getJenaDataset();
if ( rawDataset == null ) {
throw new Exception("DataSet Not Found");
}
```

## How BATS got its name

Jay Jay Bilings had a discussion with his daughter, 17 months old at the time, about her favorite animal.
marshallmcdonnell marked this conversation as resolved.
Show resolved Hide resolved
She picked the moose, but since that is already taken by several projects, they settled on her second favorite animal, the bat.
The name was then back-ronymed out of it.
115 changes: 115 additions & 0 deletions org.eclipse.ice.bats/pom.xml
@@ -0,0 +1,115 @@
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.eclipse.ice</groupId>
<artifactId>org.eclipse.ice.bats</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>Eclipse ICE BATS</name>
<description>Basic Artifact Tracking System</description>

<properties>
<java.version>11</java.version>
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.22.1</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>

<plugin>
<groupId>io.fabric8</groupId>
<artifactId>docker-maven-plugin</artifactId>
<version>0.34.1</version>
<configuration>
<images>
<image>
<name>bats-fuseki:latest</name>
<build>
<contextDir>${project.basedir}/src/main/docker</contextDir>
<dockerFile>Dockerfile.fuseki</dockerFile>
</build>
<run>
<ports>
<port>3030:3030</port>
</ports>
<volumes>
<bind>
<volume>/opt/fuseki-TDB:/data/TDB</volume>
</bind>
</volumes>
<network>
<mode>host</mode>
</network>
<wait>
<url>http://localhost:3030</url>
</wait>
</run>
</image>
</images>
</configuration>
<executions>
<execution>
<id>prepare-fuseki-database</id>
<phase>pre-integration-test</phase>
<goals>
<goal>start</goal>
</goals>
</execution>
<execution>
<id>remove-fuseki-database</id>
<phase>post-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>

</plugins>
</build>

<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.apache.jena</groupId>
<artifactId>apache-jena-libs</artifactId>
<type>pom</type>
<version>3.7.0</version>
</dependency>

<dependency>
<groupId>org.topbraid</groupId>
<artifactId>shacl</artifactId>
<version>1.1.0</version>
</dependency>

<!-- https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-core -->
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.11.2</version>
</dependency>

</dependencies>
</project>
20 changes: 20 additions & 0 deletions org.eclipse.ice.bats/src/main/docker/Dockerfile.fuseki
@@ -0,0 +1,20 @@
FROM openjdk:11.0.1-jre
marshallmcdonnell marked this conversation as resolved.
Show resolved Hide resolved

# Specify Fuseki variables
ARG FUSEKI_VERSION=3.16.0
ARG FUSEKI_NAME=apache-jena-fuseki
ARG FUSEKI_DOWNLOAD_FILE=$FUSEKI_NAME-$FUSEKI_VERSION.tar.gz

# Install Fuseki
RUN wget https://www-us.apache.org/dist/jena/binaries/$FUSEKI_DOWNLOAD_FILE && \
tar -xzvf $FUSEKI_DOWNLOAD_FILE && \
mv $FUSEKI_NAME-$FUSEKI_VERSION /opt/$FUSEKI_NAME && \
mkdir -p /opt/$FUSEKI_NAME/run/configuration

# Expose the Fuseki port
EXPOSE 3030

# Execute Fuseki from the installation directory
WORKDIR /opt/apache-jena-fuseki
ENTRYPOINT ["./fuseki-server"]