Issue running RDFUnit Docker in a Kubernetes environment

[vemonet]

Expected Behavior

Using RDFUnit in a Kubernetes environment

Actual Behavior

When running bin/rdfunit from inside a Docker container running in a Kubernetes pod we are facing the following issue:

[ERROR] No plugin found for prefix 'exec' in the current project and in the plugin groups [org.apache.maven.plugins, org.codehaus.mojo] available from the repositories [local (/root/.m2/
repository), central (https://repo.maven.apache.org/maven2)] -> [Help 1]

Steps to Reproduce the Problem

1. Install kubectl and Argo on local machine (following this tutorial: https://maastrichtu-ids.github.io/dsri-documentation/docs/guide-local-install#install-kubectl )

2. Run aksw/rdfunit in a Kubernetes pod
   It can be done reusing this pod definition: https://github.com/MaastrichtU-IDS/d2s-argo-workflows/blob/cd8b1432940595e6ff52b7efaa339f5d653aa609/tests/test-devnull-pod.yaml

3. Connect to it to run bin/rdfunit

Specifications

• Version: Kubernetes
  Client Version: version.Info{Major:"1", Minor:"11+", GitVersion:"v1.11.0+d4cacc0", GitCommit:"d4cacc0", GitTreeState:"clean", BuildDate:"2018-10-10T16:38:01Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}
  Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.0", GitCommit:"70132b0f130acc0bed193d9ba59dd186f0e634cf", GitTreeState:"clean", BuildDate:"2019-12-07T21:12:17Z", GoVersion:"go1.13.4", Compiler:"gc", Platform:"linux/amd64"}
• Platform: Ubuntu 18.04
• Subsystem:

Remarks

I realized there are a few issues in the Dockerfile such as a mkdir line that contains a typo

I think it can be solved by rewriting the Dockerfile to properly generates a packaged jar and using this jar after with java -jar (instead of using mvn -pl)

[vemonet]

Here is an example of a Dockerfile to build Java we use usually (1 stage with maven to build the jar, 1 stage with a light JRE to execute the jar : https://github.com/MaastrichtU-IDS/xml2rdf/blob/master/Dockerfile

Compiling a jar allow you to avoid any issues related to changing the path where the application is used within your container

Commands to run the test pod and connect to it (from the d2s-argo-workflow repo):

kubectl create -f tests/test-devnull-pod.yaml
kubectl exec -it test-devnull-pod -- /bin/bash

[vemonet]

I fixed RDFUnit to be packaged as a standalone jar in the RDFUnit Docker container. So using it from any path with any workdir set will work (available at https://hub.docker.com/repository/docker/umids/rdfunit )

What has been done:

• added the shade plugin from the profile cli-standalone to the default build plugins in the pom.xml (using profiles was not clear)
• rewriting the Dockerfile to build the standalone jar in a maven container before running it in a openjdk slim container
• use absolute path to run the jar in the Docker entrypoint

See commits:

• vemonet@cd57ffd
• vemonet@447c9d4

See the new Dockerfile
It is working perfectly fine, let me know if that interest you

We could avoid to change the pom.xml by selection the cli-standalone profile when running mvn package, but not sure how to do it

[jimkont]

Thanks for the report @vemonet
I agree that using a fat jar is a better option for running RDFUnit in docker

I like your change, my only concern is that by adding the shade plugin inside the default maven profile, it will be running for every maven execution and possibly delay other dev processes.

We could easily add it in a separate profile (i.e. standaloneJar) and add -PstandaloneJar in the docker command, wdyt?

[vemonet]

Hi @jimkont , yes that was my thought also

I rewrote the change to edit only the Dockerfile to properly use the cli-standalone profile when running mvn package

See commit vemonet@2452b1b

And see the Dockerfile: https://github.com/vemonet/RDFUnit/blob/develop/Dockerfile

I kept /usr/src/myapp as workdir, but we could use something shorter if you want, like /app

Additionally this also fix RDFUnit to be used in CWL workflows (CWL redefine the Docker workdir) 🙂

I did a pull request if that works for you: #99

[k00ni]

What is the status here?

[jimkont]

@vemonet can you check if we can close this issue? Is there anything else pending after your Merge request (#99)?

[vemonet]

@jimkont it is working as expected now, thanks a lot!