Skip to content

Commit

Permalink
[110] Upgrade Weaver-Webservice-Core (2.1.1-RC8). (#111)
Browse files Browse the repository at this point in the history 
#####  Upgrade Weaver-Webservice-Core (2.1.1-RC8).

- Upgrade to Java 11.
- Upgrade all dependencies with vulnerabilities.
- Upgrade tests to use Junit5.

#####  Upgrade from Java 1.8 to Java 11 in Github Actions.

#####  Remove version, Github test fails.

Failure message:
```
Error:  Failed to execute goal org.codehaus.mojo:cobertura-maven-plugin:2.7:instrument (default-cli) on project ir-iiif-service: Execution default-cli of goal org.codehaus.mojo:cobertura-maven-plugin:2.7:instrument failed: Plugin org.codehaus.mojo:cobertura-maven-plugin:2.7 or one of its dependencies could not be resolved: Could not find artifact com.sun:tools:jar:0 at specified path /opt/hostedtoolcache/jdk/11.0.15/x64/../lib/tools.jar -> [Help 1]
```

The tests worked previously prior to adding a version number.
Remove the version number and hope it does not fail.

#####  Remove JDK11 incompatible cobertura and update coveralls.

Cobertura doesn't work under JDK11 and is not well maintained anymore.

Update Coveralls to work under JDK11.
This required bringing in javax.xml.bind:jaxb-api within the plugin of coveralls.

#####  Add comment why remaining deprecated function is not removed or replaced.

#####  Add Dockerfile.

It seems that the group id cannot be passed as a parameter because the `COPY --chown=` command does not support that.
Use a relatively high number, such as 3001 for the uid and gid to reduce potential id conflicts with the system loaded by docker.
Use a single variable ARG declaration for each variable and then import them into each stage.
This must be done because ARG is lost between stages and we really should avoid exposing these via ENV.

Use a custom work directory owned by a created user.
Run maven as that user to avoid doing anything as root (security improvement).
Run the service as that user from within that directory.
The jar is moved and renamed for convenience.

Maven (and possibly java) can be picky about arguments, so use the bracket notation for passing arguments to both maven and java.

##### Move the UID/GID into USER_ID argument and other Dockerfile changes.

Move the hardcoded 3001 into USER_ID argument.
Use `apt-get` instead of `apt`, they are apparently different programs.

It turns out the user does have to be recreated across stages.
Docker was subtly failing and prenenting to perform all of the other steps.

Only the following message is evidence of the problem.
```
linux spec user: unable to find user iriifservice: no matching entries in passwd file
```

##### When building Jar files, must specify repackage.
This is done either adding repackage to the pom.xml or by adding to the command line, like this:
```
  mvn clean package spring-boot:repackage -Pjar -DskipTests=true
```
This is spring and it should be transparent, so add it to the pom.i

##### Tweak log settings. Move `file:` yaml property up to be alphabetical in ordering.
Increase verbosity of logging to `WARN`.
We probably want to see any warnings when running tests as they may matter.

##### Remove bracket notation from RUN mvn command.
Do this for consistency reasons.
  • Loading branch information
@kaladay
kaladay committed May 3, 2022
1 parent cc53172 commit 6cf8255
Show file tree
Hide file tree
Showing 58 changed files with 879 additions and 600 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/build.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,10 @@ jobs:
- name: "Setup Java"
uses: actions/setup-java@v1
with:
java-version: 1.8
java-version: 11

- name: "Maven Tests"
run: mvn clean test cobertura:cobertura jacoco:report coveralls:report -DdryRun=true
run: mvn clean test jacoco:report coveralls:report -DdryRun=true

- name: "Coverage Report"
uses: MikeEdgar/github-action@raw_coverage_file
Expand Down
62 changes: 62 additions & 0 deletions Dockerfile
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Settings.
ARG USER_ID=3001
ARG USER_NAME=iriifservice
ARG HOME_DIR=/$USER_NAME
ARG SOURCE_DIR=$HOME_DIR/source

# Maven stage.
FROM maven:3-openjdk-11-slim as maven
ARG USER_ID
ARG USER_NAME
ARG HOME_DIR
ARG SOURCE_DIR

# Create the group (use a high ID to attempt to avoid conflits).
RUN groupadd -g $USER_ID $USER_NAME

# Create the user (use a high ID to attempt to avoid conflits).
RUN useradd -d $HOME_DIR -m -u $USER_ID -g $USER_ID $USER_NAME

# Update the system.
RUN apt-get update && apt-get upgrade -y

# Set deployment directory.
WORKDIR $SOURCE_DIR

# Copy files over.
COPY ./pom.xml ./pom.xml
COPY ./src ./src

# Assign file permissions.
RUN chown -R ${USER_ID}:${USER_ID} ${SOURCE_DIR}

# Login as user.
USER $USER_NAME

# Build.
RUN mvn package -Pjar -DskipTests=true

# Switch to Normal JRE Stage.
FROM openjdk:11-jre-slim
ARG USER_ID
ARG USER_NAME
ARG HOME_DIR
ARG SOURCE_DIR

# Create the group (use a high ID to attempt to avoid conflits).
RUN groupadd -g $USER_ID $USER_NAME

# Create the user (use a high ID to attempt to avoid conflits).
RUN useradd -d $HOME_DIR -m -u $USER_ID -g $USER_ID $USER_NAME

# Login as user.
USER $USER_NAME

# Set deployment directory.
WORKDIR $HOME_DIR

# Copy over the built artifact from the maven image.
COPY --from=maven $SOURCE_DIR/target/ROOT.jar ./iriif.jar

# Run java command.
CMD ["java", "-jar", "./iriif.jar"]
170 changes: 129 additions & 41 deletions pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@
<groupId>edu.tamu</groupId>
<artifactId>ir-iiif-service</artifactId>
<version>0.5.5</version>
<packaging>war</packaging>

<name>IR IIIF Service</name>

Expand All @@ -16,23 +15,44 @@
<parent>
<groupId>edu.tamu.weaver</groupId>
<artifactId>webservice-parent</artifactId>
<version>2.0.3</version>
<version>2.1.1-RC8</version>
<relativePath/>
</parent>

<properties>
<start-class>edu.tamu.iiif.IrIiifServiceInitializer</start-class>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
<jena-libs.version>3.2.0</jena-libs.version>
<java.version>11</java.version>
<jena-libs.version>4.4.0</jena-libs.version>
<iiif-presentation.version>3.2.6</iiif-presentation.version>
<maven.compiler.source>${java.version}</maven.compiler.source>
<maven.compiler.target>${java.version}</maven.compiler.target>
<maven-plugins.version>2.22.2</maven-plugins.version>
<!-- must be valid uri with scheme file or classpath -->
<!-- e.g. file:/var/ir-iiif-service/config, classpath:/config -->
<config.uri>classpath:/config/</config.uri>
<maven.packaging>war</maven.packaging>
</properties>

<packaging>${maven.packaging}</packaging>

<profiles>
<profile>
<id>war</id>
<properties>
<maven.packaging>war</maven.packaging>
</properties>
</profile>
<profile>
<id>jar</id>
<properties>
<maven.packaging>jar</maven.packaging>
</properties>
</profile>
</profiles>

<dependencies>

<dependency>
<groupId>edu.tamu.weaver</groupId>
<artifactId>messaging</artifactId>
Expand All @@ -52,6 +72,12 @@
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
<exclusions>
<exclusion>
<groupId>io.lettuce</groupId>
<artifactId>lettuce-core</artifactId>
</exclusion>
</exclusions>
</dependency>

<dependency>
Expand All @@ -78,35 +104,55 @@
</dependency>

<dependency>
<groupId>org.springframework.restdocs</groupId>
<artifactId>spring-restdocs-mockmvc</artifactId>
<scope>test</scope>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-configuration-processor</artifactId>
<optional>true</optional>
</dependency>

<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-test</artifactId>
<scope>test</scope>
<groupId>commons-validator</groupId>
<artifactId>commons-validator</artifactId>
<version>1.7</version>
</dependency>

<dependency>
<groupId>it.ozimov</groupId>
<artifactId>embedded-redis</artifactId>
<version>0.7.2</version>
<scope>test</scope>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.11.0</version>
</dependency>

<dependency>
<groupId>commons-validator</groupId>
<artifactId>commons-validator</artifactId>
<version>1.6</version>
<groupId>org.apache.commons</groupId>
<artifactId>commons-text</artifactId>
<version>1.9</version>
</dependency>

<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>31.1-jre</version>
</dependency>

<dependency>
<groupId>redis.clients</groupId>
<artifactId>jedis</artifactId>
</dependency>

<dependency>
<groupId>org.apache.jena</groupId>
<artifactId>apache-jena-libs</artifactId>
<type>pom</type>
<version>${jena-libs.version}</version>
<exclusions>
<exclusion>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
</exclusion>
<exclusion>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</exclusion>
</exclusions>
</dependency>

<dependency>
Expand All @@ -122,16 +168,63 @@
</dependency>

<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-configuration-processor</artifactId>
<optional>true</optional>
<groupId>org.springframework.restdocs</groupId>
<artifactId>spring-restdocs-mockmvc</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-test</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>it.ozimov</groupId>
<artifactId>embedded-redis</artifactId>
<version>0.7.3</version>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
</exclusion>
<exclusion>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</exclusion>
</exclusions>
</dependency>

<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-junit-jupiter</artifactId>
<scope>test</scope>
</dependency>
</dependencies>

<build>
<finalName>ROOT</finalName>
<plugins>

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-plugins.version}</version>
</plugin>

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>${maven-plugins.version}</version>
</plugin>

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
Expand All @@ -145,6 +238,13 @@
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
</plugin>

<plugin>
Expand All @@ -156,25 +256,6 @@
</configuration>
</plugin>

<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<configuration>
<quiet>true</quiet>
<instrumentation>
<ignoreTrivial>true</ignoreTrivial>
<ignores>
<ignore>java.util.logging.*</ignore>
</ignores>
</instrumentation>
<formats>
<format>html</format>
<format>xml</format>
</formats>
<aggregate>true</aggregate>
</configuration>
</plugin>

<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
Expand All @@ -194,6 +275,13 @@
<configuration>
<repoToken></repoToken>
</configuration>
<dependencies>
<dependency>
<groupId>javax.xml.bind</groupId>
<artifactId>jaxb-api</artifactId>
<version>2.3.1</version>
</dependency>
</dependencies>
</plugin>

<plugin>
Expand Down Expand Up @@ -268,7 +356,7 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-project-info-reports-plugin</artifactId>
<version>2.7</version>
<version>3.2.2</version>
<configuration>
<dependencyLocationsEnabled>false</dependencyLocationsEnabled>
</configuration>
Expand Down
2 changes: 1 addition & 1 deletion src/main/java/edu/tamu/iiif/IrIiifServiceInitializer.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.boot.web.support.SpringBootServletInitializer;
import org.springframework.boot.web.servlet.support.SpringBootServletInitializer;

/**
* Entry point to the IR IIIF service initializer.
Expand Down
4 changes: 2 additions & 2 deletions src/main/java/edu/tamu/iiif/config/WebMvcConfig.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@

import org.springframework.context.annotation.Configuration;
import org.springframework.web.method.support.HandlerMethodArgumentResolver;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurerAdapter;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurer;

import edu.tamu.iiif.controller.resolver.ManifestRequestArgumentResolver;

@Configuration
public class WebMvcConfig extends WebMvcConfigurerAdapter {
public class WebMvcConfig implements WebMvcConfigurer {

@Override
public void addArgumentResolvers(List<HandlerMethodArgumentResolver> argumentResolvers) {
Expand Down
10 changes: 5 additions & 5 deletions src/main/java/edu/tamu/iiif/constants/Constants.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,11 @@ public class Constants {
public final static String IMAGE_IDENTIFIER = "image";

// Manifest Controller Mappings
public final static String COLLECTION_MAPPING = "/" + COLLECTION_IDENTIFIER + "/**/*";
public final static String PRESENTATION_MAPPING = "/" + PRESENTATION_IDENTIFIER + "/**/*";
public final static String SEQUENCE_MAPPING = "/" + SEQUENCE_IDENTIFIER + "/**/*";
public final static String CANVAS_MAPPING = "/" + CANVAS_IDENTIFIER + "/**/*";
public final static String IMAGE_MAPPING = "/" + IMAGE_IDENTIFIER + "/**/*";
public final static String COLLECTION_MAPPING = "/" + COLLECTION_IDENTIFIER + "/**";
public final static String PRESENTATION_MAPPING = "/" + PRESENTATION_IDENTIFIER + "/**";
public final static String SEQUENCE_MAPPING = "/" + SEQUENCE_IDENTIFIER + "/**";
public final static String CANVAS_MAPPING = "/" + CANVAS_IDENTIFIER + "/**";
public final static String IMAGE_MAPPING = "/" + IMAGE_IDENTIFIER + "/**";

// Fedora
public final static String FEDORA_FCR_METADATA = "/fcr:metadata";
Expand Down
2 changes: 1 addition & 1 deletion src/main/java/edu/tamu/iiif/controller/AbstractManifestController.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
import javax.servlet.http.HttpServletResponse;

import org.apache.commons.io.IOUtils;
import org.apache.commons.lang3.StringEscapeUtils;
import org.apache.commons.text.StringEscapeUtils;
import org.springframework.beans.factory.annotation.Autowired;

import edu.tamu.iiif.service.ManifestService;
Expand Down
Loading

0 comments on commit 6cf8255

Please sign in to comment.