Merge pull request #353 from usdot-jpo-ode/stage

Stage --> Master Release 0.0.8

.travis.yml

@@ -1,36 +1,18 @@
 language: java
+
 jdk: oraclejdk8
-#branches:
-#  only:
-#    - master
-#    - develop
-sudo: false 
-#before_install:
-#  - echo "installing private repository"
-#  - git clone https://${BITBUCKET_UN_APP_PW}@bitbucket.org/usdot-jpo-ode/jpo-ode-private.git jpo-ode-private 
-#  - cd jpo-ode-private
-#  - mvn clean
-#  - mvn install
-#  - cd ..
-#  - echo "installing 1609.2 security library repository"
-#  - git clone https://github.com/usdot-jpo-ode/jpo-security.git fedgov-cv-security-2016 
-#  - cd fedgov-cv-security-2016
-#  - mvn clean
-#  - mvn -DskipTests install
-#  - cd ..
-#install: true #must disable pre-installation of dependencies because it fails due to missing oss.runtime dependency install 
+
+sudo: false
+
 addons:
   sonarcloud:
     organization: "usdot-jpo-ode"
     token:
       secure: $SONAR_SECURITY_TOKEN_NEW
     branches:
-      - .* 
+      - .*
+
 script:
   - mvn -e clean org.jacoco:jacoco-maven-plugin:prepare-agent package sonar:sonar
-  
+
 cache: false
-#cache:
-#  directories:
-#    - "$HOME/.m2/repository"
-#    - "$HOME/.sonar/cache"

README.md

@@ -1,6 +1,6 @@
-| Travis Build Status | Sonar Code Quality |
-|---------------------|---------------------|
- [![Build Status](https://travis-ci.org/usdot-jpo-ode/jpo-ode.svg?branch=master)](https://travis-ci.org/usdot-jpo-ode/jpo-ode) | [![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=usdot.jpo.ode%3Ajpo-ode&metric=alert_status)](https://sonarcloud.io/dashboard?id=usdot.jpo.ode%3Ajpo-ode) |
+| Travis Build Status | Sonar Code Quality | Sonar Code Coverage |
+|---------------------|---------------------|---------------------|
+ [![Build Status](https://travis-ci.org/usdot-jpo-ode/jpo-ode.svg?branch=master)](https://travis-ci.org/usdot-jpo-ode/jpo-ode) | [![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=usdot.jpo.ode%3Ajpo-ode&metric=alert_status)](https://sonarcloud.io/dashboard?id=usdot.jpo.ode%3Ajpo-ode) | [![Coverage](https://sonarcloud.io/api/project_badges/measure?project=usdot.jpo.ode%3Ajpo-ode&metric=coverage)](https://sonarcloud.io/dashboard?id=usdot.jpo.ode%3Ajpo-ode) |
 
 # jpo-ode
 
@@ -460,14 +460,16 @@ travis env set SONAR_SECURITY_TOKEN <key> -pr <user-account>/<repo-name>
 ######################################
  -->
 
-## Quality Assurance
-ODE provides an automated test harness that can be used for regression and user acceptance testing. 
-The test harness is pprovided in the [qa/test-harness](ga/test-harness) directory under jpo-ode root folder. 
-The test harness uses the ODE [Validator Library](https://github.com/usdot-jpo-ode/ode-output-validator-library) repository as a submodule.
+### Quality Assurance
+
+Code quality assurance is reported through the [usdot-jpo-ode SonarCloud organization](https://sonarcloud.io/organizations/usdot-jpo-ode/projects). Code quality reports are generated by the [JaCoCo plugin for Maven](https://www.eclemma.org/jacoco/trunk/doc/maven.html) during the ODE's [webhook-triggered TravisCI build](https://github.com/usdot-jpo-ode/jpo-ode/blob/dev/.travis.yml#L16). After a successful build, the [SonarQube scanner plugin for Maven](https://docs.sonarqube.org/display/SCAN/Analyzing+with+SonarQube+Scanner+for+Maven) creates and uploads a code quality report to SonarCloud.
+
+For regression and user acceptance testing, ODE provides an automated test harness. The test harness is pprovided in the [qa/test-harness](ga/test-harness) directory under jpo-ode root folder. The test harness uses the ODE [Validator Library](https://github.com/usdot-jpo-ode/ode-output-validator-library) repository as a submodule.
 
 For more information, please see: https://github.com/usdot-jpo-ode/jpo-ode/wiki/Using-the-ODE-test-harness
 
 <a name="known-bugs"/>
+
 ## 9. Known Bugs
 
 Date: 07/2017

ReleaseNotes.md

@@ -1,23 +1,31 @@
 ## ODE Release Notes ##
 -----------------------
-### Sprint 44
-- ODE-1234 Updated the odevalidator library with conditional configurability.
-- Release of jpo-ode-1.0.6
-### Sprint 43
+### Release 1.0.8 (May 24, 2019)
+- Integrated odevalidator v0.0.6
+- Updated test-harness (see [release notes](./qa/test-harness/README.md#release-history))
+- Changed SNMP deposit timestamp format from MM/DD/YYYY to YYYY/MM/DD
+### Release 1.0.7 (Apr 15, 2019)
+- Integrated odevalidator v0.0.3
+### Release 1.0.6 (Apr 8, 2019)
+- Integrated odevalidator v0.0.1
+  - ODE-1234 Updated the odevalidator library with conditional configurability.
+### Release 1.0.5 (Mar 20, 2019)
 - ODE-1209 Fixed missing bsmSource metadata field
 - ODE-944 Creates SDW Depositor microservice using the new SDW REST API
 - ODE-1056 Modifying ODE to support both WebSocket interface or SDW Depositor microservice by configurable properties
-### Sprint 41
+### Release 1.0.4 (Feb 12, 2019)
+- ODE-1136 Fixed metadata bug causing duplicate or invalid metadata elements (locationData, odeReceivedAt, recordGeneratedAt)
 - ODE-844 Fixed GitHub issue 264: New Content Type for TIMs desired by WYDOT
 - ODE-994 Add the ODE version number to the Demo UI
-### Sprint 40
+### Release 1.0.3 (Feb 7, 2019)
 - ODE-944 Added Situation Data Warehouse depositor submodule to the ODE
+### Release 1.0.2 (Jan 22, 2019)
 - ODE-1076 New values added for metadata.recordGeneratedBy: RSU, TMC_VIA_SAT, TMC_VIA_SNMP
-### December 2018
+### Release 1.0.1 (Dec 20, 2019)
 - ODE-466 serialId element of metadata will now provide a true serial number to help identify missing records.
 - ODE-964 Added ode.rsuUsername/ode.rsuPassword application properties or alternatively ODE_RSU_USERNAME and ODE_RSU_PASSWORD environment variables to authenticate communications with RSU instead of using the REST API parameters.
 - ODE-992 Added `/version` endpoint to get the version information of a running ODE.
-### November 2018
+### Release 1.0.0
 - ODE-787 Publish J2735 version of Broadcast TIM in addition to the REST request version (see GitHub Issue #262). Please see https://github.com/usdot-jpo-ode/jpo-ode/wiki/Schema-Version-6-Change-Notice for details.
 ### Sprint 38
 - ODE-769 Initiated output schema change SOP. See https://github.com/usdot-jpo-ode/jpo-ode/wiki/SchemaVersion-5-Change-Notice for details

docker-compose-ppm-nsv.yml

@@ -21,6 +21,7 @@ services:
       KAFKA_ADVERTISED_HOST_NAME: ${DOCKER_HOST_IP}
       KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
       KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"
+      KAFKA_CREATE_TOPICS: "topic.OdeBsmPojo:1:1,topic.OdeBsmJson:1:1,topic.FilteredOdeBsmJson:1:1,topic.OdeTimJson:1:1,topic.OdeTimBroadcastJson:1:1,topic.J2735TimBroadcastJson:1:1,topic.OdeDriverAlertJson:1:1,topic.Asn1DecoderInput:1:1,topic.Asn1DecoderOutput:1:1,topic.Asn1EncoderInput:1:1,topic.Asn1EncoderOutput:1:1,topic.SDWDepositorInput:1:1"
       KAFKA_DELETE_TOPIC_ENABLED: "true"
       KAFKA_CLEANUP_POLICY: "delete" # delete old logs
       KAFKA_LOG_RETENTION_HOURS: 2

docker-compose.yml

@@ -17,6 +17,7 @@ services:
       KAFKA_ADVERTISED_HOST_NAME: ${DOCKER_HOST_IP}
       KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
       KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"
+      KAFKA_CREATE_TOPICS: "topic.OdeBsmPojo:1:1,topic.OdeBsmJson:1:1,topic.FilteredOdeBsmJson:1:1,topic.OdeTimJson:1:1,topic.OdeTimBroadcastJson:1:1,topic.J2735TimBroadcastJson:1:1,topic.OdeDriverAlertJson:1:1,topic.Asn1DecoderInput:1:1,topic.Asn1DecoderOutput:1:1,topic.Asn1EncoderInput:1:1,topic.Asn1EncoderOutput:1:1,topic.SDWDepositorInput:1:1"
       KAFKA_DELETE_TOPIC_ENABLED: "true"
       KAFKA_CLEANUP_POLICY: "delete" # delete old logs
       KAFKA_LOG_RETENTION_HOURS: 2

jpo-ode-common/pom.xml

@@ -5,7 +5,7 @@
   <parent>
     <groupId>usdot.jpo.ode</groupId>
     <artifactId>jpo-ode</artifactId>
-    <version>1.0.7</version>
+    <version>0.0.8</version>
   </parent>
 
   <artifactId>jpo-ode-common</artifactId>

jpo-ode-core/pom.xml

@@ -5,7 +5,7 @@
   <parent>
     <groupId>usdot.jpo.ode</groupId>
     <artifactId>jpo-ode</artifactId>
-    <version>1.0.7</version>
+    <version>0.0.8</version>
   </parent>
 
   <artifactId>jpo-ode-core</artifactId>
@@ -23,12 +23,12 @@
     <dependency>
       <groupId>usdot.jpo.ode</groupId>
       <artifactId>jpo-ode-common</artifactId>
-      <version>1.0.7</version>
+      <version>0.0.8</version>
     </dependency>
     <dependency>
       <groupId>usdot.jpo.ode</groupId>
       <artifactId>jpo-ode-plugins</artifactId>
-      <version>1.0.7</version>
+      <version>0.0.8</version>
     </dependency>
     <dependency>
       <groupId>org.apache.httpcomponents</groupId>

jpo-ode-core/src/test/java/us/dot/its/jpo/ode/snmp/SNMPTest.java

@@ -74,6 +74,6 @@ public void testGettersAndSetters() {
    @Test
    public void testSnmpTimestampFromIso() throws ParseException {
       String snmpTS = SNMP.snmpTimestampFromIso("2017-05-04T21:55:00-05:00");
-      assertEquals("050414111537", snmpTS);
+      assertEquals("07E105041537", snmpTS);
    }
 }

jpo-ode-plugins/pom.xml

@@ -11,7 +11,7 @@
   <parent>
     <groupId>usdot.jpo.ode</groupId>
     <artifactId>jpo-ode</artifactId>
-    <version>1.0.7</version>
+    <version>0.0.8</version>
   </parent>
   <properties>
     <!-- SonarQube Properties -->
@@ -27,7 +27,7 @@
     <dependency>
       <groupId>usdot.jpo.ode</groupId>
       <artifactId>jpo-ode-common</artifactId>
-      <version>1.0.7</version>
+      <version>0.0.8</version>
     </dependency>
     <!-- TODO open-ode
     <dependency>

jpo-ode-plugins/src/main/java/us/dot/its/jpo/ode/plugin/SNMP.java

@@ -19,7 +19,6 @@
 import java.time.ZonedDateTime;
 
 import us.dot.its.jpo.ode.model.OdeObject;
-import us.dot.its.jpo.ode.util.CodecUtils;
 import us.dot.its.jpo.ode.util.DateTimeUtils;
 
 public class SNMP extends OdeObject {
@@ -127,14 +126,15 @@ public void setStatus(int status) {
 
    public static String snmpTimestampFromIso(String isoTimestamp) throws ParseException {
       ZonedDateTime zdt = DateTimeUtils.isoDateTime(isoTimestamp);
-      byte[] bdt = new byte[6];
-      bdt[0] = (byte) zdt.getMonthValue();
-      bdt[1] = (byte) zdt.getDayOfMonth();
-      bdt[2] = (byte) (zdt.getYear() / 100);
-      bdt[3] = (byte) (zdt.getYear() % 100);
-      bdt[4] = (byte) zdt.getHour();
-      bdt[5] = (byte) zdt.getMinute();
-      return CodecUtils.toHex(bdt);
+
+      StringBuilder sb = new StringBuilder();
+
+      sb.append(String.format("%04X", zdt.getYear()));
+      sb.append(String.format("%02X", zdt.getMonthValue()));
+      sb.append(String.format("%02X", zdt.getDayOfMonth()));
+      sb.append(String.format("%02X", zdt.getHour()));
+      sb.append(String.format("%02X", zdt.getMinute()));
+      return sb.toString();
    }
 
    @Override

jpo-ode-svcs/pom.xml

@@ -4,7 +4,7 @@
   <parent>
     <groupId>usdot.jpo.ode</groupId>
     <artifactId>jpo-ode</artifactId>
-    <version>1.0.7</version>
+    <version>0.0.8</version>
   </parent>
   <artifactId>jpo-ode-svcs</artifactId>
   <packaging>jar</packaging>
@@ -84,12 +84,12 @@
     <dependency>
       <groupId>usdot.jpo.ode</groupId>
       <artifactId>jpo-ode-core</artifactId>
-      <version>1.0.7</version>
+      <version>0.0.8</version>
     </dependency>
     <dependency>
       <groupId>usdot.jpo.ode</groupId>
       <artifactId>jpo-ode-plugins</artifactId>
-      <version>1.0.7</version>
+      <version>0.0.8</version>
     </dependency>
     <dependency>
       <groupId>org.springframework</groupId>

jpo-ode-svcs/src/main/java/us/dot/its/jpo/ode/services/asn1/Asn1EncodedDataRouter.java

@@ -191,9 +191,10 @@ public void processEncodedTim(ServiceRequest request, JSONObject consumedObj) {
             }
          }
 
-         logger.debug("Sending message to RSUs...");
          if (null != request.getSnmp() && null != request.getRsus() && null != hexEncodedTim) {
-            asn1CommandManager.sendToRsus(request, hexEncodedTim);
+            logger.info("Sending message to RSUs...");
+            Map<String, String> rsuResponseList = asn1CommandManager.sendToRsus(request, hexEncodedTim);
+            logger.info("TIM deposit response {}", rsuResponseList);
          }
 
          if (request.getSdw() != null) {
@@ -284,4 +285,4 @@ public void processEncodedTimUnsecured(ServiceRequest request, JSONObject consum
 
       logger.info("TIM deposit response {}", responseList);
    }
-}
+}

jpo-ode-svcs/src/test/java/us/dot/its/jpo/ode/snmp/SnmpSessionTest.java

@@ -134,8 +134,8 @@ public void testResponseEventSNMPException(@Mocked Snmp mockSnmp,
    @Test
    public void shouldCreatePDU() throws ParseException {
 
-      String expectedResult = "[1.0.15628.4.1.4.1.2.3 = 11, 1.0.15628.4.1.4.1.3.3 = 2, 1.0.15628.4.1.4.1.4.3 = 3, 1.0.15628.4.1.4.1.5.3 = 4, 1.0.15628.4.1.4.1.6.3 = 5, 1.0.15628.4.1.4.1.7.3 = 0c:02:14:11:11:2f, 1.0.15628.4.1.4.1.8.3 = 0c:02:14:11:11:2f, 1.0.15628.4.1.4.1.9.3 = 88, 1.0.15628.4.1.4.1.10.3 = 9, 1.0.15628.4.1.4.1.11.3 = 10]";
-      String expectedResult2 = "[1.0.15628.4.1.4.1.2.3 = 11, 1.0.15628.4.1.4.1.3.3 = 2, 1.0.15628.4.1.4.1.4.3 = 3, 1.0.15628.4.1.4.1.5.3 = 4, 1.0.15628.4.1.4.1.6.3 = 5, 1.0.15628.4.1.4.1.7.3 = 0c:02:14:11:11:2f, 1.0.15628.4.1.4.1.8.3 = 0c:02:14:11:11:2f, 1.0.15628.4.1.4.1.9.3 = 88, 1.0.15628.4.1.4.1.10.3 = 9]";
+      String expectedResult = "[1.0.15628.4.1.4.1.2.3 = 11, 1.0.15628.4.1.4.1.3.3 = 2, 1.0.15628.4.1.4.1.4.3 = 3, 1.0.15628.4.1.4.1.5.3 = 4, 1.0.15628.4.1.4.1.6.3 = 5, 1.0.15628.4.1.4.1.7.3 = 07:e1:0c:02:11:2f, 1.0.15628.4.1.4.1.8.3 = 07:e1:0c:02:11:2f, 1.0.15628.4.1.4.1.9.3 = 88, 1.0.15628.4.1.4.1.10.3 = 9, 1.0.15628.4.1.4.1.11.3 = 10]";
+      String expectedResult2 = "[1.0.15628.4.1.4.1.2.3 = 11, 1.0.15628.4.1.4.1.3.3 = 2, 1.0.15628.4.1.4.1.4.3 = 3, 1.0.15628.4.1.4.1.5.3 = 4, 1.0.15628.4.1.4.1.6.3 = 5, 1.0.15628.4.1.4.1.7.3 = 07:e1:0c:02:11:2f, 1.0.15628.4.1.4.1.8.3 = 07:e1:0c:02:11:2f, 1.0.15628.4.1.4.1.9.3 = 88, 1.0.15628.4.1.4.1.10.3 = 9]";
 
 
       String rsuSRMPsid = "11";

pom.xml

@@ -11,12 +11,12 @@
 
   <scm>
     <developerConnection>scm:git:https://github.com/usdot-jpo-ode/jpo-ode.git</developerConnection>
-    <tag>jpo-ode-1.0.7</tag>
+    <tag>jpo-ode-0.0.8</tag>
   </scm>
 
   <groupId>usdot.jpo.ode</groupId>
   <artifactId>jpo-ode</artifactId>
-  <version>1.0.7</version>
+  <version>0.0.8</version>
   <packaging>pom</packaging>
   <modules>
     <module>jpo-ode-common</module>

qa/test-harness/.gitignore

@@ -0,0 +1 @@
+*.log