Address security vulnerabilities CVSS >= 7 #8980

ccaominh · 2019-12-03T01:40:24Z

Description

Update dependencies to address security vulnerabilities with CVSS scores of 7 or higher. A new Travis CI job is added to prevent new high/critical security vulnerabilities from being added.

Updated dependencies:

api-util 1.0.0 -> 1.0.3
jackson 2.9.10 -> 2.10.1
kafka 2.1.0 -> 2.1.1
libthrift 0.10.0 -> 0.13.0
protobuf 3.2.0 -> 3.11.0

The following high/critical security vulnerabilities are currently suppressed (so that the new Travis CI job can be added now) and are left as future work:

hibernate-validator:5.2.5
jackson-mapper-asl:1.9.13
libthrift:0.6.1
netty:3.10.6
nimbus-jose-jwt:4.41.1

This PR has:

been self-reviewed.
added or updated version, license, or notice information in licenses.yaml
added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.

Update dependencies to address security vulnerabilities with CVSS scores of 7 or higher. A new Travis CI job is added to prevent new high/critical security vulnerabilities from being added. Updated dependencies: - api-util 1.0.0 -> 1.0.3 - jackson 2.9.10 -> 2.10.1 - kafka 2.1.0 -> 2.1.1 - libthrift 0.10.0 -> 0.13.0 - protobuf 3.2.0 -> 3.11.0 The following high/critical security vulnerabilities are currently suppressed (so that the new Travis CI job can be added now) and are left as future work to fix: - hibernate-validator:5.2.5 - jackson-mapper-asl:1.9.13 - libthrift:0.6.1 - netty:3.10.6 - nimbus-jose-jwt:4.41.1

ccaominh · 2019-12-04T03:45:18Z

Manually tested with https://druid.apache.org/docs/latest/tutorials/tutorial-batch-hadoop.html

clintropolis · 2019-12-04T07:19:55Z

licenses.yaml

@@ -233,6 +233,30 @@ notice: |

 ---

+name: JavaBeans Activation Framework API JAR


Are these only run by CI? Unless these jars end up in the source binary packaging when building the distributions I don't believe they need to be included here.

They're part of the distribution now. Here's a diff of the distribution:

Only in before/extensions/druid-hdfs-storage: api-util-1.0.0-M20.jar Only in after/extensions/druid-hdfs-storage: api-util-1.0.3.jar Only in before/extensions/druid-kafka-extraction-namespace: kafka-clients-2.1.0.jar Only in after/extensions/druid-kafka-extraction-namespace: kafka-clients-2.1.1.jar Only in before/extensions/druid-kerberos: api-util-1.0.0-M20.jar Only in after/extensions/druid-kerberos: api-util-1.0.3.jar Only in after/extensions/druid-kinesis-indexing-service: jackson-annotations-2.10.1.jar Only in before/extensions/druid-kinesis-indexing-service: jackson-annotations-2.9.10.jar Only in after/extensions/druid-kinesis-indexing-service: jackson-core-2.10.1.jar Only in before/extensions/druid-kinesis-indexing-service: jackson-core-2.9.10.jar Only in after/extensions/druid-kinesis-indexing-service: jackson-databind-2.10.1.jar Only in before/extensions/druid-kinesis-indexing-service: jackson-databind-2.9.10.jar Only in before/extensions/druid-orc-extensions: protobuf-java-3.1.0.jar Only in after/extensions/druid-orc-extensions: protobuf-java-3.11.0.jar Only in after/extensions/druid-protobuf-extensions: error_prone_annotations-2.3.2.jar Only in before/extensions/druid-protobuf-extensions: gson-2.7.jar Only in after/extensions/druid-protobuf-extensions: gson-2.8.6.jar Only in after/extensions/druid-protobuf-extensions: protobuf-java-3.11.0.jar Only in before/extensions/druid-protobuf-extensions: protobuf-java-3.2.0.jar Only in after/extensions/druid-protobuf-extensions: protobuf-java-util-3.11.0.jar Only in before/extensions/druid-protobuf-extensions: protobuf-java-util-3.2.0.jar Only in after/lib: jackson-annotations-2.10.1.jar Only in before/lib: jackson-annotations-2.9.10.jar Only in after/lib: jackson-core-2.10.1.jar Only in before/lib: jackson-core-2.9.10.jar Only in after/lib: jackson-databind-2.10.1.jar Only in before/lib: jackson-databind-2.9.10.jar Only in after/lib: jackson-dataformat-cbor-2.10.1.jar Only in before/lib: jackson-dataformat-cbor-2.9.10.jar Only in after/lib: jackson-dataformat-smile-2.10.1.jar Only in before/lib: jackson-dataformat-smile-2.9.10.jar Only in after/lib: jackson-datatype-guava-2.10.1.jar Only in before/lib: jackson-datatype-guava-2.9.10.jar Only in after/lib: jackson-datatype-joda-2.10.1.jar Only in before/lib: jackson-datatype-joda-2.9.10.jar Only in after/lib: jackson-jaxrs-base-2.10.1.jar Only in before/lib: jackson-jaxrs-base-2.9.10.jar Only in after/lib: jackson-jaxrs-json-provider-2.10.1.jar Only in before/lib: jackson-jaxrs-json-provider-2.9.10.jar Only in after/lib: jackson-jaxrs-smile-provider-2.10.1.jar Only in before/lib: jackson-jaxrs-smile-provider-2.9.10.jar Only in after/lib: jackson-module-guice-2.10.1.jar Only in before/lib: jackson-module-guice-2.9.10.jar Only in after/lib: jackson-module-jaxb-annotations-2.10.1.jar Only in before/lib: jackson-module-jaxb-annotations-2.9.10.jar Only in after/lib: jakarta.activation-api-1.2.1.jar Only in after/lib: jakarta.xml.bind-api-2.3.2.jar Only in before/lib: protobuf-java-3.1.0.jar Only in after/lib: protobuf-java-3.11.0.jar Only in after/licenses/bin: jakarta.activation-api.EDL1

clintropolis · 2019-12-04T07:20:22Z

licenses/bin/jakarta.activation-api.EDL1

@@ -0,0 +1,13 @@
+Eclipse Distribution License - v 1.0


Likewise these might not be needed depending on the answer to whether or not this stuff is only used by CI

clintropolis · 2019-12-04T07:22:40Z

core/src/main/java/org/apache/druid/data/input/impl/prefetch/JsonIterator.java

@@ -43,7 +43,7 @@
 {
  private JsonParser jp;
  private ObjectCodec objectCodec;
-  private final TypeReference typeRef;
+  private final TypeReference<T> typeRef;


Are these changes required for something or just opportunistic?

They are required after upgrading Jackson to 2.10

ccaominh · 2019-12-04T18:22:14Z

core/src/main/java/org/apache/druid/data/input/impl/prefetch/JsonIterator.java

@@ -43,7 +43,7 @@
 {
  private JsonParser jp;
  private ObjectCodec objectCodec;
-  private final TypeReference typeRef;
+  private final TypeReference<T> typeRef;


They are required after upgrading Jackson to 2.10

ccaominh · 2019-12-04T18:23:25Z

...tion-tests/src/main/java/org/apache/druid/testing/clients/CoordinatorResourceTestClient.java

@@ -96,7 +95,7 @@ private String getLoadStatusURL()
  // return a list of the segment dates for the specified datasource
  public List<String> getMetadataSegments(final String dataSource)
  {
-    ArrayList<String> segments;
+    List<String> segments;


This was required after upgrading Jackson to 2.10 (since TypeReference<List<String>> is used below)

ccaominh · 2019-12-04T18:24:39Z

licenses.yaml

@@ -233,6 +233,30 @@ notice: |

 ---

+name: JavaBeans Activation Framework API JAR


They're part of the distribution now. Here's a diff of the distribution:

Only in before/extensions/druid-hdfs-storage: api-util-1.0.0-M20.jar Only in after/extensions/druid-hdfs-storage: api-util-1.0.3.jar Only in before/extensions/druid-kafka-extraction-namespace: kafka-clients-2.1.0.jar Only in after/extensions/druid-kafka-extraction-namespace: kafka-clients-2.1.1.jar Only in before/extensions/druid-kerberos: api-util-1.0.0-M20.jar Only in after/extensions/druid-kerberos: api-util-1.0.3.jar Only in after/extensions/druid-kinesis-indexing-service: jackson-annotations-2.10.1.jar Only in before/extensions/druid-kinesis-indexing-service: jackson-annotations-2.9.10.jar Only in after/extensions/druid-kinesis-indexing-service: jackson-core-2.10.1.jar Only in before/extensions/druid-kinesis-indexing-service: jackson-core-2.9.10.jar Only in after/extensions/druid-kinesis-indexing-service: jackson-databind-2.10.1.jar Only in before/extensions/druid-kinesis-indexing-service: jackson-databind-2.9.10.jar Only in before/extensions/druid-orc-extensions: protobuf-java-3.1.0.jar Only in after/extensions/druid-orc-extensions: protobuf-java-3.11.0.jar Only in after/extensions/druid-protobuf-extensions: error_prone_annotations-2.3.2.jar Only in before/extensions/druid-protobuf-extensions: gson-2.7.jar Only in after/extensions/druid-protobuf-extensions: gson-2.8.6.jar Only in after/extensions/druid-protobuf-extensions: protobuf-java-3.11.0.jar Only in before/extensions/druid-protobuf-extensions: protobuf-java-3.2.0.jar Only in after/extensions/druid-protobuf-extensions: protobuf-java-util-3.11.0.jar Only in before/extensions/druid-protobuf-extensions: protobuf-java-util-3.2.0.jar Only in after/lib: jackson-annotations-2.10.1.jar Only in before/lib: jackson-annotations-2.9.10.jar Only in after/lib: jackson-core-2.10.1.jar Only in before/lib: jackson-core-2.9.10.jar Only in after/lib: jackson-databind-2.10.1.jar Only in before/lib: jackson-databind-2.9.10.jar Only in after/lib: jackson-dataformat-cbor-2.10.1.jar Only in before/lib: jackson-dataformat-cbor-2.9.10.jar Only in after/lib: jackson-dataformat-smile-2.10.1.jar Only in before/lib: jackson-dataformat-smile-2.9.10.jar Only in after/lib: jackson-datatype-guava-2.10.1.jar Only in before/lib: jackson-datatype-guava-2.9.10.jar Only in after/lib: jackson-datatype-joda-2.10.1.jar Only in before/lib: jackson-datatype-joda-2.9.10.jar Only in after/lib: jackson-jaxrs-base-2.10.1.jar Only in before/lib: jackson-jaxrs-base-2.9.10.jar Only in after/lib: jackson-jaxrs-json-provider-2.10.1.jar Only in before/lib: jackson-jaxrs-json-provider-2.9.10.jar Only in after/lib: jackson-jaxrs-smile-provider-2.10.1.jar Only in before/lib: jackson-jaxrs-smile-provider-2.9.10.jar Only in after/lib: jackson-module-guice-2.10.1.jar Only in before/lib: jackson-module-guice-2.9.10.jar Only in after/lib: jackson-module-jaxb-annotations-2.10.1.jar Only in before/lib: jackson-module-jaxb-annotations-2.9.10.jar Only in after/lib: jakarta.activation-api-1.2.1.jar Only in after/lib: jakarta.xml.bind-api-2.3.2.jar Only in before/lib: protobuf-java-3.1.0.jar Only in after/lib: protobuf-java-3.11.0.jar Only in after/licenses/bin: jakarta.activation-api.EDL1

ccaominh · 2019-12-04T18:25:43Z

licenses.yaml

+license_name: Eclipse Distribution License 1.0
+version: 2.3.2
+copyright: Oracle and/or its affiliates.
+license_file_path: licenses/bin/jakarta.activation-api.EDL1


Since the EDL1 license is used for both jakarta.activation-api and jakarta.xml.bind-api, I'll rename the license file to "jakarta.EDL1"

ccaominh force-pushed the cve-check branch 8 times, most recently from 2191a73 to f2ac3a5 Compare December 4, 2019 00:08

ccaominh force-pushed the cve-check branch from f2ac3a5 to 2e129d7 Compare December 4, 2019 01:55

ccaominh marked this pull request as ready for review December 4, 2019 03:45

clintropolis reviewed Dec 4, 2019

View reviewed changes

Rename EDL1 license file

0c3809e

ccaominh commented Dec 4, 2019

View reviewed changes

Fix inspection errors

de9a4cc

clintropolis added Area - Dependencies Security labels Dec 4, 2019

clintropolis approved these changes Dec 5, 2019

View reviewed changes

clintropolis merged commit af74aca into apache:master Dec 5, 2019

ccaominh deleted the cve-check branch December 6, 2019 02:09

ccaominh mentioned this pull request Dec 13, 2019

[WIP] Address security vulnerabilities #8664

Closed

jon-wei added this to the 0.17.0 milestone Dec 17, 2019

jon-wei added the Release Notes label Dec 19, 2019

jon-wei mentioned this pull request Dec 28, 2019

0.17.0 release notes #9066

Closed

ccaominh mentioned this pull request Mar 10, 2020

Druid Vulnerability Analysis #8432

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Address security vulnerabilities CVSS >= 7 #8980

Address security vulnerabilities CVSS >= 7 #8980

ccaominh commented Dec 3, 2019 •

edited

Loading

ccaominh commented Dec 4, 2019

clintropolis Dec 4, 2019

ccaominh Dec 4, 2019

clintropolis Dec 4, 2019

clintropolis Dec 4, 2019

ccaominh Dec 4, 2019

ccaominh Dec 4, 2019

ccaominh Dec 4, 2019

ccaominh Dec 4, 2019

ccaominh Dec 4, 2019

		@@ -233,6 +233,30 @@ notice: \|

		---

		name: JavaBeans Activation Framework API JAR

Address security vulnerabilities CVSS >= 7 #8980

Address security vulnerabilities CVSS >= 7 #8980

Conversation

ccaominh commented Dec 3, 2019 • edited Loading

Description

ccaominh commented Dec 4, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ccaominh commented Dec 3, 2019 •

edited

Loading