Skip to content

Commit

Permalink
Upgrade Avro to 1.9.1 (#5938)
Browse files Browse the repository at this point in the history
### Motivation

Currently, Pulsar uses Avro 1.8.2, a version released two years ago. The latest version of Avro is 1.9.1, which uses FasterXML's Jackson 2.x instead of Codehaus's Jackson 1.x. Jackson is prone to security issues, so we should not keep using older versions.
https://blog.godatadriven.com/apache-avro-1-9-release

### Modifications

Avro 1.9 has some major changes:

- The library used to handle logical datetime values has changed from Joda-Time to JSR-310 (apache/avro#631)
- Namespaces no longer include "$" when generating schemas containing inner classes using ReflectData (apache/avro#283)
- Validation of default values has been enabled (apache/avro#288). This results in a validation error when parsing the following schema:
```json
{
  "name": "fieldName",
  "type": [
    "null",
    "string"
  ],
  "default": "defaultValue"
}
```
The default value of a nullable field must be null (cf. https://issues.apache.org/jira/browse/AVRO-1803), and the default value of the field as above is actually null. However, this PR disables the validation in order to maintain the traditional behavior.
  • Loading branch information
Masahiro Sakamoto authored and sijie committed Jan 6, 2020
1 parent d90d524 commit f81604d
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 9 deletions.
12 changes: 3 additions & 9 deletions pulsar-sql/presto-distribution/LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,7 @@ The Apache Software License, Version 2.0

* Jackson
- jackson-annotations-2.8.11.jar
- jackson-core-2.8.11.jar
- jackson-databind-2.8.11.4.jar
- jackson-dataformat-smile-2.8.11.jar
- jackson-datatype-guava-2.8.11.jar
Expand All @@ -217,9 +218,6 @@ The Apache Software License, Version 2.0
- jackson-datatype-jdk8-2.8.11.jar
- jackson-datatype-joda-2.8.11.jar
- jackson-datatype-jsr310-2.8.11.jar
- jackson-core-2.8.11.jar
- jackson-core-asl-1.9.13.jar
- jackson-mapper-asl-1.9.13.jar
- jackson-dataformat-yaml-2.8.11.jar
* Guava
- guava-25.1-jre.jar
Expand Down Expand Up @@ -355,8 +353,8 @@ The Apache Software License, Version 2.0
* Apache XBean :: Reflect
- xbean-reflect-3.4.jar
* Avro
- avro-1.8.2.jar
- avro-protobuf-1.8.2.jar
- avro-1.9.1.jar
- avro-protobuf-1.9.1.jar
* Javax
- javax.inject-1.jar
- javax.inject-1.jar
Expand Down Expand Up @@ -475,8 +473,6 @@ BSD License
- asm-util-6.0.jar
* JLine
- jline-2.14.6.jar
* ParaNamer Core
- paranamer-2.7.jar
* Zstd-jni
- zstd-jni-1.3.7-3.jar

Expand Down Expand Up @@ -545,8 +541,6 @@ Public Domain (CC0) -- licenses/LICENSE-CC0.txt
- HdrHistogram-2.1.9.jar
* AOP Alliance
- aopalliance-1.0.jar
* XZ For Java
- xz-1.5.jar
* Reactive Streams
- reactive-streams-1.0.2.jar

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ public class PulsarConnectorUtils {

public static Schema parseSchema(String schemaJson) {
Schema.Parser parser = new Schema.Parser();
parser.setValidateDefaults(false);
return parser.parse(schemaJson);
}

Expand Down

0 comments on commit f81604d

Please sign in to comment.