Skip to content

Commit

Permalink
'core' ORC extension (apache#7138)
Browse files Browse the repository at this point in the history
* orc extension reworked to use apache orc map-reduce lib, moved to core extensions, support for flattenSpec, tests, docs

* change binary handling to be compatible with avro and parquet, Rows.objectToStrings now converts byte[] to base64, change date handling

* better docs and tests

* fix it

* formatting

* doc fix

* fix it

* exclude redundant dependencies

* use latest orc-mapreduce, add hadoop jobProperties recommendations to docs

* doc fix

* review stuff and fix binaryAsString

* cache for root level fields

* more better
  • Loading branch information
clintropolis authored and gianm committed Apr 9, 2019
1 parent 9c5dc13 commit f4dc3e8
Show file tree
Hide file tree
Showing 33 changed files with 1,787 additions and 1,567 deletions.
3 changes: 3 additions & 0 deletions core/src/main/java/org/apache/druid/data/input/Rows.java
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,9 @@ public static List<String> objectToStrings(final Object inputValue)
} else if (inputValue instanceof List) {
// guava's toString function fails on null objects, so please do not use it
return ((List<?>) inputValue).stream().map(String::valueOf).collect(Collectors.toList());
} else if (inputValue instanceof byte[]) {
// convert byte[] to base64 encoded string
return Collections.singletonList(StringUtils.encodeBase64String((byte[]) inputValue));
} else {
return Collections.singletonList(String.valueOf(inputValue));
}
Expand Down
4 changes: 2 additions & 2 deletions distribution/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,8 @@
<argument>-c</argument>
<argument>org.apache.druid.extensions:mysql-metadata-storage</argument>
<argument>-c</argument>
<argument>org.apache.druid.extensions:druid-orc-extensions</argument>
<argument>-c</argument>
<argument>org.apache.druid.extensions:druid-parquet-extensions</argument>
<argument>-c</argument>
<argument>org.apache.druid.extensions:postgresql-metadata-storage</argument>
Expand Down Expand Up @@ -315,8 +317,6 @@
<argument>-c</argument>
<argument>org.apache.druid.extensions.contrib:druid-opentsdb-emitter</argument>
<argument>-c</argument>
<argument>org.apache.druid.extensions.contrib:druid-orc-extensions</argument>
<argument>-c</argument>
<argument>org.apache.druid.extensions.contrib:druid-rabbitmq</argument>
<argument>-c</argument>
<argument>org.apache.druid.extensions.contrib:druid-redis-cache</argument>
Expand Down
113 changes: 0 additions & 113 deletions docs/content/development/extensions-contrib/orc.md

This file was deleted.

Loading

0 comments on commit f4dc3e8

Please sign in to comment.