Skip to content

NIFI-2897: Fixed SelectHiveQL for CSV output of complex types#1132

Closed
mattyb149 wants to merge 1 commit intoapache:masterfrom
mattyb149:NIFI-2897
Closed

NIFI-2897: Fixed SelectHiveQL for CSV output of complex types#1132
mattyb149 wants to merge 1 commit intoapache:masterfrom
mattyb149:NIFI-2897

Conversation

@mattyb149
Copy link
Contributor

The lack of unit tests is due to the fact that we use Derby. When we get integration tests in place using MiniHS2, we can automate this kind of testing. In the meantime, here are some helpful HiveDDL statements you can use to populate a table with the different types, along with some queries that will illustrate the problem and solution:

Create a table with a field that is an array of structs:

create table arraytest (a ARRAY<STRUCT<name:STRING, age:INT>>)

Insert two elements (this assumes you have a table named "dummy" with at least one row):

insert into arraytest select array(named_struct("name","Joe","age",42),named_struct("name","Mary","age",24)) from dummy limit 1

Select the whole array (as one field):

SELECT * FROM arraytest

Select the structs as individual columns:

SELECT a[0], a[1] FROM arraytest

Select a struct and a field from the other struct:

SELECT a[0], a[1].name FROM arraytest

The first SELECT should generate a single field/column in CSV, the other two should have two columns, all properly escaped to give valid CSV.

@olegz
Copy link
Contributor

olegz commented Oct 14, 2016

LGTM. Merging

@asfgit asfgit closed this in b52b839 Oct 14, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants