Skip to content
Permalink
Browse files
[CARBONDATA-4284] Load/insert after alter add column on partition tab…
…le with complex column fails

Why is this PR needed?
Insert after alter add column on partition table with complex column fails with bufferUnderFlowException
List of columns order in TableSchema is different after alter add column.
Ex: If partition is of dimension type, when table is created the schema columns order is as
dimension columns(partition column also) + complex column
After alter add, we are changing the order of columns in schema by moving the partition column to last.
complex column + partition column
Due to this change in order, while fillDimensionAndMeasureDetails, the indexing is wrong as it
expects complex column to be last always which causes bufferUnderFlowException while flattening complex row.

What changes were proposed in this PR?
After alter add, removed changes to add partition column at last.

This closes #4215
  • Loading branch information
ShreelekhyaG authored and kunal642 committed Sep 16, 2021
1 parent 7199357 commit 3b29bcb1057166c85e05b67a061ec616e809831d
Showing 2 changed files with 11 additions and 5 deletions.
@@ -387,11 +387,6 @@ class AlterTableColumnSchemaGenerator(

allColumns = CarbonScalaUtil.reArrangeColumnSchema(allColumns)

if (tableInfo.getFactTable.getPartitionInfo != null) {
val par = tableInfo.getFactTable.getPartitionInfo.getColumnSchemaList
allColumns = allColumns.filterNot(b => par.contains(b)) ++= par.asScala
}

def getLocalDictColumnList(tableProperties: mutable.Map[String, String],
columns: mutable.ListBuffer[ColumnSchema]): (mutable.ListBuffer[ColumnSchema],
mutable.ListBuffer[ColumnSchema]) = {
@@ -481,6 +481,17 @@ class TestAlterTableAddColumns extends QueryTest with BeforeAndAfterAll {
sql("DROP TABLE IF EXISTS alter_com")
}

test("test add column to partition table with complex column") {
sql("drop table if exists alter_com")
sql("create table alter_com(id int, map1 map<int,int>) " +
"partitioned by(name string) stored as carbondata")
sql("insert into alter_com values( 1,map(1,2),'sh')")
sql("ALTER TABLE alter_com ADD COLUMNS(intF int)")
sql("insert into alter_com values(1,map(1,2),1,'df')")
checkAnswer(sql("select * from alter_com"),
Seq(Row(1, Map(1 -> 2), null, "sh"), Row(1, Map(1 -> 2), 1, "df")))
}

test("Validate default values of complex columns added by alter command") {
sql("DROP TABLE IF EXISTS alter_com")
sql("CREATE TABLE alter_com(doubleField double, arr1 array<long> ) STORED AS carbondata")

0 comments on commit 3b29bcb

Please sign in to comment.