Skip to content

Commit

Permalink
[CARBONDATA-1719][Pre-Aggregate][Bug] Fixed bug to handle data incons…
Browse files Browse the repository at this point in the history
…istency on concurrent data load and pre-aggregate table creation

Problem: On concurrent data load and pre-aggregate table creation, datamap was not getting populated with the load data even after data load completes.

This closes #1697
  • Loading branch information
SangeetaGulia authored and kumarvishal09 committed Jan 4, 2018
1 parent 0c65aba commit 69de387
Showing 1 changed file with 4 additions and 2 deletions.
Expand Up @@ -20,9 +20,10 @@ package org.apache.spark.sql.execution.command.preaaggregate
import scala.collection.JavaConverters._
import scala.collection.mutable

import org.apache.spark.sql.CarbonEnv
import org.apache.spark.sql.catalyst.TableIdentifier
import org.apache.spark.sql.execution.command.management.CarbonAlterTableCompactionCommand
import org.apache.spark.sql.execution.command.AlterTableModel
import org.apache.spark.sql.execution.command.management.CarbonAlterTableCompactionCommand

import org.apache.carbondata.core.metadata.schema.table.AggregationDataMapSchema
import org.apache.carbondata.core.util.CarbonUtil
Expand All @@ -38,7 +39,8 @@ object LoadPostAggregateListener extends OperationEventListener {
val loadEvent = event.asInstanceOf[LoadTablePreStatusUpdateEvent]
val sparkSession = loadEvent.sparkSession
val carbonLoadModel = loadEvent.carbonLoadModel
val table = carbonLoadModel.getCarbonDataLoadSchema.getCarbonTable
val table = CarbonEnv.getCarbonTable(Option(carbonLoadModel.getDatabaseName),
carbonLoadModel.getTableName)(sparkSession)
if (CarbonUtil.hasAggregationDataMap(table)) {
// getting all the aggergate datamap schema
val aggregationDataMapList = table.getTableInfo.getDataMapSchemaList.asScala
Expand Down

0 comments on commit 69de387

Please sign in to comment.