Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CARBONDATA-4093] Added logs for MV and method to verify if mv is in Sync during query #4060

Closed
wants to merge 2 commits into from
Closed

[CARBONDATA-4093] Added logs for MV and method to verify if mv is in Sync during query #4060

wants to merge 2 commits into from

Conversation

Indhumathi27
Copy link
Contributor

@Indhumathi27 Indhumathi27 commented Dec 19, 2020 •
edited

Why is this PR needed?

Added logs for MV and method to verify if mv is in Sync during query

What changes were proposed in this PR?

  1. Move MV Enable Check to beginning to avoid transform logical plan
  2. Add Logger if exception is occurred during fetching mv schema
  3. Check if MV is in Sync and allow Query rewrite
  4. Reuse reading LoadMetadetails to get mergedLoadMapping
  5. Set NO-Dict Schema types for insert-partition flow - missed from [CARBONDATA-4077]

Does this PR introduce any user interface change?

  • No

Is any new testcase added?

  • Yes

@CarbonDataQA2
Copy link

Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3453/

@CarbonDataQA2
Copy link

Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5213/

@CarbonDataQA2
Copy link

Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3455/

@CarbonDataQA2
Copy link

Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5215/

akashrn5
akashrn5 reviewed Dec 21, 2020
View reviewed changes
core/src/main/java/org/apache/carbondata/core/view/MVManager.java
@@ -234,7 +247,16 @@ public static String getUpdatedSegmentMap(String mergedLoadName,
public List<MVStatusDetail> getEnabledStatusDetails() throws IOException {
List<MVStatusDetail> statusDetails = new ArrayList<>();
for (String database : this.getDatabases()) {
statusDetails.addAll(this.getEnabledStatusDetails(database));
try {
statusDetails.addAll(this.getEnabledStatusDetails(database));
Copy link
Contributor

@akashrn5 akashrn5 Dec 21, 2020 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since already it failed to access in getSchemas(database), please see if this is required or not, if not please revert

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is needed. as it try to get MV_STATUS file for all databases.

@CarbonDataQA2
Copy link

Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5218/

@CarbonDataQA2
Copy link

Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3457/

@CarbonDataQA2
Copy link

Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5223/

@CarbonDataQA2
Copy link

Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3462/

@akashrn5
Copy link
Contributor

LGTM

@asfgit asfgit closed this in aae93c1 Dec 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@akashrn5 akashrn5 akashrn5 left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@Indhumathi27 @CarbonDataQA2 @akashrn5