Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Min Max DataMap Example #1329

Closed
wants to merge 1 commit into from
Closed

Conversation

sounakr
Copy link
Contributor

@sounakr sounakr commented Sep 6, 2017

Min Max DataMap Example

@jackylk
Copy link
Contributor

jackylk commented Sep 6, 2017

Great, thanks for working on this!
My first comment is that can you move this example to example module, since "letting developer extend datamap in their own library" is one of the goal of datamap framework. We should verify the datamap framework toward this goal.

@@ -61,6 +61,9 @@
public static final CacheType<TableSegmentUniqueIdentifier, SegmentTaskIndexWrapper>
DRIVER_BLOCKLET_DATAMAP = new CacheType("driver_blocklet_datamap");

public static final CacheType<TableSegmentUniqueIdentifier, SegmentTaskIndexWrapper>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't add cache type, no need to handle cache in example

/**
* Datamap implementation for blocklet.
*/
public class DataMapMinMax implements DataMap, Cacheable {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to use Unsafe in example, don't confuse users with unsafe in example. just make it as simple as possibe

import org.apache.carbondata.core.util.CarbonProperties


class DataMapMinMaxWriter extends QueryTest with BeforeAndAfterAll {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Append testcase to the file name


}

public void updateMinMaxIndex(String filePath) throws IOException, MemoryException {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it shouldn't be read from index file. Use the Datamap writer to write example index file and read here

@CarbonDataQA
Copy link

Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/9/

@sounakr sounakr closed this Sep 14, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants