Skip to content
This repository has been archived by the owner on Jun 14, 2024. It is now read-only.

Add lineage to covering index records #104

Closed
pirz opened this issue Aug 4, 2020 · 0 comments · Fixed by #121
Closed

Add lineage to covering index records #104

pirz opened this issue Aug 4, 2020 · 0 comments · Fixed by #121
Assignees
Labels
enhancement New feature or request

Comments

@pirz
Copy link
Contributor

pirz commented Aug 4, 2020

Describe the issue

For each index record, add lineage to capture the data record it is coming from. Assume data records are organized in Hive partitioned manner, where all the records which have the same value for a given partitioning key are stored in files under the same directory whose name has PartitionKey=VALUE. By adding lineage to index records, we want to add extra columns to each record and store the value of partition key(s) and file name according to the data file a record is coming from.

To Reproduce

N/A

Expected behavior

Once an index is created successfully, each index record should have columns to capture its source data record's partition key(s"' values and data file name.

Environment

All environments.

@pirz pirz added untriaged This is the default tag for a newly created issue enhancement New feature or request and removed untriaged This is the default tag for a newly created issue labels Aug 4, 2020
@pirz pirz self-assigned this Aug 4, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant