-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Open
Labels
area:code-qualityCode quality improvementsCode quality improvementsarea:metadata-tableMetadata table relatedMetadata table relatedfrom-jirapriority:mediumModerate impact; usability gapsModerate impact; usability gapstype:featureNew features and enhancementsNew features and enhancements
Milestone
Description
KeyField can never be empty for File. If so, there is really no need for falling back to sequential integer keys in the HFileDataBlock::serializeRecords() code path.
{noformat}
// Build the record key
final Field schemaKeyField = records.get(0).getSchema().getField(this.keyField);
if (schemaKeyField == null) {
// Missing key metadata field. Use an integer sequence key instead.
useIntegerKey = true;
keySize = (int) Math.ceil(Math.log(records.size())) + 1;
}
while (itr.hasNext()) {
IndexedRecord record = itr.next();
String recordKey;
if (useIntegerKey) {
recordKey = String.format("%" + keySize + "s", key++);
} else {
recordKey = record.get(schemaKeyField.pos()).toString();
}
{noformat}
JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-2954
- Type: Wish
- Fix version(s):
- 1.1.0
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
area:code-qualityCode quality improvementsCode quality improvementsarea:metadata-tableMetadata table relatedMetadata table relatedfrom-jirapriority:mediumModerate impact; usability gapsModerate impact; usability gapstype:featureNew features and enhancementsNew features and enhancements