Skip to content

Allow adding a unique suffix to segment name during batch ingestion #8782

@KKcorps

Description

@KKcorps

Currently, if a user creates segments from the batch data that

  • contains either no time column or
  • contains same time for all the records (e.g. Current data in yyyy-MM-dd format)

the generated segments have same names. This creates issue because the segments end up replacing each other even when the push type is set to APPEND

The solution is to either add a current timestamp column in the table or provide a config in SegmentNameGenerationSpec to append currentTimestamp, uuid or something else at the end of the segment name

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions