Skip to content

[Bug] Segment name doesn't support '/' #6772

@xiangfu0

Description

@xiangfu0

Since we support segment name with start/end time in it, if the time format is in yy/mm/dd then it introduces special characters like '/' into segment name.

Segment upload path if fine, we can see the segment is loaded into the controller, but it's stored into a directory format.
E.g. segment name: covid_cases_OFFLINE_2020/10/23_2021/01/28_0 is stored into many levels of directory:

➜ find pinot-docker-demo/pinot/controller/data/Deleted_Segments/covid_cases
pinot-docker-demo/pinot/controller/data/Deleted_Segments/covid_cases
pinot-docker-demo/pinot/controller/data/Deleted_Segments/covid_cases/covid_cases_OFFLINE_2020
pinot-docker-demo/pinot/controller/data/Deleted_Segments/covid_cases/covid_cases_OFFLINE_2020/10
pinot-docker-demo/pinot/controller/data/Deleted_Segments/covid_cases/covid_cases_OFFLINE_2020/10/23_2021
pinot-docker-demo/pinot/controller/data/Deleted_Segments/covid_cases/covid_cases_OFFLINE_2020/10/23_2021/01
pinot-docker-demo/pinot/controller/data/Deleted_Segments/covid_cases/covid_cases_OFFLINE_2020/10/23_2021/01/28_0

From zk side, the segment is also stored into many levels:
image

Then this prevents segment download from pinot server(I don't even bother this part as it's already pretty wrong from controller and zk side.)

The log is:

pinot-server_1      | Got temporary error status code: 500 while downloading segment from: http://pinot-controller:9000/segments/covid_cases/covid_cases_OFFLINE_2020%2F10%2F23_2021%2F01%2F28_0 to: /tmp/data/server/data/segment/covid_cases_OFFLINE/tmp-covid_cases_OFFLINE_2020/10/23_2021/01/28_0-b65e3a35-d867-4a83-8404-707172c18e4e/covid_cases_OFFLINE_2020/10/23_2021/01/28_0.enc
pinot-server_1      | org.apache.pinot.common.exception.HttpErrorStatusException: Got error status code: 500 (Internal Server Error) with reason: "Failed to get reason" while sending request: http://pinot-controller:9000/segments/covid_cases/covid_cases_OFFLINE_2020%2F10%2F23_2021%2F01%2F28_0
pinot-server_1      | 	at org.apache.pinot.common.utils.FileUploadDownloadClient.downloadFile(FileUploadDownloadClient.java:670) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-04e2bcce5026f5911183a27f5e8fed259572dfaf]

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions