-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Issue 13217][pulsar-package-management] Fix the filesystem storage failure #13218
Conversation
@@ -248,7 +248,7 @@ private String metadataPath(PackageName packageName) { | |||
} | |||
|
|||
private String packagePath(PackageName packageName) { | |||
return packageName.toRestPath(); | |||
return packageName.toRestPath() + "/pack"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would suggest changing the /pack
to /data
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
BTW, could you please add a test to cover this case?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done, we should delete the packageName.toRestPath()
path at the same time otherwise list-versions
command will print this parent path (i.e. version path like v1.0
).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we need to add a test to cover this case
Sure, I'm working on this. |
@zymap I have addressed above comments, please help to take a review again. |
3140de5
to
210d186
Compare
/pulsarbot run-failure-checks |
ping @zymap |
…ailure (apache#13218) * Fix the filesystem storage failure * Add a integration test to cover this change
--- Fixes apache#15362 *Motivation* The PR apache#13218 supports saving package data into filesystem. But it introduces an regression for the old versions. For example, we have a package function://public/default/package@v0.1, it will save the meta to the path function/public/default/package/v0.1/meta, and save the data to the path function/public/default/package/v0.1. By default, we are using distributed log as the package storage, and it supports saving data at a directory. But some storage like filesystem don't have the similar ability, it needs another path for saving data. This api provides the ability to support saving the data in another place. If you specify the data path as `/data`, the package will saved into function/public/default/package/v0.1/data. *Modifications* - make the data path configurable in the storage implementation.
…ion (#15367) --- Fixes #15362 *Motivation* The PR #13218 supports saving package data into filesystem. But it introduces a regression for the old versions. For example, we have a package function://public/default/package@v0.1, it will save the meta to the path function/public/default/package/v0.1/meta, and save the data to the path function/public/default/package/v0.1. By default, we are using distributed log as the package storage, and it supports saving data in a directory. But some storage like filesystem doesn't have the similar ability, it needs another path for saving data. This API provides the ability to support saving the data in another place. If you specify the data path as `/data`, the package will be saved into function/public/default/package/v0.1/data. *Modifications* - make the data path configurable in the storage implementation.
…ion (#15367) --- Fixes #15362 *Motivation* The PR #13218 supports saving package data into filesystem. But it introduces a regression for the old versions. For example, we have a package function://public/default/package@v0.1, it will save the meta to the path function/public/default/package/v0.1/meta, and save the data to the path function/public/default/package/v0.1. By default, we are using distributed log as the package storage, and it supports saving data in a directory. But some storage like filesystem doesn't have the similar ability, it needs another path for saving data. This API provides the ability to support saving the data in another place. If you specify the data path as `/data`, the package will be saved into function/public/default/package/v0.1/data. *Modifications* - make the data path configurable in the storage implementation. (cherry picked from commit aa9aa18)
…ion (apache#15367) --- Fixes apache#15362 *Motivation* The PR apache#13218 supports saving package data into filesystem. But it introduces a regression for the old versions. For example, we have a package function://public/default/package@v0.1, it will save the meta to the path function/public/default/package/v0.1/meta, and save the data to the path function/public/default/package/v0.1. By default, we are using distributed log as the package storage, and it supports saving data in a directory. But some storage like filesystem doesn't have the similar ability, it needs another path for saving data. This API provides the ability to support saving the data in another place. If you specify the data path as `/data`, the package will be saved into function/public/default/package/v0.1/data. *Modifications* - make the data path configurable in the storage implementation. (cherry picked from commit aa9aa18) (cherry picked from commit 46d6a7f)
Motivation
FIxes #13217.
Modifications
Make
packagePath
andmetadataPath
at the same level.Documentation
no-need-doc