[Issue 13217][pulsar-package-management] Fix the filesystem storage failure #13218
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
merlimat
approved these changes
Dec 9, 2021
zymap
reviewed
Dec 10, 2021
| @@ -248,7 +248,7 @@ private String metadataPath(PackageName packageName) { | |||
| } | |||
|
|
|||
| private String packagePath(PackageName packageName) { | |||
| return packageName.toRestPath(); | |||
| return packageName.toRestPath() + "/pack"; | |||
There was a problem hiding this comment.
I would suggest changing the /pack to /data
There was a problem hiding this comment.
BTW, could you please add a test to cover this case?
There was a problem hiding this comment.
Done, we should delete the packageName.toRestPath() path at the same time otherwise list-versions command will print this parent path (i.e. version path like v1.0).
zymap
requested changes
Dec 10, 2021
Sure, I'm working on this. |
|
@zymap I have addressed above comments, please help to take a review again. |
murong00
force-pushed
the
branch-13218
branch
from
3140de5
to
210d186
Compare
|
/pulsarbot run-failure-checks |
|
ping @zymap |
zymap
approved these changes
Dec 13, 2021
fxbing
pushed a commit
to fxbing/pulsar
that referenced
this pull request
Dec 19, 2021
…ailure (apache#13218) * Fix the filesystem storage failure * Add a integration test to cover this change
zymap
added a commit
to zymap/pulsar
that referenced
this pull request
Apr 28, 2022
--- Fixes apache#15362 *Motivation* The PR apache#13218 supports saving package data into filesystem. But it introduces an regression for the old versions. For example, we have a package function://public/default/package@v0.1, it will save the meta to the path function/public/default/package/v0.1/meta, and save the data to the path function/public/default/package/v0.1. By default, we are using distributed log as the package storage, and it supports saving data at a directory. But some storage like filesystem don't have the similar ability, it needs another path for saving data. This api provides the ability to support saving the data in another place. If you specify the data path as `/data`, the package will saved into function/public/default/package/v0.1/data. *Modifications* - make the data path configurable in the storage implementation.
zymap
added a commit
that referenced
this pull request
May 6, 2022
…ion (#15367) --- Fixes #15362 *Motivation* The PR #13218 supports saving package data into filesystem. But it introduces a regression for the old versions. For example, we have a package function://public/default/package@v0.1, it will save the meta to the path function/public/default/package/v0.1/meta, and save the data to the path function/public/default/package/v0.1. By default, we are using distributed log as the package storage, and it supports saving data in a directory. But some storage like filesystem doesn't have the similar ability, it needs another path for saving data. This API provides the ability to support saving the data in another place. If you specify the data path as `/data`, the package will be saved into function/public/default/package/v0.1/data. *Modifications* - make the data path configurable in the storage implementation.
codelipenghui
pushed a commit
that referenced
this pull request
May 20, 2022
…ion (#15367) --- Fixes #15362 *Motivation* The PR #13218 supports saving package data into filesystem. But it introduces a regression for the old versions. For example, we have a package function://public/default/package@v0.1, it will save the meta to the path function/public/default/package/v0.1/meta, and save the data to the path function/public/default/package/v0.1. By default, we are using distributed log as the package storage, and it supports saving data in a directory. But some storage like filesystem doesn't have the similar ability, it needs another path for saving data. This API provides the ability to support saving the data in another place. If you specify the data path as `/data`, the package will be saved into function/public/default/package/v0.1/data. *Modifications* - make the data path configurable in the storage implementation. (cherry picked from commit aa9aa18)
nicoloboschi
pushed a commit
to datastax/pulsar
that referenced
this pull request
May 23, 2022
…ion (apache#15367) --- Fixes apache#15362 *Motivation* The PR apache#13218 supports saving package data into filesystem. But it introduces a regression for the old versions. For example, we have a package function://public/default/package@v0.1, it will save the meta to the path function/public/default/package/v0.1/meta, and save the data to the path function/public/default/package/v0.1. By default, we are using distributed log as the package storage, and it supports saving data in a directory. But some storage like filesystem doesn't have the similar ability, it needs another path for saving data. This API provides the ability to support saving the data in another place. If you specify the data path as `/data`, the package will be saved into function/public/default/package/v0.1/data. *Modifications* - make the data path configurable in the storage implementation. (cherry picked from commit aa9aa18) (cherry picked from commit 46d6a7f)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
FIxes #13217.
Modifications
Make
packagePathandmetadataPathat the same level.Documentation
no-need-doc