Fix encoding #2
Merged
Fix encoding #2
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
without this the prefix in the response would not be URL encoded before: ``` <Prefix> sub-folder <>/sub2/ </Prefix> ``` after ``` <Prefix> sub-folder+%3C%3E/sub2/ </Prefix> ```
at least for the `mc` client of minio this makes things clearer
having a folder/file name in the backend that contains unusual characters e.g. `bucket2/sub-folder <>/a space $%&#.txt`
results without this change in this output when running `mc ls`
`[2023-06-16 17:19:10 +0545] 11B STANDARD /bucket2/sub-folder+%3C%3E/sub2/a+space+%24%25%26%23.txt`
also such a folder/bucket cannot be downloaded using `mc cp`
after the change the output of `mc ls` is correct and same as if it connects to AWS S3
`[2023-06-16 17:19:10 +0545] 11B STANDARD a space $%&#.txt`
and also the `mc tree -f` output looks correct
```
proxy/bucket2/
└─ sub-folder <>
├─ a space $%&#.txt
└─ sub2
└─ a space $%&#.txt
```
individual-it
force-pushed
the
fixEncoding
branch
from
a23f051
to
7338612
Compare
individual-it
added a commit
to JankariTech/rclone
that referenced
this pull request
Jun 21, 2023
using the mc client file encoding were wrong see rclone/gofakes3#2 for details
individual-it
added a commit
to JankariTech/rclone
that referenced
this pull request
Jun 22, 2023
using the mc client file encoding were wrong see rclone/gofakes3#2 for details
5 tasks
individual-it
added a commit
to JankariTech/rclone
that referenced
this pull request
Jun 23, 2023
using the mc (minio) client file encoding were wrong see rclone/gofakes3#2 for details
individual-it
added a commit
to JankariTech/rclone
that referenced
this pull request
Jun 26, 2023
using the mc (minio) client file encoding were wrong see rclone/gofakes3#2 for details
individual-it
added a commit
to JankariTech/rclone
that referenced
this pull request
Jul 4, 2023
using the mc (minio) client file encoding were wrong see rclone/gofakes3#2 for details
individual-it
added a commit
to JankariTech/rclone
that referenced
this pull request
Sep 1, 2023
using the mc (minio) client file encoding were wrong see rclone/gofakes3#2 for details
saw-jan
pushed a commit
to JankariTech/rclone
that referenced
this pull request
Sep 12, 2023
using the mc (minio) client file encoding were wrong see rclone/gofakes3#2 for details
saw-jan
pushed a commit
to JankariTech/rclone
that referenced
this pull request
Sep 21, 2023
using the mc (minio) client file encoding were wrong see rclone/gofakes3#2 for details
saw-jan
pushed a commit
to JankariTech/rclone
that referenced
this pull request
Sep 25, 2023
using the mc (minio) client file encoding were wrong see rclone/gofakes3#2 for details
saw-jan
pushed a commit
to JankariTech/rclone
that referenced
this pull request
Oct 18, 2023
using the mc (minio) client file encoding were wrong see rclone/gofakes3#2 for details
saw-jan
pushed a commit
to JankariTech/rclone
that referenced
this pull request
Oct 26, 2023
using the mc (minio) client file encoding were wrong see rclone/gofakes3#2 for details
saw-jan
pushed a commit
to JankariTech/rclone
that referenced
this pull request
Nov 2, 2023
using the mc (minio) client file encoding were wrong see rclone/gofakes3#2 for details
ncw
pushed a commit
to rclone/rclone
that referenced
this pull request
Nov 14, 2023
using the mc (minio) client file encoding were wrong see rclone/gofakes3#2 for details
ncw
pushed a commit
to rclone/rclone
that referenced
this pull request
Nov 16, 2023
using the mc (minio) client file encoding were wrong see rclone/gofakes3#2 for details
saw-jan
pushed a commit
to JankariTech/rclone
that referenced
this pull request
Jan 8, 2024
serve s3: fix file name encoding using s3 serve with mc client using the mc (minio) client file encoding were wrong see rclone/gofakes3#2 for details create fs and vfs per request use JankariTech gofakes3 add method newVfs to cache vfs with given key create remote fs on serve and vfs per request remove unnecessary return create new fs on every request test header forwarding improve variable naming test parallel requests add docker setup for S3 to WebDav proxy README for S3 to WebDav proxy Credit PointCab GmbH to finance the changes
saw-jan
pushed a commit
to JankariTech/rclone
that referenced
this pull request
Jan 8, 2024
serve s3: fix file name encoding using s3 serve with mc client using the mc (minio) client file encoding were wrong see rclone/gofakes3#2 for details create fs and vfs per request use JankariTech gofakes3 add method newVfs to cache vfs with given key create remote fs on serve and vfs per request remove unnecessary return create new fs on every request test header forwarding improve variable naming test parallel requests add docker setup for S3 to WebDav proxy README for S3 to WebDav proxy Credit PointCab GmbH to finance the changes
saw-jan
pushed a commit
to JankariTech/rclone
that referenced
this pull request
Jan 9, 2024
using the mc (minio) client file encoding were wrong see rclone/gofakes3#2 for details
saw-jan
pushed a commit
to JankariTech/rclone
that referenced
this pull request
Jan 10, 2024
using the mc (minio) client file encoding were wrong see rclone/gofakes3#2 for details
saw-jan
pushed a commit
to JankariTech/rclone
that referenced
this pull request
Jan 10, 2024
using the mc (minio) client file encoding were wrong see rclone/gofakes3#2 for details
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
running rclone to serve an S3 server with the code of rclone/rclone#7062 (original PR: rclone/rclone#6461) and Nextcloud as WebDAV backend I've noticed that having file/folder names that need encoding lead to issues.
When using the
mc lsthe files are listed in the encoded form and they cannot be copied viamc cp --recursivebecause mc will try to access the S3 keys by encoding again the names it received and so resulting in a double encoded URLThis PR also encodes the prefix & sets the encoding type in the response XML (details in the commit descriptions)