Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

serve: add s3 server #6461

Closed
wants to merge 14 commits into from
Closed

serve: add s3 server #6461

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
50285a1
feat: fix encode error; impl encode func
Mikubill Sep 21, 2022
3e840c3
Merge branch 'master' of https://github.com/rclone/rclone
Mikubill Sep 21, 2022
d4b9546
fix: error handling for getObjectsList
Mikubill Sep 21, 2022
fcf8634
fix: incorrect path and metadata handler
Mikubill Sep 22, 2022
b748639
feat: add auth support; fix go1.17 compile error
Mikubill Oct 3, 2022
e4629de
fix: windows path issue, go1.17 compatibility
Mikubill Oct 4, 2022
635617a
docs: add docs and flag
Mikubill Oct 4, 2022
8e455c1
fix: move sign pkg to gofakes3; use new lib/http
Mikubill Oct 27, 2022
4041b31
fix: auth by httplib, authKey format, maybe race
Mikubill Oct 27, 2022
72eccbf
docs: update docs for changed args
Mikubill Oct 27, 2022
3b16d3d
fix: encodings, remove charEncoder
Mikubill Oct 27, 2022
4f44bc7
fix: invalid arg desc
Mikubill Oct 29, 2022
cdf67e8
fix: auth error, update gofakes3
Mikubill Oct 29, 2022
5f07e0d
fix: auth handler
Mikubill Oct 30, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,5 @@ fuzz-build.zip
*.rej
Thumbs.db
__pycache__
.devcontainer
.DS_Store
8 changes: 8 additions & 0 deletions backend/s3/s3.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,9 @@ func init() {
}, {
Value: "RackCorp",
Help: "RackCorp Object Storage",
}, {
Value: "Rclone",
Help: "Rclone S3 Server",
}, {
Value: "Scaleway",
Help: "Scaleway Object Storage",
Expand Down Expand Up @@ -2612,6 +2615,11 @@ func setQuirks(opt *Options) {
listObjectsV2 = false // untested
virtualHostStyle = false
urlEncodeListings = false
case "Rclone":
listObjectsV2 = true
urlEncodeListings = true
virtualHostStyle = false
useMultipartEtag = false
case "Storj":
// Force chunk size to >= 64 MiB
if opt.ChunkSize < 64*fs.Mebi {
Expand Down
25 changes: 25 additions & 0 deletions cmd/serve/s3/auth.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
package s3

import (
"net/http"

"github.com/rclone/rclone/cmd/serve/s3/signature"
)

func (p *Server) authMiddleware(handler http.Handler) http.Handler {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to make the auth proxy work with this server?

https://github.com/rclone/rclone/blob/master/cmd/serve/proxy/proxy.go

return http.HandlerFunc(func(w http.ResponseWriter, rq *http.Request) {

// first pass to auth handler
if Opt.authPair != "" {
if result := signature.Verify(rq); result != signature.ErrNone {
resp := signature.GetAPIError(result)
w.WriteHeader(resp.HTTPStatusCode)
w.Header().Add("content-type", "application/xml")
_, _ = w.Write(signature.EncodeAPIErrorToResponse(resp))
return
}
}

handler.ServeHTTP(w, rq)
})
}