Skip to content

Home

Jump to bottom Edit New page
Robin H. Johnson edited this page Jun 23, 2015 · 34 revisions

This wiki is temporarily to hold notes on development of the Ceph RadosGW S3 static website code.

TODO

  • implement errorpage
    • TEST: redirections: HttpRedirectCode, HttpErrorCodeReturnedEquals
    • ErrorDocument:
      • Needs connection to RADOS to fetch page
      • Needs to disable the existing error formatter stuff to send this content
  • Redirect, part 2:
    • per-object redirect from x-amz-website-redirect-location header
    • Per AmazonS3, RoutingRules take precedence x-amz-website-redirect-location
  • Redirect, part N:
    • HttpRedirectCode in RoutingRules gives a self-contradictory error of The provided HTTP redirect code (314) is not valid. Valid codes are 3XX except 300.
    • The ONLY codes actually accepted are: 301 302 303 304 305 307 308
    • We should validate that input to match Amazon
    • Ditto Protocol is only http or https
  • Tests
    • DONE: Bucket tests with base IndexDoc + ErrorDoc
    • DONE: Redirect_all tests
    • DONE: Redirect rule tests
    • refactoring tests to remove more duplication in setup/teardown, maybe Unittest.Testsuite?
    • Testing against AmazonS3 shows that changing WebsiteConfiguration can take 10+ seconds to propogate
  • Docs
    • Admin
    • User (maybe just say use boto? need to give more hints I think)

FUTURE

  • Update S3 specs to describe the error conditions
    • Having ErrorDoc configured, but marked private can give MULTIPLE errors in the HTML output; Eg 404 NoSuchKey, AccessDenied, with an extra message An Error Occurred While Attempting to Retrieve a Custom Error Document
    • RoutingRule.Protocol must be http or https
    • RoutingRule.HttpRedirectCode must be one of: 301, 302, 303, 304, 305, 307, 308
  • redirect computation questions:
    • Q: How should it be handled when multiple redirects get applied in succession?
      • Need to compare with S3
    • Known bugs
  • Amazon's S3website endpoint exposes the existence of buckets
    • Requests for /
      • returns 404 NoSuchBucket for non-authenticated requests to buckets that do not exist;
      • returns 404 NoSuchWebsiteConfiguration for authenticated requests to buckets that do exist, but don't have websiteconf
      • returns 403 AccessDenied for buckets that exist, have websiteconf, but IndexDoc is missing or private
      • This means that you can iterate to discover existing buckets very easily.
      • For the moment, we rather return 403 AccessDenied for ALL of the above cases.
      • Fixing this differently requires a user that can probe ANY buckets for existence, not just their own one.
    • Requests for non-/
      • Need to verify how redirects are handled in this case
  • Finalize RGWRegion design changes for handling different hostnames/endpoints per API
    • Discussed with Yehuda already, need to finalize and formally propose on list
  • CivetWeb only binds to IPv4, docs have an example of port=[::]:80 that doesn't work
  • Implement AWS-signature-V4 http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-auth-using-authorization-header.html

DONE

  • implement errorpage
    • DONE, but untested: redirections: HttpRedirectCode, HttpErrorCodeReturnedEquals
    • DONE framework
  • implement redirect computation
    • DONE pending questions.
    • DONE: Base string matching
    • DONE: Error code matching
  • HTML error pages
    • Only when errorpage is not set, instead of XML errors
  • Wrote Fuzzer testcases for website code
  • Q: Boto seems to fail at handling of complex RoutingRules
  • Q: Should we enforce the same website endpoints?
    • Q: How does real S3 behave for them when website mode is off?
    • Q: what is the actual point of them in real S3
    • Two possible modes:
      • Base decision on (bucket in website mode) && (DNS endpoint used): Don't support ANY authenticated actions or operations OTHER than GET+HEAD on objects (no esp bucket actions) [this is what Amazon does]
      • Base decision on (bucket in website mode) && (no auth headers)
    • A: both implemented now, with selection between them.
  • implement base redirect
  • Existing documents explicitly requested should work
    • Done for public
  • ListBucket needs to be overridden to return the suffix document
  • trace op_get
  • Q: Does real S3 enforce objects must-be-public?

Codebases

References

Documentation notes

Add a custom footer
Add a custom sidebar
Clone this wiki locally