s3_compliance.rst

Rados Gateway S3 API Compliance

Warning

This document is a draft, it might not be accurate

Naming code reference

Here comes a BNF definition on how to name a feature in the code for referencing purpose : :

name ::= request_type "_" ( header | operation ) ( "_" header_option )?

request_type ::= "req" | "res"

header ::= string

operation ::= method resource

method ::= "GET" | "PUT" | "POST" | "DELETE" | "OPTIONS" | "HEAD"

resource ::= string

header_option ::= string

Common Request Headers

S3 Documentation reference : http://docs.aws.amazon.com/AmazonS3/latest/API/RESTCommonRequestHeaders.html

Header	Supported?	Code Links	Tests links
Authorization	Yes	ceph/src/rgw/rgw_rest_s3.cc Line 1962 in 8a2eb18 int RGW_Auth_S3::authorize(RGWRados *store, struct req_state *s) ceph/src/rgw/rgw_rest_s3.cc Line 2051 in 8a2eb18 if (auth_sign != digest)
Content-Length	Yes
Content-Type	Yes
Content-MD5	Yes	ceph/src/rgw/rgw_op.cc Line 1249 in b139a7c supplied_md5_b64, supplied_md5_b64 + strlen(supplied_md5_b64)); ceph/src/rgw/rgw_op.cc Line 1306 in b139a7c buf_to_hex(m, CEPH_CRYPTO_MD5_DIGESTSIZE, calc_md5);
Date	Yes	ceph/src/rgw/rgw_auth_s3.cc Line 164 in 8a2eb18 const char *str = info.env->get("HTTP_DATE");
Expect	Yes	ceph/src/rgw/rgw_rest.cc Line 1227 in 8a2eb18 s->expect_cont = (expect && !strcasecmp(expect, "100-continue")); ceph/src/rgw/rgw_rest_s3.cc Line 802 in 8a2eb18 if (s->expect_cont) { ceph/src/rgw/rgw_main.cc Line 372 in 76040d9 if (s->expect_cont)
Host	?
x-amz-date	Yes	ceph/src/rgw/rgw_auth_s3.cc Line 169 in 8a2eb18 req_date = info.env->get("HTTP_X_AMZ_DATE"); should take precedence over DATE as mentioned here -> http://docs.aws.amazon.com/AmazonS3/latest/API/RESTCommonRequestHeaders.html
x-amz-security-token	No

Common Response Headers

S3 Documentation reference : http://docs.aws.amazon.com/AmazonS3/latest/API/RESTCommonResponseHeaders.html

Header	Supported?	Code Links	Tests links
Content-Length	Yes
Connection	?
Date	?
ETag	Yes	ceph/src/rgw/rgw_op.cc Line 1312 in b139a7c policy.encode(aclbl); ceph/src/rgw/rgw_op.cc Line 1436 in b139a7c ceph/src/rgw/rgw_op.cc Line 2222 in b139a7c attrs[RGW_ATTR_ETAG] = etag_bl; ceph/src/rgw/rgw_rest_s3.cc Line 118 in 8a2eb18 dump_etag(s, etag); ceph/src/rgw/rgw_rest_s3.cc Line 268 in 8a2eb18 s->formatter->dump_format("ETag", "\"%s\"", iter->etag.c_str()); ceph/src/rgw/rgw_rest_s3.cc Line 516 in 8a2eb18 dump_etag(s, etag.c_str()); ceph/src/rgw/rgw_rest_s3.cc Line 1336 in 8a2eb18 s->formatter->dump_string("ETag", etag); ceph/src/rgw/rgw_rest_s3.cc Line 1486 in 8a2eb18 s->formatter->dump_string("ETag", etag); ceph/src/rgw/rgw_rest_s3.cc Line 1548 in 8a2eb18 s->formatter->dump_string("ETag", info.etag);
Server	No
x-amz-delete-marker	No
x-amz-id-2	No
x-amz-request-id	Yes	https://github.com/ceph/ceph/commit/b711e3124f8f73c17ebd19b38807a1b77f201e44
x-amz-version-id	No

Operations on the Service

S3 Documentation reference : http://docs.aws.amazon.com/AmazonS3/latest/API/RESTServiceOps.html

Type	Operation	Supported?	Code links	Tests links
GET	Service	Yes	ceph/src/rgw/rgw_rest_s3.cc Line 2094 in 8a2eb18 return new RGWHandler_ObjStore_Service_S3; ceph/src/rgw/rgw_rest_s3.cc Line 1676 in 8a2eb18 return new RGWListBuckets_ObjStore_S3; ceph/src/rgw/rgw_rest_s3.cc Line 185 in 8a2eb18 void RGWListBuckets_ObjStore_S3::send_response_begin(bool has_buckets)

Operations on Buckets

S3 Documentation reference : http://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketOps.html

Type	Operation	Supported?	Code links	Tests links
DELETE	Bucket	Yes	ceph/src/rgw/rgw_rest_s3.cc Line 1728 in 8a2eb18 RGWOp *RGWHandler_ObjStore_Bucket_S3::op_delete() ceph/src/rgw/rgw_rest_bucket.cc Line 250 in e910421 RGWOp *RGWHandler_Bucket::op_delete() ceph/src/rgw/rgw_rest_bucket.cc Line 212 in e910421 void RGWOp_Object_Remove::execute() ceph/src/rgw/rgw_bucket.cc Line 856 in 2594831 int RGWBucketAdminOp::remove_object(RGWRados *store, RGWBucketAdminOpState& op_state) ceph/src/rgw/rgw_bucket.cc Line 513 in 2594831 int RGWBucket::remove_object(RGWBucketAdminOpState& op_state, std::string *err_msg) ceph/src/rgw/rgw_bucket.cc Line 286 in 2594831 int rgw_remove_object(RGWRados *store, rgw_bucket& bucket, std::string& object) ceph/src/rgw/rgw_rest_s3.cc Line 461 in 8a2eb18 void RGWDeleteBucket_ObjStore_S3::send_response()
DELETE	Bucket cors	?	ceph/src/rgw/rgw_rest_s3.cc Line 1731 in 8a2eb18 return new RGWDeleteCORS_ObjStore_S3; ceph/src/rgw/rgw_op.cc Line 1916 in b139a7c
DELETE	Bucket lifecycle	No
DELETE	Bucket policy	?
DELETE	Bucket tagging	?
DELETE	Bucket website	No
GET	Bucket	Yes	ceph/src/rgw/rgw_rest_s3.cc Line 1676 in 8a2eb18 return new RGWListBuckets_ObjStore_S3; ceph/src/rgw/rgw_rest_s3.cc Line 185 in 8a2eb18 void RGWListBuckets_ObjStore_S3::send_response_begin(bool has_buckets)
GET	Bucket acl	Yes	ceph/src/rgw/rgw_rest_s3.cc Line 1697 in 8a2eb18 return new RGWGetACLs_ObjStore_S3; ceph/src/rgw/rgw_op.cc Line 1728 in b139a7c void RGWGetACLs::execute() ceph/src/rgw/rgw_rest_s3.cc Line 1344 in 8a2eb18 void RGWGetACLs_ObjStore_S3::send_response()
GET	Bucket cors	?	ceph/src/rgw/rgw_rest_s3.cc Line 1698 in 8a2eb18 } else if (is_cors_op()) { ceph/src/rgw/rgw_op.cc Line 1845 in b139a7c void RGWGetCORS::execute() ceph/src/rgw/rgw_main.cc Line 345 in 76040d9 handler->read_cors_config();
GET	Bucket lifecycle	No
GET	Bucket location	No
GET	Bucket policy	?	ceph/src/rgw/rgw_rest_bucket.cc Line 232 in e910421 return new RGWOp_Get_Policy; ceph/src/rgw/rgw_rest_bucket.cc Line 58 in e910421 void RGWOp_Get_Policy::execute()
GET	Bucket logging	?	ceph/src/rgw/rgw_rest_s3.cc Line 1695 in 8a2eb18 return new RGWGetBucketLogging_ObjStore_S3; ceph/src/rgw/rgw_rest_s3.cc Line 287 in 8a2eb18 void RGWGetBucketLogging_ObjStore_S3::send_response()
GET	Bucket notification	No
GET	Bucket tagging	No
GET	Bucket Object versions	No
GET	Bucket requestPayment	No
GET	Bucket versioning	No
GET	Bucket website	No
GET	List Multipart uploads	Yes	ceph/src/rgw/rgw_rest_s3.cc Line 1701 in 8a2eb18 return new RGWListBucketMultiparts_ObjStore_S3; ceph/src/rgw/rgw_rest.cc Line 877 in 8a2eb18 int RGWListMultipart_ObjStore::get_params() ceph/src/rgw/rgw_op.cc Line 2355 in b139a7c int RGWListBucketMultiparts::verify_permission() ceph/src/rgw/rgw_op.cc Line 2363 in b139a7c void RGWListBucketMultiparts::execute()
HEAD	Bucket	Yes	ceph/src/rgw/rgw_rest_s3.cc Line 1713 in 8a2eb18 return get_obj_op(false); ceph/src/rgw/rgw_rest_s3.cc Line 1689 in 8a2eb18 return new RGWStatBucket_ObjStore_S3; ceph/src/rgw/rgw_op.cc Line 826 in b139a7c int RGWStatBucket::verify_permission() ceph/src/rgw/rgw_op.cc Line 834 in b139a7c void RGWStatBucket::execute()
PUT	Bucket	Yes	ceph/src/rgw/rgw_rest_s3.cc Line 1725 in 8a2eb18 return new RGWCreateBucket_ObjStore_S3; ceph/src/rgw/rgw_rest_s3.cc Line 382 in 8a2eb18 int RGWCreateBucket_ObjStore_S3::get_params() ceph/src/rgw/rgw_rest_s3.cc Line 437 in 8a2eb18 void RGWCreateBucket_ObjStore_S3::send_response() ceph/src/rgw/rgw_op.cc Line 901 in b139a7c int RGWCreateBucket::verify_permission() ceph/src/rgw/rgw_op.cc Line 945 in b139a7c void RGWCreateBucket::execute()
PUT	Bucket acl	Yes	ceph/src/rgw/rgw_rest_s3.cc Line 1721 in 8a2eb18 return new RGWPutACLs_ObjStore_S3; ceph/src/rgw/rgw_rest_s3.cc Line 1354 in 8a2eb18 int RGWPutACLs_ObjStore_S3::get_policy_from_state(RGWRados *store, struct req_state *s, stringstream& ss) ceph/src/rgw/rgw_rest_s3.cc Line 1373 in 8a2eb18 void RGWPutACLs_ObjStore_S3::send_response() ceph/src/rgw/rgw_op.cc Line 1739 in b139a7c int RGWPutACLs::verify_permission() ceph/src/rgw/rgw_op.cc Line 1753 in b139a7c void RGWPutACLs::execute()
PUT	Bucket cors	?	ceph/src/rgw/rgw_rest_s3.cc Line 1723 in 8a2eb18 return new RGWPutCORS_ObjStore_S3; ceph/src/rgw/rgw_rest_s3.cc Line 1398 in 8a2eb18 void RGWPutCORS_ObjStore_S3::send_response() ceph/src/rgw/rgw_op.cc Line 1858 in b139a7c int RGWPutCORS::verify_permission() ceph/src/rgw/rgw_op.cc Line 1866 in b139a7c void RGWPutCORS::execute()
PUT	Bucket lifecycle	No
PUT	Bucket policy	?
PUT	Bucket logging	?
PUT	Bucket notification	No
PUT	Bucket tagging	?
PUT	Bucket requestPayment	No
PUT	Bucket versioning	No
PUT	Bucket website	No

Operations on Objects

S3 Documentation reference : http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectOps.html

Type	Operation	Supported?	Code links	Tests links
DELETE	Object	Yes	ceph/src/rgw/rgw_rest_s3.cc Line 1796 in 8a2eb18 return new RGWDeleteObj_ObjStore_S3; ceph/src/rgw/rgw_op.cc Line 1516 in b139a7c int RGWDeleteObj::verify_permission() ceph/src/rgw/rgw_op.cc Line 1524 in b139a7c void RGWDeleteObj::execute()
DELETE	Multiple objects	Yes	ceph/src/rgw/rgw_rest_s3.cc Line 1739 in 8a2eb18 return new RGWDeleteMultiObj_ObjStore_S3; ceph/src/rgw/rgw_rest_s3.cc Line 1616 in 8a2eb18 void RGWDeleteMultiObj_ObjStore_S3::send_status() ceph/src/rgw/rgw_rest_s3.cc Line 1626 in 8a2eb18 void RGWDeleteMultiObj_ObjStore_S3::begin_response() ceph/src/rgw/rgw_rest_s3.cc Line 1641 in 8a2eb18 void RGWDeleteMultiObj_ObjStore_S3::send_partial_response(pair<string,int>& result) ceph/src/rgw/rgw_rest_s3.cc Line 1667 in 8a2eb18 void RGWDeleteMultiObj_ObjStore_S3::end_response() ceph/src/rgw/rgw_op.cc Line 1516 in b139a7c int RGWDeleteObj::verify_permission() ceph/src/rgw/rgw_op.cc Line 1524 in b139a7c void RGWDeleteObj::execute()
GET	Object	Yes	ceph/src/rgw/rgw_rest_s3.cc Line 1767 in 8a2eb18 return get_obj_op(true); ceph/src/rgw/rgw_rest_s3.cc Line 71 in 8a2eb18 int RGWGetObj_ObjStore_S3::send_response_data(bufferlist& bl, off_t bl_ofs, off_t bl_len) ceph/src/rgw/rgw_op.cc Line 397 in b139a7c int RGWGetObj::verify_permission() ceph/src/rgw/rgw_op.cc Line 424 in b139a7c int RGWGetObj::read_user_manifest_part(rgw_bucket& bucket, RGWObjEnt& ent, RGWAccessControlPolicy *bucket_policy, off_t start_ofs, off_t end_ofs) ceph/src/rgw/rgw_op.cc Line 497 in b139a7c int RGWGetObj::iterate_user_manifest_parts(rgw_bucket& bucket, string& obj_prefix, RGWAccessControlPolicy *bucket_policy, ceph/src/rgw/rgw_op.cc Line 562 in b139a7c int RGWGetObj::handle_user_manifest(const char *prefix) ceph/src/rgw/rgw_op.cc Line 626 in b139a7c int RGWGetObj::get_data_cb(bufferlist& bl, off_t bl_ofs, off_t bl_len) ceph/src/rgw/rgw_op.cc Line 641 in b139a7c void RGWGetObj::execute() ceph/src/rgw/rgw_op.cc Line 706 in b139a7c int RGWGetObj::init_common()
GET	Object acl	Yes
GET	Object torrent	No
HEAD	Object	Yes	ceph/src/rgw/rgw_rest_s3.cc Line 1777 in 8a2eb18 return get_obj_op(false); ceph/src/rgw/rgw_rest_s3.cc Line 71 in 8a2eb18 int RGWGetObj_ObjStore_S3::send_response_data(bufferlist& bl, off_t bl_ofs, off_t bl_len) ceph/src/rgw/rgw_op.cc Line 397 in b139a7c int RGWGetObj::verify_permission() ceph/src/rgw/rgw_op.cc Line 424 in b139a7c int RGWGetObj::read_user_manifest_part(rgw_bucket& bucket, RGWObjEnt& ent, RGWAccessControlPolicy *bucket_policy, off_t start_ofs, off_t end_ofs) ceph/src/rgw/rgw_op.cc Line 497 in b139a7c int RGWGetObj::iterate_user_manifest_parts(rgw_bucket& bucket, string& obj_prefix, RGWAccessControlPolicy *bucket_policy, ceph/src/rgw/rgw_op.cc Line 562 in b139a7c int RGWGetObj::handle_user_manifest(const char *prefix) ceph/src/rgw/rgw_op.cc Line 626 in b139a7c int RGWGetObj::get_data_cb(bufferlist& bl, off_t bl_ofs, off_t bl_len) ceph/src/rgw/rgw_op.cc Line 641 in b139a7c void RGWGetObj::execute() ceph/src/rgw/rgw_op.cc Line 706 in b139a7c int RGWGetObj::init_common()
OPTIONS	Object	Yes	ceph/src/rgw/rgw_rest_s3.cc Line 1814 in 8a2eb18 return new RGWOptionsCORS_ObjStore_S3; ceph/src/rgw/rgw_rest_s3.cc Line 1418 in 8a2eb18 void RGWOptionsCORS_ObjStore_S3::send_response() ceph/src/rgw/rgw_op.cc Line 1951 in b139a7c void RGWOptionsCORS::get_response_params(string& hdrs, string& exp_hdrs, unsigned *max_age) { ceph/src/rgw/rgw_op.cc Line 1968 in b139a7c int RGWOptionsCORS::validate_cors_request() { ceph/src/rgw/rgw_op.cc Line 1993 in b139a7c void RGWOptionsCORS::execute()
POST	Object	Yes	ceph/src/rgw/rgw_rest_s3.cc Line 1742 in 8a2eb18 return new RGWPostObj_ObjStore_S3; ceph/src/rgw/rgw_rest_s3.cc Line 631 in 8a2eb18 int RGWPostObj_ObjStore_S3::read_with_boundary(bufferlist& bl, uint64_t max, bool check_crlf, ceph/src/rgw/rgw_rest_s3.cc Line 694 in 8a2eb18 int RGWPostObj_ObjStore_S3::read_line(bufferlist& bl, uint64_t max, ceph/src/rgw/rgw_rest_s3.cc Line 700 in 8a2eb18 int RGWPostObj_ObjStore_S3::read_data(bufferlist& bl, uint64_t max, ceph/src/rgw/rgw_rest_s3.cc Line 707 in 8a2eb18 int RGWPostObj_ObjStore_S3::read_form_part_header(struct post_form_part *part, ceph/src/rgw/rgw_rest_s3.cc Line 759 in 8a2eb18 bool RGWPostObj_ObjStore_S3::part_str(const string& name, string *val) ceph/src/rgw/rgw_rest_s3.cc Line 771 in 8a2eb18 bool RGWPostObj_ObjStore_S3::part_bl(const string& name, bufferlist *pbl) ceph/src/rgw/rgw_rest_s3.cc Line 781 in 8a2eb18 void RGWPostObj_ObjStore_S3::rebuild_key(string& key) ceph/src/rgw/rgw_rest_s3.cc Line 795 in 8a2eb18 int RGWPostObj_ObjStore_S3::get_params() ceph/src/rgw/rgw_rest_s3.cc Line 929 in 8a2eb18 int RGWPostObj_ObjStore_S3::get_policy() ceph/src/rgw/rgw_rest_s3.cc Line 1037 in 8a2eb18 int RGWPostObj_ObjStore_S3::complete_get_params() ceph/src/rgw/rgw_rest_s3.cc Line 1059 in 8a2eb18 int RGWPostObj_ObjStore_S3::get_data(bufferlist& bl) ceph/src/rgw/rgw_rest_s3.cc Line 1134 in 8a2eb18 void RGWPostObj_ObjStore_S3::send_response() ceph/src/rgw/rgw_op.cc Line 1344 in b139a7c int RGWPostObj::verify_permission() ceph/src/rgw/rgw_op.cc Line 1360 in b139a7c void RGWPostObj::dispose_processor(RGWPutObjProcessor *processor) ceph/src/rgw/rgw_op.cc Line 1365 in b139a7c void RGWPostObj::execute()
POST	Object restore	?
PUT	Object	Yes
PUT	Object acl	Yes
PUT	Object copy	Yes
PUT	Initiate multipart upload	Yes
PUT	Upload Part	Yes
PUT	Upload Part copy	?
PUT	Complete multipart upload	Yes
PUT	Abort multipart upload	Yes
PUT	List parts	Yes