New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
rgw: frontend subsystem rework #10767
Commits on Oct 21, 2016
-
rgw: add submodule for Beast header-only library
Beast is a c++ http protocol library build on top of boost::asio Signed-off-by: Casey Bodley <cbodley@redhat.com> Signed-off-by: Matt Benjamin <mbenjamin@redhat.com> Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: add skeleton for boost::asio frontend
Signed-off-by: Casey Bodley <cbodley@redhat.com> Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
vstart: add --rgw_frontend option
Signed-off-by: Casey Bodley <cbodley@redhat.com>
-
rgw: implement Frontend interface for asio
Signed-off-by: Casey Bodley <cbodley@redhat.com>
-
rgw: RGWEnv::set takes boost::string_ref
Signed-off-by: Casey Bodley <cbodley@redhat.com>
-
rgw: asio frontend calls process_request
Signed-off-by: Casey Bodley <cbodley@redhat.com>
-
rgw: implement ClientIO interface for asio + beast
Signed-off-by: Casey Bodley <cbodley@redhat.com> Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: asio frontend uses async_read
Signed-off-by: Casey Bodley <cbodley@redhat.com>
-
rgw: s3 GetObj sends status first
Signed-off-by: Casey Bodley <cbodley@redhat.com>
-
rgw: s3 PutObj sends status first
Signed-off-by: Casey Bodley <cbodley@redhat.com>
-
rgw: mark asio frontend as experimental
Users will need to set "rgw-asio-frontend" as an experimental feature. Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
-
rgw: introduce RGWStreamIOBase to separate all non-IO related things.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: move transfer accounting management from RGWClientIO to RGWStrea…
…mIO. Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: improve const-correctness in the front-end infrastructure.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: strip out AWSv4 signature calculation from RGWStreamIOBase.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: further cleaning in RGWStreamIOBase.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: rename RGWStreamIOBase to RGWStreamIOEngine.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: add basic infrastructure for decorators over RGWStreamIOBase.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: dissect accounting interface from RGWClientIO.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: dissect non-core ops from RGWStreamIOEngine to RGWStreamIOFacade.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: implement RGWStreamIOLegacyWrapper for using new front-ends with…
… RGWStreamIO. Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: RGWMongoose uses RGWStreamIOEngine instead of RGWStreamIO.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: implement RGWStreamIOAccountingEngine.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: RGWClientIO::get_env() is pure virtual now.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: implement RGWStreamIOBufferingEngine.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: eradicate dynamic memory allocations from RGWEnv.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: ONLY rename RGWMongoose to RGWCivetWeb and marks overrides.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: ONLY rename RGWMongooseFrontend to RGWCivetWebFrontend.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: implement RGWStreamIOConLenControllingEngine.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: implement RGWStreamIOReorderingEngine.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: implement RGWStreamIOChunkingEngine.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: switch out RGWCivetWeb to the new frontend infrastructure.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: cleaning in the glue code for CivetWeb.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: port the ASIO frontend to the new RGWStreamIOEngine infrastructure.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: port the FCGI frontend to the new RGWStreamIOEngine infrastructure.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: ONLY improve formatting in the FCGI frontend.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: port the LoadGen frontend to the new RGWStreamIOEngine infrastru…
…cture. Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: cleanup in the LoadGen frontend.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: handle errors via exceptions in RGWStreamIOEngine.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: switch to std::size_t in the front-end infrastructure.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: add support for exhibiting URL subtree over a specifc frontend i…
…nstance. Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: ONLY reformat RGWFrontend and RGWCivetWebFrontend.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: implement RGWStreamIO[Engine]::send_header().
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: switch to RGWStreamIO::send_header().
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: drop RGWStreamIOFacade::print().
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: RGWStreamIOReorderingEngine understands send_header().
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: implement RGWStreamIOEngine::{send,recv}_body().
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: switch to RGWStreamIOEngine::{send,recv}_body().
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: rework quoting value of HTTP headers.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: add control logic for chunked transfer encoding.
Needed for tempest.api.object_storage.test_object_slo.ObjectSloTest.test_delete_large_object. Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: replace dump_pair() with dump_header_if_nonempty().
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: ONLY move RGWDecoratedStreamIO.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: remove the legacy, non-throwing variant of RGWStreamIO.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: change naming based on RGWStreamIO in favor of RGWRestfulIO.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: delegate transfer accounting to RGWRestfulIOAccountingEngine.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: rename and split STREAM_IO into RESTFUL_IO and ACCOUNTING_IO.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: make RGWRestfulIOEngine::Exception alias to std::system_error.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: ONLY replace std::size_t with size_t.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: switch to namespaces and change naming in the frontend subsystem.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: fix off-by-one in RGWAsioClientIO::init_env.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: fix setting HTTP/0.0 in AsioConnection::write_bad_request.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: improve debugs in RGWAsioClientIO and AsioConnection.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: employ necessary filter over RGWFCGX frontend.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: use filters over AsioConnection.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: switch to size_t and exception in rgw::io::BasicClient::complete…
…_request. Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: fix perfect forwarding of template arguments in IO filters.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: add support for dispatching requests with respect to frontend pr…
…efix. Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: protect the {ACCOUNTING,AWS_AUTHv4,RESTFUL}_IO casts with assert.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: ONLY move rgw_client_io_decoimpl.h to rgw_client_io_filters.h.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: don't use boost::string_ref::to_string() method.
The method has been introduced in Boost 1.54. However, some systems we are targeting (like CentOS 7) offer older version of the library. Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: document interfaces of the rgw::io namespace.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: fix interface compliance of RGWCivetWeb::write_data().
This is a potential fix for partial writes and handling of error codes that might come from mg_write() of the CivetWeb front-end. None of the potential issues has been observed in testing. The commit also documents the same aspects regarding the ASIO and FastCGI front-ends. Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: make the ASIO front-end compilation optional.
This is because of versioning issues between Boost and Beast on CentOS 7. It is intended that this patch will be reverted after merging the in-tree Boost facility. Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
Commits on Oct 25, 2016
-
rgw: sanitize length of metadata attributes.
std::string and thus boost::string_ref ARE OBLIGED to carry multiple 0x00 and count them to the length of a string. We need to take that into consideration and sanitize the size of a ceph::buffer::list used to store metadata values (x-amz-meta-*, X-Container-Meta-*, etags). Otherwise we might send 0x00 to clients. Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
-
rgw: implement buffered HTTP header composition in front-ends.
Without the patch front-ends issueed a lot of small IO operations leading to increased overhead on syscalls and to the fragmentation of an HTTP message across multiple TCP segments. The later was occuring when the Nagle's algorithm hadn't been able to form a single TCP segment (usually when running on extremely fast network interfaces like loopback). Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>