Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ratelimit: support returning custom response bodies for non-OK responses from the external ratelimit service #14189

Merged
merged 20 commits into from
Dec 16, 2020
Merged

ratelimit: support returning custom response bodies for non-OK responses from the external ratelimit service #14189

Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
9967154
ratelimit: support returning custom response bodies for
Sep 22, 2020
d6bdebf
test/common/network/filter_manager_impl_test: fix test
esmet Nov 25, 2020
9b29a79
test/extensions/filters/network/thrift_proxy: fix test
esmet Nov 25, 2020
c76e1f3
test/extensions/filters/common/ratelimit/ratelimit_impl_test: fix test
esmet Nov 25, 2020
b8134f9
test/extensions/filters/network/ratelimit/ratelimit_test: fix test
esmet Nov 30, 2020
f4fb93d
ratelimit: rename overwrite_content_type -> from_local_reply
esmet Dec 2, 2020
32cb168
fix test
esmet Dec 2, 2020
68eec79
Simplify
esmet Dec 2, 2020
d340935
Add release notes
esmet Dec 2, 2020
e1492f1
Fix release notes
esmet Dec 3, 2020
1fd9496
Fix
esmet Dec 3, 2020
88fa661
Clean up test comments
esmet Dec 3, 2020
7346fa6
Raw string literal
esmet Dec 3, 2020
7c018bc
body -> body_bytes
esmet Dec 11, 2020
582eb42
body -> raw_body
esmet Dec 11, 2020
c2a236a
Address review comments
esmet Dec 11, 2020
108e9ec
Fix docs link
esmet Dec 11, 2020
d4e5eb2
Kick CI
esmet Dec 12, 2020
2b49ffe
Kick CI
esmet Dec 12, 2020
b3143e7
Merge remote-tracking branch 'origin/master' into ratelimit-response-…
esmet Dec 14, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 4 additions & 0 deletions api/envoy/service/ratelimit/v3/rls.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ message RateLimitRequest {
}

// A response from a ShouldRateLimit call.
// [#next-free-field: 6]
message RateLimitResponse {
option (udpa.annotations.versioning).previous_message_type =
"envoy.service.ratelimit.v2.RateLimitResponse";
Expand Down Expand Up @@ -131,4 +132,7 @@ message RateLimitResponse {

// A list of headers to add to the request when forwarded
repeated config.core.v3.HeaderValue request_headers_to_add = 4;

// A response body to send to the downstream client when the response code is not OK.
string body = 5;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be a DataSource (e.g. see what's done in DirectResponseAction)?

There are other places in the API where we even allow things like log template values to be included and JSON returned, e.g. SubstitutionFormatString in HCM local reply mapper. That could be another reasonable way to future proof and align with the existing precedent for this kind of body.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This body comes from the external ratelimit service so I think the better comparison would be the auth response message in the ext_authz filter.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To add more context, I think the DirectResponseAction is used to configure a response generated locally at Envoy. That means you can configure a data source local to the Envoy container and have access to variables within Envoy's SubstitutionFormatString. Since this body is configuring a response generated externally in the RateLimitService as part of its RateLimitResponse, it makes sense to represent that as just a string on the wire as protobuf (as we do in ext_authz https://github.com/envoyproxy/envoy/blob/master/api/envoy/service/auth/v3/external_auth.proto#L59).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, makes sense!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we please make this bytes and not a string? We have endless problems where we have things that proto can't encode in a string field and technically body can be raw bytes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. We'll want to make the same change to ext_authz by adding a new body_bytes field and deprecating the existing string body field. For consistency, I'll change this field to also be body_bytes with type bytes. This way ext_authz and ratelimit will have a similar UX (which was part of the goal of this PR). Does that work?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Up to you whether you want to do the deprecation dance on extauth but for new API I would definitely do it with bytes.

}
1 change: 1 addition & 0 deletions docs/root/version_history/current.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ New Features
* overload: add :ref:`envoy.overload_actions.reduce_timeouts <config_overload_manager_overload_actions>` overload action to enable scaling timeouts down with load.
* ratelimit: added support for use of various :ref:`metadata <envoy_v3_api_field_config.route.v3.RateLimit.Action.metadata>` as a ratelimit action.
* ratelimit: added :ref:`disable_x_envoy_ratelimited_header <envoy_v3_api_msg_extensions.filters.http.ratelimit.v3.RateLimit>` option to disable `X-Envoy-RateLimited` header.
* ratelimit: added :ref:`body <envoy_v3_api_field_service.ratelimit.v3.RateLimitResponse.body>` field to support custom response bodies for non-OK responses from the external ratelimit service.
* sds: improved support for atomic :ref:`key rotations <xds_certificate_rotation>` and added configurable rotation triggers for
:ref:`TlsCertificate <envoy_v3_api_field_extensions.transport_sockets.tls.v3.TlsCertificate.watched_directory>` and
:ref:`CertificateValidationContext <envoy_v3_api_field_extensions.transport_sockets.tls.v3.CertificateValidationContext.watched_directory>`.
Expand Down
4 changes: 4 additions & 0 deletions generated_api_shadow/envoy/service/ratelimit/v3/rls.proto
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion source/extensions/filters/common/ratelimit/ratelimit.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,8 @@ class RequestCallbacks {
*/
virtual void complete(LimitStatus status, DescriptorStatusListPtr&& descriptor_statuses,
Http::ResponseHeaderMapPtr&& response_headers_to_add,
Http::RequestHeaderMapPtr&& request_headers_to_add) PURE;
Http::RequestHeaderMapPtr&& request_headers_to_add,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional: While you're here, could you add Doxygen style-param to this? I think it will be useful to note that response_body can contain non-UTF8 value.

const std::string& response_body) PURE;
};

/**
Expand Down
4 changes: 2 additions & 2 deletions source/extensions/filters/common/ratelimit/ratelimit_impl.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,14 +106,14 @@ void GrpcClientImpl::onSuccess(
DescriptorStatusListPtr descriptor_statuses = std::make_unique<DescriptorStatusList>(
response->statuses().begin(), response->statuses().end());
callbacks_->complete(status, std::move(descriptor_statuses), std::move(response_headers_to_add),
std::move(request_headers_to_add));
std::move(request_headers_to_add), response->body());
callbacks_ = nullptr;
}

void GrpcClientImpl::onFailure(Grpc::Status::GrpcStatus status, const std::string&,
Tracing::Span&) {
ASSERT(status != Grpc::Status::WellKnownGrpcStatus::Ok);
callbacks_->complete(LimitStatus::Error, nullptr, nullptr, nullptr);
callbacks_->complete(LimitStatus::Error, nullptr, nullptr, nullptr, "");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit. EMPTY_STRING

callbacks_ = nullptr;
}

Expand Down
25 changes: 18 additions & 7 deletions source/extensions/filters/http/ratelimit/ratelimit.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ Http::FilterHeadersStatus Filter::encode100ContinueHeaders(Http::ResponseHeaderM
}

Http::FilterHeadersStatus Filter::encodeHeaders(Http::ResponseHeaderMap& headers, bool) {
populateResponseHeaders(headers);
populateResponseHeaders(headers, false);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional. Nit. populateResponseHeaders(headers, /*from_local_reply=*/false);. Following @htuch's way on annotating boolean params, whch I found realy useful for code reading. :).

return Http::FilterHeadersStatus::Continue;
}

Expand Down Expand Up @@ -143,7 +143,8 @@ void Filter::onDestroy() {
void Filter::complete(Filters::Common::RateLimit::LimitStatus status,
Filters::Common::RateLimit::DescriptorStatusListPtr&& descriptor_statuses,
Http::ResponseHeaderMapPtr&& response_headers_to_add,
Http::RequestHeaderMapPtr&& request_headers_to_add) {
Http::RequestHeaderMapPtr&& request_headers_to_add,
const std::string& response_body) {
state_ = State::Complete;
response_headers_to_add_ = std::move(response_headers_to_add);
Http::HeaderMapPtr req_headers_to_add = std::move(request_headers_to_add);
Expand Down Expand Up @@ -195,8 +196,8 @@ void Filter::complete(Filters::Common::RateLimit::LimitStatus status,
config_->runtime().snapshot().featureEnabled("ratelimit.http_filter_enforcing", 100)) {
state_ = State::Responded;
callbacks_->sendLocalReply(
Http::Code::TooManyRequests, "",
[this](Http::HeaderMap& headers) { populateResponseHeaders(headers); },
Http::Code::TooManyRequests, response_body,
[this](Http::HeaderMap& headers) { populateResponseHeaders(headers, true); },
config_->rateLimitedGrpcStatus(), RcDetails::get().RateLimited);
callbacks_->streamInfo().setResponseFlag(StreamInfo::ResponseFlag::RateLimited);
} else if (status == Filters::Common::RateLimit::LimitStatus::Error) {
Expand All @@ -208,8 +209,8 @@ void Filter::complete(Filters::Common::RateLimit::LimitStatus status,
}
} else {
state_ = State::Responded;
callbacks_->sendLocalReply(Http::Code::InternalServerError, "", nullptr, absl::nullopt,
RcDetails::get().RateLimitError);
callbacks_->sendLocalReply(Http::Code::InternalServerError, response_body, nullptr,
absl::nullopt, RcDetails::get().RateLimitError);
callbacks_->streamInfo().setResponseFlag(StreamInfo::ResponseFlag::RateLimitServiceError);
}
} else if (!initiating_call_) {
Expand All @@ -236,8 +237,18 @@ void Filter::populateRateLimitDescriptors(const Router::RateLimitPolicy& rate_li
}
}

void Filter::populateResponseHeaders(Http::HeaderMap& response_headers) {
void Filter::populateResponseHeaders(Http::HeaderMap& response_headers, bool from_local_reply) {
if (response_headers_to_add_) {
// If the ratelimit service is sending back the content-type header and we're
// populating response headers for a local reply, overwrite the existing
// content-type header.
//
// We do this because sendLocalReply initially sets content-type to text/plain
// whenever the response body is non-empty, but we want the content-type coming
// from the ratelimit service to be authoritative in this case.
if (from_local_reply && !response_headers_to_add_->getContentTypeValue().empty()) {
response_headers.remove(Http::Headers::get().ContentType);
}
Http::HeaderUtility::addHeaders(response_headers, *response_headers_to_add_);
response_headers_to_add_ = nullptr;
}
Expand Down
5 changes: 3 additions & 2 deletions source/extensions/filters/http/ratelimit/ratelimit.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -148,15 +148,16 @@ class Filter : public Http::StreamFilter, public Filters::Common::RateLimit::Req
void complete(Filters::Common::RateLimit::LimitStatus status,
Filters::Common::RateLimit::DescriptorStatusListPtr&& descriptor_statuses,
Http::ResponseHeaderMapPtr&& response_headers_to_add,
Http::RequestHeaderMapPtr&& request_headers_to_add) override;
Http::RequestHeaderMapPtr&& request_headers_to_add,
const std::string& response_body) override;

private:
void initiateCall(const Http::RequestHeaderMap& headers);
void populateRateLimitDescriptors(const Router::RateLimitPolicy& rate_limit_policy,
std::vector<Envoy::RateLimit::Descriptor>& descriptors,
const Router::RouteEntry* route_entry,
const Http::HeaderMap& headers) const;
void populateResponseHeaders(Http::HeaderMap& response_headers);
void populateResponseHeaders(Http::HeaderMap& response_headers, bool from_local_reply);
void appendRequestHeaders(Http::HeaderMapPtr& request_headers_to_add);
VhRateLimitOptions getVirtualHostRateLimitOption(const Router::RouteConstSharedPtr& route);

Expand Down
3 changes: 2 additions & 1 deletion source/extensions/filters/network/ratelimit/ratelimit.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,8 @@ void Filter::onEvent(Network::ConnectionEvent event) {

void Filter::complete(Filters::Common::RateLimit::LimitStatus status,
Filters::Common::RateLimit::DescriptorStatusListPtr&&,
Http::ResponseHeaderMapPtr&&, Http::RequestHeaderMapPtr&&) {
Http::ResponseHeaderMapPtr&&, Http::RequestHeaderMapPtr&&,
const std::string&) {
status_ = Status::Complete;
config_->stats().active_.dec();

Expand Down
3 changes: 2 additions & 1 deletion source/extensions/filters/network/ratelimit/ratelimit.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,8 @@ class Filter : public Network::ReadFilter,
void complete(Filters::Common::RateLimit::LimitStatus status,
Filters::Common::RateLimit::DescriptorStatusListPtr&& descriptor_statuses,
Http::ResponseHeaderMapPtr&& response_headers_to_add,
Http::RequestHeaderMapPtr&& request_headers_to_add) override;
Http::RequestHeaderMapPtr&& request_headers_to_add,
const std::string& response_body) override;

private:
enum class Status { NotStarted, Calling, Complete };
Expand Down
2 changes: 1 addition & 1 deletion source/extensions/filters/network/thrift_proxy/filters/ratelimit/ratelimit.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ void Filter::onDestroy() {
void Filter::complete(Filters::Common::RateLimit::LimitStatus status,
Filters::Common::RateLimit::DescriptorStatusListPtr&& descriptor_statuses,
Http::ResponseHeaderMapPtr&& response_headers_to_add,
Http::RequestHeaderMapPtr&& request_headers_to_add) {
Http::RequestHeaderMapPtr&& request_headers_to_add, const std::string&) {
// TODO(zuercher): Store headers to append to a response. Adding them to a local reply (over
// limit or error) is a matter of modifying the callbacks to allow it. Adding them to an upstream
// response requires either response (aka encoder) filters or some other mechanism.
Expand Down
3 changes: 2 additions & 1 deletion source/extensions/filters/network/thrift_proxy/filters/ratelimit/ratelimit.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,8 @@ class Filter : public ThriftProxy::ThriftFilters::PassThroughDecoderFilter,
void complete(Filters::Common::RateLimit::LimitStatus status,
Filters::Common::RateLimit::DescriptorStatusListPtr&& descriptor_statuses,
Http::ResponseHeaderMapPtr&& response_headers_to_add,
Http::RequestHeaderMapPtr&& request_headers_to_add) override;
Http::RequestHeaderMapPtr&& request_headers_to_add,
const std::string& response_body) override;

private:
void initiateCall(const ThriftProxy::MessageMetadata& metadata);
Expand Down
2 changes: 1 addition & 1 deletion test/common/network/filter_manager_impl_test.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -416,7 +416,7 @@ stat_prefix: name
.WillOnce(Return(&conn_pool));

request_callbacks->complete(Extensions::Filters::Common::RateLimit::LimitStatus::OK, nullptr,
nullptr, nullptr);
nullptr, nullptr, "");

conn_pool.poolReady(upstream_connection);

Expand Down
16 changes: 9 additions & 7 deletions test/extensions/filters/common/ratelimit/ratelimit_impl_test.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,15 +38,17 @@ class MockRequestCallbacks : public RequestCallbacks {
public:
void complete(LimitStatus status, DescriptorStatusListPtr&& descriptor_statuses,
Http::ResponseHeaderMapPtr&& response_headers_to_add,
Http::RequestHeaderMapPtr&& request_headers_to_add) override {
Http::RequestHeaderMapPtr&& request_headers_to_add,
const std::string& response_body) override {
complete_(status, descriptor_statuses.get(), response_headers_to_add.get(),
request_headers_to_add.get());
request_headers_to_add.get(), response_body);
}

MOCK_METHOD(void, complete_,
(LimitStatus status, const DescriptorStatusList* descriptor_statuses,
const Http::ResponseHeaderMap* response_headers_to_add,
const Http::RequestHeaderMap* request_headers_to_add));
const Http::RequestHeaderMap* request_headers_to_add,
const std::string& response_body));
};

class RateLimitGrpcClientTest : public testing::Test {
Expand Down Expand Up @@ -91,7 +93,7 @@ TEST_F(RateLimitGrpcClientTest, Basic) {
response = std::make_unique<envoy::service::ratelimit::v3::RateLimitResponse>();
response->set_overall_code(envoy::service::ratelimit::v3::RateLimitResponse::OVER_LIMIT);
EXPECT_CALL(span_, setTag(Eq("ratelimit_status"), Eq("over_limit")));
EXPECT_CALL(request_callbacks_, complete_(LimitStatus::OverLimit, _, _, _));
EXPECT_CALL(request_callbacks_, complete_(LimitStatus::OverLimit, _, _, _, _));
client_.onSuccess(std::move(response), span_);
}

Expand All @@ -110,7 +112,7 @@ TEST_F(RateLimitGrpcClientTest, Basic) {
response = std::make_unique<envoy::service::ratelimit::v3::RateLimitResponse>();
response->set_overall_code(envoy::service::ratelimit::v3::RateLimitResponse::OK);
EXPECT_CALL(span_, setTag(Eq("ratelimit_status"), Eq("ok")));
EXPECT_CALL(request_callbacks_, complete_(LimitStatus::OK, _, _, _));
EXPECT_CALL(request_callbacks_, complete_(LimitStatus::OK, _, _, _, _));
client_.onSuccess(std::move(response), span_);
}

Expand All @@ -127,7 +129,7 @@ TEST_F(RateLimitGrpcClientTest, Basic) {
Tracing::NullSpan::instance(), stream_info_);

response = std::make_unique<envoy::service::ratelimit::v3::RateLimitResponse>();
EXPECT_CALL(request_callbacks_, complete_(LimitStatus::Error, _, _, _));
EXPECT_CALL(request_callbacks_, complete_(LimitStatus::Error, _, _, _, _));
client_.onFailure(Grpc::Status::Unknown, "", span_);
}

Expand All @@ -150,7 +152,7 @@ TEST_F(RateLimitGrpcClientTest, Basic) {
response = std::make_unique<envoy::service::ratelimit::v3::RateLimitResponse>();
response->set_overall_code(envoy::service::ratelimit::v3::RateLimitResponse::OK);
EXPECT_CALL(span_, setTag(Eq("ratelimit_status"), Eq("ok")));
EXPECT_CALL(request_callbacks_, complete_(LimitStatus::OK, _, _, _));
EXPECT_CALL(request_callbacks_, complete_(LimitStatus::OK, _, _, _, _));
client_.onSuccess(std::move(response), span_);
}
}
Expand Down