Trailers in chunked frames

[Wonshtrum]

Transfer encoding chunked HTTP/1.1 frames enable the streaming of dynamically generated bodies, with potentially unknown sizes at the start of the transfer. In the same way, it enables sending headers whose values are dynamically generated and unknown at the beginning of the transfer. Those headers are called "trailers", their name must be declared in advance in the Trailer header and will be sent after the last chunk of the body. Sozu doesn't seem to support this feature. Upon receiving a chunked frame it will transfer it successfully up to the last chunk and reset the connection.

sozu/lib/src/protocol/http/mod.rs

Lines 1434 to 1455 in 6f32ca5

	// the entire request was transmitted
	Some(RequestState::Request(_, _, _))
	| Some(RequestState::RequestWithBody(_, _, _, _))
	| Some(RequestState::RequestWithBodyChunks(_, _, _, Chunk::Ended)) => {
	// return the buffer to the pool
	// if there's still data in there, keep it for pipelining
	if self.must_continue_request()
	&& self.frontend_buffer.as_ref().map(|buf| buf.empty()) == Some(true)
	{
	self.frontend_buffer = None;
	}
	self.frontend_readiness.interest.remove(Ready::readable());
	self.backend_readiness.interest.insert(Ready::readable());
	self.backend_readiness.interest.remove(Ready::writable());
	// cancel the front timeout while we are waiting for the server to answer
	self.container_frontend_timeout.cancel();
	if let Some(token) = self.backend_token.as_ref() {
	self.container_backend_timeout.set(*token);
	}
	SessionResult::Continue
	}

sozu/lib/src/protocol/http/mod.rs

Lines 1243 to 1298 in 6f32ca5

	Some(ResponseState::Response(_, _))
	| Some(ResponseState::ResponseWithBody(_, _, _))
	| Some(ResponseState::ResponseWithBodyChunks(_, _, Chunk::Ended)) => {
	let frontend_keep_alive = self
	.request_state
	.as_ref()
	.map(|r| r.should_keep_alive())
	.unwrap_or(false);
	let backend_keep_alive = self
	.response_state
	.as_ref()
	.map(|r| r.should_keep_alive())
	.unwrap_or(false);
	save_http_status_metric(self.get_response_status());
	self.log_request_success(metrics);
	metrics.reset();
	if self.closing {
	debug!("{} closing proxy, no keep alive", self.log_context());
	self.frontend_readiness.reset();
	self.backend_readiness.reset();
	return StateResult::CloseSession;
	}
	//FIXME: we could get smarter about this
	// with no keepalive on backend, we could open a new backend ConnectionError
	// with no keepalive on front but keepalive on backend, we could have
	// a pool of connections
	match (frontend_keep_alive, backend_keep_alive) {
	(true, true) => {
	debug!("{} keep alive front/back", self.log_context());
	self.reset();
	self.frontend_readiness.interest =
	Ready::readable() | Ready::hup() | Ready::error();
	self.backend_readiness.interest = Ready::hup() | Ready::error();
	StateResult::Continue
	}
	(true, false) => {
	debug!("{} keep alive front", self.log_context());
	self.reset();
	self.frontend_readiness.interest =
	Ready::readable() | Ready::hup() | Ready::error();
	self.backend_readiness.interest = Ready::hup() | Ready::error();
	StateResult::CloseBackend
	}
	_ => {
	debug!("{} no keep alive", self.log_context());
	self.frontend_readiness.reset();
	self.backend_readiness.reset();
	StateResult::CloseSession
	}
	}
	}

As #897, this issue is mainly to document and acknowledge a bug whose severity is minor as trailers are almost never used in HTTP/1.1 due to there convoluted nature (which explains why it wasn't reported long ago). We may not fix this in the current version. But again, the entire stack is undergoing a full rewriting with HTX so the question of support rises again. Moreover, HTTP/2.0 is more convenient as it allows one to send trailers without declaring them in advance. Consequently, this feature may be more used but can't be perfectly translated to HTTP/1.1.