Lambda-http: setting multiple cookies using multiValueHeaders is not supported in API Gateway version 2.0

[l3ku]

From https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-develop-integrations-lambda.html:

Format 2.0 doesn't have multiValueHeaders or multiValueQueryStringParameters fields. Duplicate headers are combined with commas and included in the headers field. Duplicate query strings are combined with commas and included in the queryStringParameters field.

Format 2.0 includes a new cookies field. All cookie headers in the request are combined with commas and added to the cookies field. In the response to the client, each cookie becomes a set-cookie header.

I have a use case, where I need to pass several set-cookie headers in the response from Lambda to API Gateway 2.0 using the lambda-http crate. However, when I set multiple SET_COOKIE headers on the response, only one of them is returned from API Gateway, presumably since API Gateway only handles the one header which is present in the headers field of LambdaResponse.

Is there a way to work around this other than adding the cookies field as part of the LambdaResponse in some manner, like using a similar enum as LambdaRequest for different API Gateway versions?

[rimutaka]

@miere, I'm replying to your question here to keep the PR discussion clean. We really need someone to enable the new Discussions feature.

An honest question: why is this runtime maintaining its own implementation of the AWS Events where one could get them from the aws_lambda_event crate?

If I recall this correctly, there was a conversation of AWS taking over the maintenance of their events crate and then it would be added as a dependency. I cannot find that thread, but I think @softprops was looking into it.

[softprops]

I’m not actively working on an open issue atm but just a reminder about the lambda/http crate.

The idea was to transparently use the rust community http crate as an interface for various flavors of aws http like triggers. The underlying ser/deser impl for for those has many optimizations for that deser fields directly to the http crate types when possible for example. The idea was this would be valuable for those coming from rust and new to lambda having familiar ecosystem types to work with.

a generic events crate would likely have a more unopinionated set of fields that directly map to aws trigger event types. I believe you could use the events crate and default lambda runtime for this for this today.

The trade off is your lose the flexibility to transparently switch between http triggers without changing you application and lose the leverage of the ecosystem built up around the rust http crate and it’s types.

I still think an option for and events crate is still worth maintaining because there are many non http triggers that have structures clients likely won’t want to repeat custom deserializers for. A lack of a formal schema for these was the previous blocker. Previous attempts attempted to infer types by parsing other language bindings. This is fragile as other language types may not have a direct mapping to rust types. Inferred types would not have the same level of correctness as a formal definition that a schema would provide.

[calavera]

I merged @l3ku into our fork, see #274, and I definitely agree that structure compatibility is a pita. We had to change some definitions later on because the lambda emulator that aws open sourced sends data that cannot be handed correctly, even with @l3ku's change.

See https://github.com/netlify/aws-lambda-rust-runtime/pull/4 and https://github.com/netlify/aws-lambda-rust-runtime/pull/6

[miere]

@rimutaka thanks, mate. Discussions might be helpful to ease and centralise discussions, leveraging a more open source community.

[miere]

@softprops I see what you meant, mate. Now that you've explained your reasoning, I can see where you've come from and what you want to achieve with it. Let me, though, give you another perspective on this. I reckon the Runtime and the HTTP crates have different lifecycles, therefore they might have conflicting improvement interests since day one.

From what I can see, the Runtime crate is bound to the existing AWS' internal API contract and the changes made on its communication protocol. It's something we have absolutely no control - meaning that we can't trigger modifications on it aside from design/usability tweaks. On the other hand, it's expected to not be changed quite frequently, being a stable crate updated to reflect changes in AWS Lambda itself. In practice, it would be enough if we have a simple crate the provides a convenient abstraction to consume/listen to lambda events. In fact, aside from a few quirky choices (e.g. the Error type alias that we have to reimplement frequently), the crate is pretty stable and could have been published to crates.

Assuming the above true, it's easier to see the HTTP crate as something bound to users convenience. Therefore, it might need much more iterations and a shorter feedback-loop to become something really valuable to its users. It means that it might take longer to reach its 1.0 state. Unquestionably, we might need bigger discussions and diversity of usage, otherwise, we might expect a version 2.0 quite soon. Stability and robustness are key factors for wide HTTP crate adoption.

Then it comes one of the most critical points: delivered value. A transparent layer between ALB, AGWv1 and AGWv2 might sound like a requirement in the first place but I have my doubts if this is really true. No matter what reason the user has to switch between one another, it is not a task likely to happen weekly or daily. It might be something that would happen once in the lifetime of a given project. In fact, I'd consider it a whole project itself, as one will have a multitude of challenges to tackle aside of the contract signature - different deployment structures, changes on the reactive pipeline, different ways to perform authorization, and so on and so forth.

To sum up, I can see Lambda Http evolving into something bigger as a separated project. I'd use it if it abstracts away (not partially but) everything related to handling HTTP requests in the AWS ecosystem. But what I'm really keen, is to see a stable AWS Lambda Runtime as soon as possible - allowing the community to produce extra crates based on it. If the Runtime crate becomes something really small, I can't see AWS coming with any excuses to keep it updated - perhaps turning the Rust runtime into a first-class citizen.