[Migrated] HTTP API support - API Gateway V2

[jneves]

Originally from: Miserlou/Zappa#2099 by ryancausey

It looks like API Gateway has a new class of APIs called HTTP APIs which more closely fit with what Zappa aims to do. It might be good to start working on implementing support for this in Zappa.

https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api.html

The way Zappa deploys API gateways would have to be updated to also support this kind of API.

It also looks like there's a new 2.0 version of the event format sent to the lambda and the response format from lambda to the API gateway. This will have to be pinned to use the 1.0 formats until support for the 2.0 formats are added in the lambda handler provided by Zappa.

https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-develop-integrations-lambda.html

[bahoo]

I'm going to take up a bit of space in this issue to contribute some ideas toward getting this resolved.

I've got most of the code written, and need to start thinking about how rubber meets road. It would seem to me that:

1. Virtually all extant Zappa projects are API Gateway v1
2. Zappa aside, API Gateway v2 can support v1-era Rest APIs that are perfectly compatible ( someone please check me on this )
3. We'll need to write the code such that Zappa can work seamlessly between the two, to accommodate old and new projects, with perhaps a built-in bias toward the eventuality of v2.

As some starting place, I'm gonna suggest:

1. New Zappa projects ( created through init ) should have a new property set in their zappa_settings.json file, to indicate a preference toward API Gateway v2. Let's call this api_gateway for the time being, set to 1 or 2.
2. Zappa should treat this value as canon; if it's present, rely on it. If it's absent, default to v1. That way, newly created projects will have that property set ( in init ) and move effortlessly into v2. Extant projects can hum about happily using v1.
3. Over time this will likely evolve since it seems obvious v2 is the future.

I should have something to share likely in the coming week. And I'll broadly say, having used Zappa fairly reliably for several years now, the regional HTTP APIs do absolutely seem worth the plunge 👍 responsiveness especially, but also in terms of cost. If Zappa ( especially with slim_handler set to True ) has ever seemed a bit sluggish, or if the AWS Console interface has ever seemed clunky, especially where routes and integrations are concerned, it would seem that help is on the way.

Feedback welcome on all the fronts — this is a significant change, and vigorous community input will make it a smooth transition. Thanks folks. :)

[DilLip-Chowdary-Codes]

Hi, @jneves @bahoo seems like HTTP API support has not been added till now.

Let me know if you've implemented this feature in your forks or also let me know if there are any challenges with implementing it.

[monkut]

So the incoming payload format and response format would are updated for 2.0?

From the user perspective, is there an advantage to migrating to 2.0?
Would we lose any functionality if we migrated fully to 2.0?

[bahoo]

@DilLip-Chowdary-Codes thanks for the nudge. Haven't run into too many hiccups, I'll get this cleaned up and pushed suitable for a public fork :D

@monkut To be honest I haven't seen any great upside of 2.0 and I don't know if there's any real threat of e.g. 1.0 being deprecated, but I generally lean toward migrating toward the newer version to help avoid being in the hot seat later if it were to get deprecated or decommissioned.

[monkut]

@bahoo Thanks for the clarification. Yes, it's good to be ready in case they depreciate it, but in terms of priority if there aren't any clear advantages to 2.0, there's no rush to get it supported.

If you have something that support it though, feel free to create a PR and we can move toward getting it in!

[sridhar562345]

@bahoo when can we expect this PR? Waiting for this feature :)

Until then can we have a look at your fork?

[bahoo]

Been sitting on this too long, sorry y'all.

This branch is frankly a mess but to the best of my recollection it:

• Upgrades to API Gateway v2, which
• Opens up the door for using HTTP APIs; specify a new apigateway_protocol key in your stage's settings file, to be HTTP, and it should take it from there: bahoo/Zappa@a40979f#diff-550c3276a99c37a050752e67ae30179b174a970f666ccb95824a81d4160998b6R2613 )

Not documented, janky code and print statements abound, etc. etc. but ! Some starting place: https://github.com/bahoo/Zappa

Adding some other, unsolicited, much bigger scope and probably way off topic, color commentary on how this fork / Zappa might evolve, as Lambda has continued to evolve:

• At some point, Lambda added direct function URLs, which could simplify the stack quite a bit: https://docs.aws.amazon.com/lambda/latest/dg/lambda-urls.html
• Lambda has also been leaning more heavily into Layers, and I think they have some bundled way to merge e.g. many layers together into a runtime environment. Zappa's magic worked its magic by just zipping the environment packages together into one directly, but one downside of that is that each deploy meant pushing all those packages up with it too ( e.g. ~50-100MB upload even if your project is only one single Python file with a bunch of dependencies ). The new layer world mitigates this a bit; you could indeed just upload that one single Python file and everything else is already there, merged and ready to go. 💁🏻‍♂️

Zappa's whole premise was solving those two problems more easily for developers ( you have a function, let's deploy all the other API Gateway tapestry to get you a URL; and let's package / bundle all the Python necessary to get that function running ). I'm extremely bullish on Lambda still. I've leaned a bit away from Zappa specifically for smaller projects, and wondered if either a fork or a successor might be in order to play a bit more nicely with where AWS is taking Lambda. Just haven't had the spoons to look at it very deeply, yet. If there are efforts or discussion underway, here or anywhere 💁🏻‍♂️ I'm interested in contributing to those long term.

[sridhar562345]

@bahoo seems like your repo is private, can you check it once?

[bahoo]

Just made public :D ty.

[bahoo]

And sorry I'm going to meander off topic — sticking exclusively to HTTP APIs and eschewing REST APIs and some of the other tapestry Zappa offers, has indeed proved super simple 💁🏻‍♂️ I am probably going to forge ahead with a minimalist fork / successor project of some sort ! My question to anyone reading this, has that been done already ?

Zappa's an amazing, pioneering project, I've fanboyed for years and will definitely continue to keep it in the rotation for tools ( and frankly I would have years of projects to migrate off it if I ever do take such a plunge, lol ). But I do think there's a lane for ( perhaps even, zappa_settings.json compatible ) spiritual successor, a pretty drastically simplified codebase, but with the same premise of going from WSGI to serverless in a couple commands.

Not seeing anything on a casual Google, if anyone knows or sees anything, @ / link me ? <3

[paul31917]

@bahoo Sorry for bother you. I am a newbie for zappa and aws api gateway. I used to use rest api gateway , and zappa package to zip a lambda code , and then upload the zip to s3 through the aws console. Now I want to switch the api gateway to http . I have already built a http api gateway using the aws web console, but I always got 404 not found when calling the api through http gateway endpoint.
How can I adjust my zappa settings ? Or could you give me any advise please?

[github-actions]

Hi there! Unfortunately, this Issue has not seen any activity for at least 90 days. If the Issue is still relevant to the latest version of Zappa, please comment within the next 10 days if you wish to keep it open. Otherwise, it will be automatically closed.

[github-actions]

Hi there! Unfortunately, this Issue was automatically closed as it had not seen any activity in at least 100 days. If the Issue is still relevant to the latest version of Zappa, please open a new Issue.