v2 API Proposal Document #339

aryan-25 · 2024-08-08T09:06:58Z

Hello 👋

I am Aryan, a Computer Science student and currently an intern at Apple on the Swift on Server team. As part of my internship, I have been working with @fabianfett to propose a new v2 API for the swift-aws-lambda-runtime library.

swift-aws-lambda-runtime is an important library for the Swift on Server ecosystem. The initial API was written before async/await was introduced to Swift. When async/await was introduced, shims were added to bridge between the underlying SwiftNIO EventLoop interfaces and async/await.

However, just like gRPC-swift and postgres-nio, we now want to shift to solely using async/await constructs instead of EventLoop interfaces. For this, large parts of the current API have to be reconsidered. This also provides a good opportunity to add support for new AWS Lambda features such as response streaming.

We have written a document that explains the current limitations of the library and proposes a detailed design for the v2 API.

Please read the proposal and voice your opinions either here or on the forum post. We are looking forward to your feedback!

cc: @sebsto @tachyonics @FranzBusch @Lukasa @tomerd @weissi @adam-fowler @ktoso @0xTim

sebsto · 2024-08-08T09:08:17Z

Thank you @aryan-25 ! I'll have a look this morning

sebsto · 2024-08-08T10:04:32Z

This is not related to this doc, but should we move the Documentation.docc directory just underneath Sources ?
@fabianfett wdyt ?

Current struct :

Sources/AWSLambdaRuntimeCore
└── Documentation.docc
    ├── Resources
    │   ├── code
    │   └── tutorials
    └── tutorials

Proposed struct

Sources/Documentation.docc
├── Resources
│   ├── code
│   └── tutorials
└── tutorials

0xTim

Great steps forward!

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

0xTim · 2024-08-08T11:49:05Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+
+`LambdaContext` will be largely unchanged, but the `eventLoop` property as well as the `allocator` property (of type `ByteBuffer`) will be removed.
+
+A new function `backgroundTask()` will also be added. This will allow tasks to be run in the background while and after the response is/has been sent. Please note that `LambdaContext` will not be Sendable anymore.


Suggested change

A new function `backgroundTask()` will also be added. This will allow tasks to be run in the background while and after the response is/has been sent. Please note that `LambdaContext` will not be Sendable anymore.

A new function `addBackgroundTask(_:)` will also be added. This will allow tasks to be run in the background while and after the response is/has been sent. Please note that `LambdaContext` will not be Sendable anymore.

What's the reasoning behind making the context non-sendable? Is it for the background task stuff? It should be fine given the scope of sharing it will be pretty narrow but it could cause issues with strict concurrency for those that aren't aware

In order to allow adding background tasks in a structured way, we will need to back the LambdaContext with a TaskGroup. Since TaskGroup is not Sendable we can't mark LambdaContext as Sendable. However if users need any property (all of them are Sendable) out of the LambdaContext they can just get that property out of it and then pass it around or close over it.

Why does the context need to hold the TaskGroup? I smell structured concurrency violations :)

0xTim · 2024-08-08T11:53:14Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+
+### StreamingLambdaHandler
+
+The new `StreamingLambdaHandler` protocol is the base protocol to implement a Lambda function. Most users will not use this protocol and instead use the `LambdaHandler` protocol defined in the `Codable` Support section.


Just for clarity it might be nice to have a section talking about the LambdaHandler - I spent a bit of time trying to find the detailed explanation of it before realising it's in the top section

0xTim · 2024-08-08T11:54:14Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+
+### LambdaRuntime
+
+`LambdaRuntime` is the class that communicates with the Lambda control plane as defined in [Building a custom runtime for AWS Lambda](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom.html) and forward the invocations to the provided `StreamingLambdaHandler`. It will conform to `ServiceLifecycle.Service` to provide support for `swift-service-lifecycle`.


Does this work with LambdaHandler as well, or does that conform to StreamingLambdaHandler to ferry calls through?

In order to use a LambdaHandler users will wrap it in a LambdaCodableAdapter. LambdaCodableAdapter conforms to StreamingLambdaHandler. So the LambdaRuntime type that you will use when using LambdaHandler will be:

let runtime: LambdaRuntime<LambdaCodableAdapter<MyLambdaHandler, Event, Output, JSONDecoder, JSONEncoder>>

but users basically never have to use that type explicitly. So to answer your question: Yes it works with LambdaHandler as well, but via the LambdaCodableAdapter.

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

0xTim · 2024-08-08T12:05:45Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+
+We decided to implement the approach in which a `LambdaResponseWriter` is passed to the function, since the approach in which a `LambdaResponse` is returned can trivially be built on top of it. This is not true vice versa.
+
+We welcome the discussion on this topic and are open to change our minds and API here.


It would be interesting to see if anyone has any use cases that integrate something like Swift Middleware

I think this is a fair point and I would love to see how middleware would work with this writer approach. IMO middleware do make some sense for Lambda

If using Middleware with a request and context, do you think the response writer would be placed in the context or be a separate parameter, thus meaning a departure from the current swift-middleware setup

Regarding whether you provide a response writer, versus returning a response that includes a writer I not sure. I have gone back and forward on this when building HB. In most cases I have not seen any need for a response writer but there are still situations where they makes things clearer. eg Tracing (You want to finished your request span once the response has been written, not when you have returned a response which includes the closure that'll write the response).

In the end you can implement returning a response on top of an API what uses a response writer but not the reverse so for flexibility it should probably be a response writer.

I made this point above, middleware definitely does make sense for lambda (for example tracing should just be a middleware that can be used anywhere) and should just be integrated IMO. My only quasi-concern with the passed-in-writer-approach is that for it to work with middleware, the middleware protocol has to suppress Copyable conformance on one of its associated types - its writer type, something that isn't going to be supported in Swift 6.0.

That said I do think the argument that this is a lower-level and more flexible API makes sense.

I don't think we want to integrate the swift-middleware library directly. I think if users have those needs they should consider using hummingbird/smoke/vapor in lambda mode, which then brings the option to use middleware.

This shall not mean that writing a middleware can be impossible. We should allow users to write middleware that they can stack.

Also I think swift-middleware is currently not targeted for anytime soon. So in order to make progress we should not depend on it. Integrating swift-middleware (as is) on top of this proposed API is absolutely possible and therefore should not be considered a blocker.

We can debate whether we think the swift-middleware library should be integrated directly but I do agree it shouldn't be considered a blocker for this work/release.

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

FranzBusch · 2024-08-08T13:14:43Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+    /// Write a response part into the stream and end the response stream
+    consuming func writeAndFinish(_ buffer: ByteBuffer) async throws
+    /// Report an error in the response stream
+    consuming func reportError(_ error: any Error) async throws


This seems like it is finishing as well? Do just want to spell this finish(error: (any Error)?) instead?

Not sure I agree here. I think the reportError is more explicit in what it does, than a finish with an error.

I tend to agree with @FranzBusch. Underneath this call it will write the error into some trailing headers and end the response. So it is always going to finish the stream. reportError sounds like something akin to emitting a log or metric.

I would propose spelling the API as finishWithMidstreamError(_: any Error) or some variation of to mirror the terminology Lambda itself uses and to be explicit about what the API will do.

This seems to dovetail with the comments about making handle support throwing of errors. Do we want users to report errors, via reportError or via throwing an error. Given this will end the stream I think I agree with Franz.

After thinking about this more, I think we should just make the handler method throwing. In this case we will do the error reporting, which means users don't have to deal with it, which in turn means we should remove this method here.

@FranzBusch, @tachyonics, @adam-fowler Can you get behind this?

The only issue with that is the user can't customise their response to the error, they might want to report the error differently if they haven't started streaming response data versus being in the middle of streaming a response.

They can still do this by doing a custom do/catch. It just makes the default case easier

FranzBusch · 2024-08-08T13:16:42Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+    /// the runtime can ask for further events. Note that AWS will continue to
+    /// charge you after the response has been returned, but work is still
+    /// processing in the background.
+    public func addBackgroundTask(_ body: sending @escaping () async -> ())


This doesn't feel necessary to me with the current API proposal. Since we pass in an independent writer. A user can just write some bytes, the call writer then finish and then do any background work before returning from the handle method. They can even set that up in child tasks in the handle method to start the background work while handling the actual request.

That's a great call. Yes if we keep the API that passes in a LambdaResponseWriter we can put the user in charge!

Agree. This is unstructured concurrency and given that the user controls finish, they know when the response is done.

@weissi this would not be unstructured, as the underlying runtime would inject a taskGroup into the context and that taskGroup would be used to schedule the child task here. The underlying runtime would then ensure that all tasks have completed, before it would ask for more work. This approach would be necessary, if we used an approach in which the user returns a LambdaResponse:

protocol LambdaStreamingResponse { func handle(_ event: ByteBuffer, context: LambdaContext) async throws -> LambdaResponse }

However this is needed for the LambdaHandler API:

public protocol CodableLambdaHandler: ~Copyable { associatedtype Event associatedtype Output // only way to schedule background work that can continue after returning the Output here // is by having the addBackgroundTask on `LambdaContext` here. func handle(_ event: Event, context: LambdaContext) async throws -> Output }

@FranzBusch given this, do we want to have two different LambdaContexts? Or are we fine with keeping the LambdaContext.addBackgroundTask even in the situation where it isn't really needed? I think I would opt for keeping it (so that we have it for the 99% use-case)

@fabianfett I'm pretty sure that it's is unstructured, it just manages to now use a Task { to achieve it. It passes the background work into a reference which will then be owned by something further up the stack.

The test is easy, if you write this code

do { // creating a useless piece of structure (so we can observe Structured Concurrency) thing.addBackgroundWork { print("AAAA") } } // the structure ends here print("BBBB")

Is there any way in which AAAA could print after BBBB? If so: unstructured.

The problem with splitting it across two methods is if the post work needs context from the initial work.

but we can pass the context, no?

@weissi what if you want to create a task that starts immediately but is allowed to run longer... I totally get the benefits. I think we need something that is better than Task {} here. And I think addBackgroundTask {} is significantly better here, as it will guarantee that you get runtime until your background task finishes. We could not guarantee this for Task {}. Also two methods on the interface won't work with the closure API.

@fabianfett Not sure I understand what "a task that starts immediately but is allowed to run longer" is. Could you provide some example code?

An API call that

does a short computation which is returned to you immediately

spawns a longer computation, which's result is send to you via mail once it finishes (can take up to 15min)

Both should start as soon as the API is invoked. We can potentially question the API design here. But this is possible with Lambda and we should allow users to do this, if they opt to.

Probably the most consistent approach for @fabianfett's use case is to provide a wrapping CodableResponseWriter (potentially with just a writeAndFinish API) while allowing work to continue past the result being returned to the control plane. It is not as simple as returning the result directly. We could provide both forms but that does add its own complexity to the library overall.

FranzBusch · 2024-08-08T13:16:59Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+    /// `Logger` to log with.
+    ///
+    /// - note: The `LogLevel` can be configured using the `LOG_LEVEL` environment variable.
+    public var logger: Logger { get }


Wondering how we feel about this in relation to apple/swift-log#315. Also why do we call out the env variable here? Shouldn't we just pass in the logger to the runtime and that logger determines the log level?

I think your benchmark shows clearly that using TaskLocals is more expensive than passing around a logger time wise.
Given that Lambdas are payed by the milliseconds I certainly don't want to make using TaskLocals the only approach.
Also I want this API to land sooner rather than later and not block on the above discussion that can be seen as an addition.

+1 Around logging and TaskLocals, it shouldn't be the only approach.

Explicitly passing a correctly-configured logger into the Runtime should be the recommended approach IMO rather than being able to partially configure logging through environment variables.

Yep, there'll be a lot of discussion around using a TaskLocal logger and it shouldn't hold this work up.

FranzBusch · 2024-08-08T13:17:13Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+    public var invokedFunctionARN: String { get }
+
+    /// The timestamp that the function times out.
+    public var deadline: DispatchWallTime { get }


Can we use ContinuousClock.Instant here?

Nope this is epoch time.

As per Lambda runtime API

Lambda-Runtime-Deadline-Ms – The date that the function times out in Unix time milliseconds. For example, 1542409706888.

Got you so we would need a UTCClock.Instant here

Yes. Sadly we don't have that yet. Only way out would be if Lambda built its own Clock. But I don't think we want to do that.

Maybe you should create a LambdaUTCTimeClock type that conforms to all the new hotness but implements the clock. With the hope that one day you can deprecate it and to typealias LambdaUTCTimeClock = UTClock with a bit of luck.

@weissi would you build the sleeping on top of dispatch?

@weissi would you build the sleeping on top of dispatch?

No.

I'd just create a custom Clock type (because the language lacks UTCClock for some reason). Once you have your own clock (which you could for example back by clock_gettime and struct timespec but hidden as implementation detail).

How would you implement the required sleeping methods though?
https://developer.apple.com/documentation/swift/clock/sleep(for:tolerance:)

You can just use the normal Task.sleep on the non-suspending clock for that. UTC time and the non-suspending clock run at the same speed

FranzBusch · 2024-08-08T13:22:55Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+enum Lambda {
+    /// This returns the default EventLoop that a LambdaRuntime is scheduled on.
+    /// It uses `NIOSingletons.posixEventLoopGroup.next()` under the hood.
+    public static var defaultEventLoop: any EventLoop { get }


Like noted I don't see why we would need to create a separate static var for this?

answered above.

FranzBusch · 2024-08-08T13:23:19Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+We also add an enum to store a static function and a property on. We put this on the static `Lambda` because `LambdaRuntime` is generic and thus has bad ergonomics for static properties and functions.
+
+```swift
+enum Lambda {


Is this namespace really worth it if we remove the EL default and can we then just make the reportStartupError global method?

FranzBusch · 2024-08-08T13:24:02Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+    public static var defaultEventLoop: any EventLoop { get }
+
+    /// Report a startup error to the Lambda Control Plane API
+    public static func reportStartupError(any Error) async


What if we make this a with method so that users can do any setup and we report any error that happens there. Instead of them having to do a do/catch?

I think we will opt for catching the error directly.

FranzBusch · 2024-08-08T13:27:21Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+#### LambdaHandler
+
+```swift
+public protocol LambdaHandler {


Just out of curiosity we can't solve this with protocol inheritance right? We do need the various different protocols and adapters?

We do have protocol inheritance today and I don't like it. The problem here is that there is never a case where the protocol inheritance here makes sense from a user point of view. You either want to use pure streaming or you want to use Codable. But there is no point in which you opt to override the streaming API when you implement LambdaHandler. Worst of all injecting custom Encoders and Decoders only works through globals.

FranzBusch · 2024-08-08T13:28:22Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+
+We decided to implement the approach in which a `LambdaResponseWriter` is passed to the function, since the approach in which a `LambdaResponse` is returned can trivially be built on top of it. This is not true vice versa.
+
+We welcome the discussion on this topic and are open to change our minds and API here.


I think this is a fair point and I would love to see how middleware would work with this writer approach. IMO middleware do make some sense for Lambda

tachyonics · 2024-08-08T21:39:58Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+for _each_ different handler protocol and for both `String` and `JSON` formats. This has resulted in a lot of
+boilerplate code which can very easily be made generic and simplified in v2.
+
+### New features


We touched on this briefly in a previous discussion but do we want to directly integrate middleware into the library - or at the very least ensure that there is a clear path to allow users to do so themselves?

Let's do the discussion here

tachyonics · 2024-08-08T22:08:24Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+    /// Write a response part into the stream and end the response stream
+    consuming func writeAndFinish(_ buffer: ByteBuffer) async throws
+    /// Report an error in the response stream
+    consuming func reportError(_ error: any Error) async throws


I tend to agree with @FranzBusch. Underneath this call it will write the error into some trailing headers and end the response. So it is always going to finish the stream. reportError sounds like something akin to emitting a log or metric.

I would propose spelling the API as finishWithMidstreamError(_: any Error) or some variation of to mirror the terminology Lambda itself uses and to be explicit about what the API will do.

tachyonics · 2024-08-08T22:10:43Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+### LambdaContext
+
+`LambdaContext` will be largely unchanged, but the `eventLoop` property as well as the `allocator` property (of type
+`ByteBufferAllocator`) will be removed.


As we discussed earlier, it would be useful to add context here as to why allocator is being removed - it that it is related to an optimisation that was never provided.

We generally want to reduce number of NIO types that we expose in APIs. NIO shall eventually become an implementation detail.

If you look at ByteBufferAllocator you can see that it does not optimize creating allocations today. Instead the pattern that is used over and over is reusing existing ByteBuffers as much as possible. The new CodableAdapter helps significantly in that regard. Is there a use-case for which you need a ByteBufferAllocator?

cc @weissi please correct me if this is wrong.

I think it's fine to remove this. The original idea was that you could use more efficient allocators for ByteBuffers but we never actually used that. Swift itself is still not able to use alternative allocators (except if you globally set malloc/free to a different one) so I don't think this was as useful as we thought it would be (outside of ByteBuffer tests).

tachyonics · 2024-08-08T22:19:01Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+    /// `Logger` to log with.
+    ///
+    /// - note: The `LogLevel` can be configured using the `LOG_LEVEL` environment variable.
+    public var logger: Logger { get }


+1 Around logging and TaskLocals, it shouldn't be the only approach.

Explicitly passing a correctly-configured logger into the Runtime should be the recommended approach IMO rather than being able to partially configure logging through environment variables.

tachyonics · 2024-08-08T22:37:26Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+    ///                     If no response or error is written to the `responseWriter` it will
+    ///                     report an error to the invoker.
+    ///   - context: The LambdaContext containing the invocation's metadata
+    mutating func handle(_ event: ByteBuffer, responseWriter: consuming some LambdaResponseWriter, context: LambdaContext) async


So a question around a throwing function and Non-copyable types.

In the happy case, a user will have to consume responseWriter within the body of this function - so some kind of explicit finish* call. I am ok with this although a little more verbose, it is explicit in what is happening.

In the throwing case, I am assuming the writer will still need to be consumed prior to throwing. Is this correct and if so, what would the runtime do with a thrown error but a now-consumed writer?

tachyonics · 2024-08-08T22:50:40Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+    public init(
+        handler: consuming sending Handler,
+        eventLoop: EventLoop = Lambda.defaultEventLoop,
+        logger: Logger = Logger(label: "Lambda")


+1 to @0xTim's point. There should be some basic logging by default for discoverability and I would think it is the expectation of people coming from the rest of the ecosystem. @FranzBusch why don't you want to default the logger?

That said, the v1 implementation was way too noisy IMO, particularly at higher log levels. We should make sure the runtime is a good citizen and conforms to https://www.swift.org/documentation/server/guides/libraries/log-levels.html (ie. mostly logging at trace but enough logs at debug to show that the library is working to first-time users)

tachyonics · 2024-08-08T22:57:49Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+        handler: Handler,
+        encoder: Encoder,
+        decoder: Decoder


Would it make more sense for handler to be the final argument so it could be a trailing closure if the user wants?

Is there an end-to-end example somewhere of how this looks from main() for the case where custom coders need to be used?

tachyonics · 2024-08-08T22:58:27Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+        decoder: Decoder
+    ) where Output: Encodable
+
+    /// For handler with a void output -- the user doesn't specify an encoder.


Why wouldn't we have the inverse for a void input?

Interesting point, we didn't support this before. Are there Lambdas without input?

I think the answer is "no" (or at least an empty JSON {})
Source: the runtime API and the invoke API.

I asked the team to confirm

I confirm a payload is required. Minimum payload is {}

tachyonics · 2024-08-08T23:17:36Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+
+We decided to implement the approach in which a `LambdaResponseWriter` is passed to the function, since the approach in which a `LambdaResponse` is returned can trivially be built on top of it. This is not true vice versa.
+
+We welcome the discussion on this topic and are open to change our minds and API here.


I made this point above, middleware definitely does make sense for lambda (for example tracing should just be a middleware that can be used anywhere) and should just be integrated IMO. My only quasi-concern with the passed-in-writer-approach is that for it to work with middleware, the middleware protocol has to suppress Copyable conformance on one of its associated types - its writer type, something that isn't going to be supported in Swift 6.0.

That said I do think the argument that this is a lower-level and more flexible API makes sense.

tachyonics · 2024-08-08T23:20:09Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+let runtime = LambdaRuntime { (event: Input, context: LambdaContext) in
+    Greeting(echoedMessage: event.message)
+}
+
+try await runtime.run()


For this, the simplest use case, would it make sense to provide a static main() on LambdaRuntime that just calls its run method so users can use @main if this is all they need to do?

Not sure I can follow here. How would that code look like?

Wouldn't it look like-

@main LambdaRuntime { (event: Input, context: LambdaContext) in Greeting(echoedMessage: event.message) }

weissi · 2024-08-09T09:03:29Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+
+The current API extensively uses the `EventLoop` family of interfaces from SwiftNIO in many areas. To use these
+interfaces correctly though, it requires developers to exercise great care and be aware of certain details such as never
+running blocking code on the same `EventLoop` the library uses. Developers also need to understand the various transform


same applies to async/await. I'd scratch this

weissi · 2024-08-09T09:07:37Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+
+```swift
+/// A writer object to write the Lambda response stream into
+public protocol LambdaResponseWriter: ~Copyable {


I think making this ~Copyable will lead to a load of issues. Can we have a motivation on why we allow non-copyable types to implement this? If we allow non-copyable types here, then anything accepting LambdaResponseWriters needs to support this which sounds like a tall ask.

LambdaResponseWriter is not a protocol that users would need to implement. It is marked NonCopyable as we want to enforce through the compiler that users can't write() after they have called finish().

If we allow non-copyable types here, then anything accepting LambdaResponseWriters needs to support this which sounds like a tall ask.

Would you mind elaborating on this a bit?

Sure, but don't you provide a ~Copyable LambdaResponseWriter to the user?

weissi · 2024-08-09T09:15:15Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+    /// the runtime can ask for further events. Note that AWS will continue to
+    /// charge you after the response has been returned, but work is still
+    /// processing in the background.
+    public func addBackgroundTask(_ body: sending @escaping () async -> ())


Agree. This is unstructured concurrency and given that the user controls finish, they know when the response is done.

weissi · 2024-08-09T09:15:24Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+
+```swift
+/// The base StreamingLambdaHandler protocol
+public protocol StreamingLambdaHandler: ~Copyable {


I think making this ~Copyable will lead to a load of issues. Can we have a motivation on why we allow non-copyable types to implement this? If we allow non-copyable types here, then anything accepting StreamingLambdaHandler s needs to support this which sounds like a tall ask.

Can we have a motivation on why we allow non-copyable types to implement this?

It is not necessary to have a non copyable Handler. We just allow it. Tbh. the motivation here is mostly, because we can.

If we allow non-copyable types here, then anything accepting StreamingLambdaHandlers needs to support this which sounds like a tall ask.

I think the only thing that should accept this protocol is LambdaRuntime and we have support for non copyable already implemented in our prototype. What else would consume a StreamingLambdaHandler?

What else would consume a StreamingLambdaHandler?

The user's tests?

we have support for non copyable already implemented in our prototype.

Sure, but it could cause problems further down the line, no? Like if you figure out that you will need to copy it

The user's tests?

I think we would provide a TestRuntime that does that for the user.

Sure, but it could cause problems further down the line, no? Like if you figure out that you will need to copy it

Makes sense. But in this case requirements would have changed so much that we should investigate a v3.

Really not sure about that. I don't think it gives you any benefit but it restricts your code's evolution. Don't see the point :).

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

weissi · 2024-08-09T09:22:16Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+
+```swift
+/// The base LambdaHandler protocol
+public protocol StreamingLambdaHandler: ~Copyable {


same question around allowing ~Copyable types here.

Let's discuss here: #339 (comment) once we have a solution we will fix or resolve here.

weissi · 2024-08-09T09:24:00Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+    /// - Parameter logger: A logger
+    public init(
+        handler: consuming sending Handler,
+        eventLoop: EventLoop = Lambda.defaultEventLoop,


Will we still have the ability to run everything in just one thread (the main thread)? I think for many lambdas you'll never want to fork off a second thread at all. NIO thread & async/await should just run in the main thread (but as global default executor not as @MainActor)

czechboy0 · 2024-08-09T10:49:41Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+        _ event: ByteBuffer,
+        responseWriter: consuming some LambdaResponseWriter,
+        context: LambdaContext
+    ) async {


Out of curiosity: at which point is the HTTP response started/the head sent? Before the user handler is invoked? Lazily right before the first responseWriter.write(...) call?

This would be lazily before the first write(). If the user calls writeAndFinish() just once, we would send the response back in non streaming mode (if it fits into the 6MB limit).

What about

responseWriter.withResponseStream { stream in try await stream.write(...) try await stream.write(...) try await stream.write(...) try await stream.write(...) try await stream.write(...) }

where the head is sent on { and the finish is sent on }? That way things can't go wrong. If you throw that'd mean reportError.

That'd kinda answer both of @czechboy0's questions pretty clearly. No implicit, lazy head anymore and no way to forget finish.

@weissi would you mind elaborating what the issue is with the ~Copyable approach. I feel like we reinvent the closure hell now in structured concurrency land.

Also did you deliberately skip the try await before responseWriter.withResponseStream or is that an oversight?

~Copyable doesn't help you if you don't call finish. You can't mark deinit as @available(*, unavailable) deinit {}. It does prevent double-calling but it also comes with massive strings attached, namely that the user can't copy it if the type's actually ~Copyable.

The head has to be sent on the first write because the control plane has different endpoints for a complete failure vs a successful response or a midstream error. So we need to wait until at least a partial response to determine what endpoint to send the response to.

czechboy0 · 2024-08-09T10:49:57Z

Sources/AWSLambdaRuntimeCore/Documentation.docc/Proposals/0001-v2-api.md

+            responseWriter.write(ByteBuffer(string: #"\#(i)\n\r"#))
+            try? await Task.sleep(for: .milliseconds(500))
+        }
+        responseWriter.finish()


What happens if the user doesn't call finish explicitly?

We would call it for you. We should decide if we send it in deinit or once the call completes.

…e blocks

v2 API Proposal Document

c19ae23

0xTim reviewed Aug 8, 2024

View reviewed changes

aryan-25 added 2 commits August 8, 2024 13:47

Fix headings and run formatter

250f94b

Add line breaks

ea76068

fabianfett mentioned this pull request Aug 8, 2024

[Draft] Detached tasks #334

Open

FranzBusch reviewed Aug 8, 2024

View reviewed changes

aryan-25 added 2 commits August 8, 2024 14:35

Fix inaccuracies

77ad9a5

Add explanation for LambdaContext not being Sendable

4eea6ec

tachyonics reviewed Aug 8, 2024

View reviewed changes

weissi reviewed Aug 9, 2024

View reviewed changes

czechboy0 reviewed Aug 9, 2024

View reviewed changes

Remove mention of LambdaCodableAdapter being a class and update cod…

be97266

…e blocks


		`LambdaContext` will be largely unchanged, but the `eventLoop` property as well as the `allocator` property (of type `ByteBuffer`) will be removed.

		A new function `backgroundTask()` will also be added. This will allow tasks to be run in the background while and after the response is/has been sent. Please note that `LambdaContext` will not be Sendable anymore.

	A new function `backgroundTask()` will also be added. This will allow tasks to be run in the background while and after the response is/has been sent. Please note that `LambdaContext` will not be Sendable anymore.
	A new function `addBackgroundTask(_:)` will also be added. This will allow tasks to be run in the background while and after the response is/has been sent. Please note that `LambdaContext` will not be Sendable anymore.


		### StreamingLambdaHandler

		The new `StreamingLambdaHandler` protocol is the base protocol to implement a Lambda function. Most users will not use this protocol and instead use the `LambdaHandler` protocol defined in the `Codable` Support section.


		### LambdaRuntime

		`LambdaRuntime` is the class that communicates with the Lambda control plane as defined in [Building a custom runtime for AWS Lambda](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom.html) and forward the invocations to the provided `StreamingLambdaHandler`. It will conform to `ServiceLifecycle.Service` to provide support for `swift-service-lifecycle`.


		We decided to implement the approach in which a `LambdaResponseWriter` is passed to the function, since the approach in which a `LambdaResponse` is returned can trivially be built on top of it. This is not true vice versa.

		We welcome the discussion on this topic and are open to change our minds and API here.

v2 API Proposal Document #339

Are you sure you want to change the base?

v2 API Proposal Document #339

Conversation

aryan-25 commented Aug 8, 2024 • edited Loading

sebsto commented Aug 8, 2024

sebsto commented Aug 8, 2024

0xTim left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adam-fowler Aug 8, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

weissi Aug 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sebsto Aug 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aryan-25 commented Aug 8, 2024 •

edited

Loading

adam-fowler Aug 8, 2024 •

edited

Loading

weissi Aug 9, 2024 •

edited

Loading

sebsto Aug 9, 2024 •

edited

Loading