Add URLEncodedForm decoding/encoding #16
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Closed
adam-fowler
added a commit
that referenced
this pull request
Aug 10, 2023
* Add support for NIOTransportServices on iOS, macOS * Add Network framework TLS options to configuration * Fix TLS options on macOS * Add NIOTS test * swift format * Split HBHTTPServer.Configuration into separate file * initialize Configuration with TSTLSOptions * Both HBHTTPServer.Configuration.init() should be available on mac
adam-fowler
added a commit
that referenced
this pull request
Aug 14, 2023
* Initial commit * Add .swiftformat * Update README.md * Log errors * Add documentation building scripts * Update README.md * Comments, remove public from a few symbols * Don't make string optional in HBHTTPError(_:message) * Added HBHTTPResponseError protocol Gives errors a chance to decide how a response is generated from them * Update README.md point to new location for docs * Moving code about * HBRequestBodyStreamer.consume returns one ByteBuffer as a time * Send .end on stream error * HBRequestBody now has a queue of EventLoopPromise Couldn't get the single promise to work in all situations * Added backpressure on streamed requests. Don't read more while buffer to be processed is over a certain amount * Added tests for streaming request bodies * GH actions * swift format * weird edit from SF * Added HBRequestBodyStreamer.consumeAll(on:_) * Close channel when we receive a quiesce event * Remove connection close headers, as server shuts down correctly now * swift format * ChannelInitializer takes an array of ChannelHandlers (#7) * Make all channel handlers removeable (#8) * Require all channel handlers to be removable * Update HTTPServer.swift * Add testing of other repositories with current state of hummingbird-core * Add support for dropping streamed request bodies This fixes an issue where a request body is never processed but back pressure stalls the channel because the request body is too large. Once we have finished processing the request we "drop" the body so any more request body streaming is ignored * Fix HTTP2 * Body streaming (#9) * Move HBRequestBodyStreamer into its own file * Allow for nil streamer Check for nil byteBuffer. If you receive this no body was included in the request * Create FUNDING.yml * Use add for server name header * Add backlog configuration * Default pipelining assistance to true * HBRequestBodyStreamer.drop now returns an EventLoopFuture HBRequestBodyStreamer.drop now returns an EventLoopFuture that is fulfilled when we receive the .end tag. HTTPServerHandler isn't allowed to close the channel until HBRequestBodyStreamer.drop is fulfilled * Use onComplete callback instead of EventLoopFuture (#10) * Use onComplete callback instead of Future * Swift format * Update README.md * GH action changes * HBHTTPResponder comment * Update nightly.yml * Stick response code inside eventLoop.submit {} Need to ensure we are running on the correct eventLoop * Only use `EventLoop` submit if not running on `EventLoop` (#11) * If already in eventloop then don't call submit() * Add test to ensure we don't break running on EmbeddedChannel * Swift format, swift 5.3 * Minor update from project-template * Make HBHTTPResponder.Logger non-optional Also don't log errors with log level error * Clean up channel handler adding (#14) * Clean up channel handler adding * Remove transport services for the moment * Add macOS CI * Remove tcpNoDelay option setting * tcpOption(.tcp_nodelay) not socketOption(.tcp_nodelay) * Remove `HBHTTPServer.ChannelPosition` as we don't use it anymore * Errors, return with a 0 content-length Added test for errors * Add file header using swift format * writeResponse: Use execute not submit as we don't care about result Also saves the allocation of an EventLoopFuture * Unrecognised errors are logged with info * Use mint to run swift-format in sanity CI * Add support for NIOTransportServices on iOS, macOS (#16) * Add support for NIOTransportServices on iOS, macOS * Add Network framework TLS options to configuration * Fix TLS options on macOS * Add NIOTS test * swift format * Split HBHTTPServer.Configuration into separate file * initialize Configuration with TSTLSOptions * Both HBHTTPServer.Configuration.init() should be available on mac * Fix up `TSTLSOptions` to work with enum associated value changes Swift 5.5 has introduced new error Enum cases with associated values cannot be marked potentially unavailable with '@available' * Docker update/Swift 5.4 * HTTP decode optimisation for when there is only one body part (#17) * Optimise http decode when only one body part sent * swift format * Add testStreamedRequestDrop as drop was no longer being tested * Add `TSTLSOptions.options(serverIdentity)` (#19) * Add TSTLSOptions.options(serverIdentity,trustRoots,...) * Reduce TSTLSOptions.options parameters to just serverIdentity I'm not able to test the other options * Add test for TLS while using NIOTransportServices * swift format * Fix ambiguous use of .none error * Fix TLS test on iOS by using non TS HTTPClient * Control request streamer buffer editing Ensure `HBRequestBodyStreamer` buffer is only ever edited on request eventloop * Stop using AsyncHTTPClient for testing (#20) * Add HummingbirdCoreXCT Which includes a basic HTTP client Removes dependency on AsyncHTTPClient * Separate Client.init and connect * Create TimeoutPromise.swift * Add connection close test and HTTP pipelining test * swift format * Add HTTP task to client So I can receive response in return value of `execute`. This is more of a standard API for HTTP client. The client assumes there will be a response for every request and does not pipeline the requests. Will need to add a timeout facility now * Remove EventLoopStream, update tests * tidy up * HBHTTPClientConnection -> HBXCTClient * Add idle channel handler to catch read timeout (#21) * Add idle channel handler to catch read timeout * Increase timeouts for slow process tests * Add content-length header to XCTClient request * Set Decoder state before calling `fireChannelRead` * Allow for HBRequestBody.byteBuffer to be converted to a stream * Updates from project template * TLSConfiguration init changes * Use port 0 for tests * Use NIOCore instead on NIO (#22) Add NIOPosix imports where applicable * Add AsyncSequence support to HBStreamerProtocol (#23) * Add AsyncSequence support to HBStreamerProtocol Add var `sequence` to HBStreamerProtocol which can be used as an `AsyncSequence`. Unfortunately I can't extend a protocol conditionally with a protocol conformance otherwise I would have just extended `HBStreamerProtocol` to conform to `AsyncSequence` * CI for swift 5.5, swift format changes * Async/Await comments * Update from project template * Update to swift 5.5 docker in CI * Add canImport(_Concurrency) to new concurrency code (#24) * Remove _NIOConcurrency imports * Combine HTTP ChannelHandlers (#25) * Combine HTTP encode into server handler Quick test to see if this is faster * Add configuration to server handler * Combine decode handler into server handler * Conform HBHTTPServerHandler to ChannelDuplexHandler So read(context) gets called and comments tidy up stream writing * Remove decode handler * swift format * Don't propagate errors when server handler state is idle * Optimisations: Stop capturing whole HBHTTPRequest in closures (#26) * writeResponse don't catch whole HBHTTPRequest * replace writeResponse request with streamer * move request var extraction to top of readRequest * Remove comment * Added final to class definitions * Use swiftformat 0.48.17 (#28) * Remove unnecessary writeHTTPParts allocation (#30) * Reduce allocation in writeHTTPParts Don't use result from writeAndFlush so don't create promise for it. Return static successful void future * Don't check for errors in writeResponse * swift format * sanity.sh -> validate.sh * ByteBuffer Response Streamer version 2 (#31) * Add HBByteBufferStreamer response streamer Renamed HBRequestBodyStreamer to HBByteBufferStreamer Use in both HBRequest and HBResponse * Add test * Add HBResponseBody.stream(HBByteBufferStreamer) Remove public from symbol HBResponseByteBufferStreamer * swift format * Remove thread sanitizer * Disable code coverage on macOS async tests Currently not working in github actions * swift format * Use HBStreamerProtocol so you can pass request streams directly * Add back pressure support to `HBByteBufferStreamer` (#33) * Add back pressure to HBByteBufferStreamer * Add tests for HBByteBufferStreamer * Various fixes for HBByteBufferStreamer (#34) Renamed dropped -> isFInished Set isFinished after feeding error or end Don't add new promise on until we are sure everything is good Both consumeAll functions now use whenComplete instead of map * Add .vscode to .gitignore * Add CustomStringConvertible conformance To HBHTTPRequest, HBRequestBody, HBHTTPResponse, HBResponseBody * Fix HBRequest/ResponseBody description * Updates from project template * Use NIOHTTP2 v1.19.2 * Use 5.6 nightlies * Added script to rebuild TLS certificates (#36) * Add script to rebuild TLS certificates * Use sniServerName * Swift format * Update project settings * Upgrade to swift-nio-http2 v1.20.0 Includes fix for crash GHSA-q36x-r5x4-h4q6 * Swift 5.6 * Add Sendable conformances (#27) * Sendable conformance * Update validate script and run * Response sendable conformance * @preconcurrency * Update ByteBufferStreamer.swift * Fix tests * Remove @preconcurrency in front of NIO imports * Up swift-nio version * GH action update * More GH action update * Update availability to include macOS 10.15 etc (#37) * Rewrite TimeoutPromise used in HummingbirdCoreXCT (#38) * Rewrite TimeoutPromise used in HummingbirdCoreXCT * Use TimeoutPromise instead of IdleChannelHandler * Remove public from `TimeoutPromise` * More comments * Template updates (Swift 5.7) * Add Swift Package Index documentation link * Use codecov@v3 action * Added context details to assertion error message (#42) When HTTPServerHandler.channelRead finds itself in a unexpected combination of inbound data and channel state, this change makes the error message print the state and the part to make troubleshooting easier. Changes only the printed error message, no changes to functionality. * Fix HTTP2 connection closing * Add Idle state handler configuration (#43) * Initial idle state handling code * Close channel on read idle state This will close the channel if we get a read idle state event while in the process of reading an HTTP packet * Add read/write timeout values * Add close() function to XCTClient syncShutdown also closes the client * Add testReadIdleHandler, testWriteIdleHandler * Allow adding of upgraders to channel initialisers (#44) * Make HTTP2ChannelInitializer public * Update test certificates * Add public init() to HTTP2ChannelInitializer * Use channel.pipeline.syncOperations when building HTTP1 child channel (#45) * Avoid flatSubmit in ByteBufferStreamer (#47) * Avoid eventLoop flatSubmit * Update comment * comment update * Add maxSize to `HBRequestBody.consumeBody` (#48) * Add maxSize to `consumeAll` * Add test for consumeBody(maxSize:on) * Fix consumeAll when called from a Task (#49) * Fix consumeAll when called from a Task * Add async HBRequestBody.consumeBody * Project template update * Swift 5.8 (#50) * Swift 5.8 * Remove #if compiler(>=5.5.2) * ByteBufferStreamer: Only create promises when consumes run out of input (#51) * Deprecate HBSendable (#52) * Deprecate HBSendable * Add some Sendable conformance I missed * Wrap HBByteBufferStreamer internals in NIOLoopBound (#53) * NIOLoopBoundBox HBByteBufferStreamer internal state Move all HBByteBufferStreamer state into an InternalState class and wrap that in a NIOLoopBoundBox * Remove public from internal function * Move backpressure checks into ByteBufferStreamer * onLoop -> runOnLoop * Swift 5.6 fixes * Deprecate consume(on:) * Sendable fixes * Catch invalid header error (#55) So it is easier to debug when invalid headers are written. * Make HBStaticStreamer Sendable with NIOLockedValueBox (#54) * Require Swift 5.7, macOS 10.15 * Server actor and strict concurrency (#60) * Make Server and test client Sendable Server is an actor ChannelInitializers are immutable TestClient APIs are all async/await * Changes to tests for async server/client * Sendable functions ByteBufferStreamer * Move responder to init * Make HBHTTPService a Service * Fixing sendable warnings * Remove unused type * Minor ordering issue causes crash * Merged serverNotRunning and serverShutdown errors * Added comments, improved fatalError message * Make a var a let * Remove -strict-concurrency * Include server channel in onServerRunning callback * Move hummingbird-core sources and tests * Update Package.swift * Delete hummingbird-core subfolder --------- Co-authored-by: PO Bengtsson <po@niffic.se>
adam-fowler
added a commit
that referenced
this pull request
Aug 14, 2023
* Initial commit * Add .swiftformat * Update README.md * Log errors * Add documentation building scripts * Update README.md * Comments, remove public from a few symbols * Don't make string optional in HBHTTPError(_:message) * Added HBHTTPResponseError protocol Gives errors a chance to decide how a response is generated from them * Update README.md point to new location for docs * Moving code about * HBRequestBodyStreamer.consume returns one ByteBuffer as a time * Send .end on stream error * HBRequestBody now has a queue of EventLoopPromise Couldn't get the single promise to work in all situations * Added backpressure on streamed requests. Don't read more while buffer to be processed is over a certain amount * Added tests for streaming request bodies * GH actions * swift format * weird edit from SF * Added HBRequestBodyStreamer.consumeAll(on:_) * Close channel when we receive a quiesce event * Remove connection close headers, as server shuts down correctly now * swift format * ChannelInitializer takes an array of ChannelHandlers (#7) * Make all channel handlers removeable (#8) * Require all channel handlers to be removable * Update HTTPServer.swift * Add testing of other repositories with current state of hummingbird-core * Add support for dropping streamed request bodies This fixes an issue where a request body is never processed but back pressure stalls the channel because the request body is too large. Once we have finished processing the request we "drop" the body so any more request body streaming is ignored * Fix HTTP2 * Body streaming (#9) * Move HBRequestBodyStreamer into its own file * Allow for nil streamer Check for nil byteBuffer. If you receive this no body was included in the request * Create FUNDING.yml * Use add for server name header * Add backlog configuration * Default pipelining assistance to true * HBRequestBodyStreamer.drop now returns an EventLoopFuture HBRequestBodyStreamer.drop now returns an EventLoopFuture that is fulfilled when we receive the .end tag. HTTPServerHandler isn't allowed to close the channel until HBRequestBodyStreamer.drop is fulfilled * Use onComplete callback instead of EventLoopFuture (#10) * Use onComplete callback instead of Future * Swift format * Update README.md * GH action changes * HBHTTPResponder comment * Update nightly.yml * Stick response code inside eventLoop.submit {} Need to ensure we are running on the correct eventLoop * Only use `EventLoop` submit if not running on `EventLoop` (#11) * If already in eventloop then don't call submit() * Add test to ensure we don't break running on EmbeddedChannel * Swift format, swift 5.3 * Minor update from project-template * Make HBHTTPResponder.Logger non-optional Also don't log errors with log level error * Clean up channel handler adding (#14) * Clean up channel handler adding * Remove transport services for the moment * Add macOS CI * Remove tcpNoDelay option setting * tcpOption(.tcp_nodelay) not socketOption(.tcp_nodelay) * Remove `HBHTTPServer.ChannelPosition` as we don't use it anymore * Errors, return with a 0 content-length Added test for errors * Add file header using swift format * writeResponse: Use execute not submit as we don't care about result Also saves the allocation of an EventLoopFuture * Unrecognised errors are logged with info * Use mint to run swift-format in sanity CI * Add support for NIOTransportServices on iOS, macOS (#16) * Add support for NIOTransportServices on iOS, macOS * Add Network framework TLS options to configuration * Fix TLS options on macOS * Add NIOTS test * swift format * Split HBHTTPServer.Configuration into separate file * initialize Configuration with TSTLSOptions * Both HBHTTPServer.Configuration.init() should be available on mac * Fix up `TSTLSOptions` to work with enum associated value changes Swift 5.5 has introduced new error Enum cases with associated values cannot be marked potentially unavailable with '@available' * Docker update/Swift 5.4 * HTTP decode optimisation for when there is only one body part (#17) * Optimise http decode when only one body part sent * swift format * Add testStreamedRequestDrop as drop was no longer being tested * Add `TSTLSOptions.options(serverIdentity)` (#19) * Add TSTLSOptions.options(serverIdentity,trustRoots,...) * Reduce TSTLSOptions.options parameters to just serverIdentity I'm not able to test the other options * Add test for TLS while using NIOTransportServices * swift format * Fix ambiguous use of .none error * Fix TLS test on iOS by using non TS HTTPClient * Control request streamer buffer editing Ensure `HBRequestBodyStreamer` buffer is only ever edited on request eventloop * Stop using AsyncHTTPClient for testing (#20) * Add HummingbirdCoreXCT Which includes a basic HTTP client Removes dependency on AsyncHTTPClient * Separate Client.init and connect * Create TimeoutPromise.swift * Add connection close test and HTTP pipelining test * swift format * Add HTTP task to client So I can receive response in return value of `execute`. This is more of a standard API for HTTP client. The client assumes there will be a response for every request and does not pipeline the requests. Will need to add a timeout facility now * Remove EventLoopStream, update tests * tidy up * HBHTTPClientConnection -> HBXCTClient * Add idle channel handler to catch read timeout (#21) * Add idle channel handler to catch read timeout * Increase timeouts for slow process tests * Add content-length header to XCTClient request * Set Decoder state before calling `fireChannelRead` * Allow for HBRequestBody.byteBuffer to be converted to a stream * Updates from project template * TLSConfiguration init changes * Use port 0 for tests * Use NIOCore instead on NIO (#22) Add NIOPosix imports where applicable * Add AsyncSequence support to HBStreamerProtocol (#23) * Add AsyncSequence support to HBStreamerProtocol Add var `sequence` to HBStreamerProtocol which can be used as an `AsyncSequence`. Unfortunately I can't extend a protocol conditionally with a protocol conformance otherwise I would have just extended `HBStreamerProtocol` to conform to `AsyncSequence` * CI for swift 5.5, swift format changes * Async/Await comments * Update from project template * Update to swift 5.5 docker in CI * Add canImport(_Concurrency) to new concurrency code (#24) * Remove _NIOConcurrency imports * Combine HTTP ChannelHandlers (#25) * Combine HTTP encode into server handler Quick test to see if this is faster * Add configuration to server handler * Combine decode handler into server handler * Conform HBHTTPServerHandler to ChannelDuplexHandler So read(context) gets called and comments tidy up stream writing * Remove decode handler * swift format * Don't propagate errors when server handler state is idle * Optimisations: Stop capturing whole HBHTTPRequest in closures (#26) * writeResponse don't catch whole HBHTTPRequest * replace writeResponse request with streamer * move request var extraction to top of readRequest * Remove comment * Added final to class definitions * Use swiftformat 0.48.17 (#28) * Remove unnecessary writeHTTPParts allocation (#30) * Reduce allocation in writeHTTPParts Don't use result from writeAndFlush so don't create promise for it. Return static successful void future * Don't check for errors in writeResponse * swift format * sanity.sh -> validate.sh * ByteBuffer Response Streamer version 2 (#31) * Add HBByteBufferStreamer response streamer Renamed HBRequestBodyStreamer to HBByteBufferStreamer Use in both HBRequest and HBResponse * Add test * Add HBResponseBody.stream(HBByteBufferStreamer) Remove public from symbol HBResponseByteBufferStreamer * swift format * Remove thread sanitizer * Disable code coverage on macOS async tests Currently not working in github actions * swift format * Use HBStreamerProtocol so you can pass request streams directly * Add back pressure support to `HBByteBufferStreamer` (#33) * Add back pressure to HBByteBufferStreamer * Add tests for HBByteBufferStreamer * Various fixes for HBByteBufferStreamer (#34) Renamed dropped -> isFInished Set isFinished after feeding error or end Don't add new promise on until we are sure everything is good Both consumeAll functions now use whenComplete instead of map * Add .vscode to .gitignore * Add CustomStringConvertible conformance To HBHTTPRequest, HBRequestBody, HBHTTPResponse, HBResponseBody * Fix HBRequest/ResponseBody description * Updates from project template * Use NIOHTTP2 v1.19.2 * Use 5.6 nightlies * Added script to rebuild TLS certificates (#36) * Add script to rebuild TLS certificates * Use sniServerName * Swift format * Update project settings * Upgrade to swift-nio-http2 v1.20.0 Includes fix for crash GHSA-q36x-r5x4-h4q6 * Swift 5.6 * Add Sendable conformances (#27) * Sendable conformance * Update validate script and run * Response sendable conformance * @preconcurrency * Update ByteBufferStreamer.swift * Fix tests * Remove @preconcurrency in front of NIO imports * Up swift-nio version * GH action update * More GH action update * Update availability to include macOS 10.15 etc (#37) * Rewrite TimeoutPromise used in HummingbirdCoreXCT (#38) * Rewrite TimeoutPromise used in HummingbirdCoreXCT * Use TimeoutPromise instead of IdleChannelHandler * Remove public from `TimeoutPromise` * More comments * Template updates (Swift 5.7) * Add Swift Package Index documentation link * Use codecov@v3 action * Added context details to assertion error message (#42) When HTTPServerHandler.channelRead finds itself in a unexpected combination of inbound data and channel state, this change makes the error message print the state and the part to make troubleshooting easier. Changes only the printed error message, no changes to functionality. * Fix HTTP2 connection closing * Add Idle state handler configuration (#43) * Initial idle state handling code * Close channel on read idle state This will close the channel if we get a read idle state event while in the process of reading an HTTP packet * Add read/write timeout values * Add close() function to XCTClient syncShutdown also closes the client * Add testReadIdleHandler, testWriteIdleHandler * Allow adding of upgraders to channel initialisers (#44) * Make HTTP2ChannelInitializer public * Update test certificates * Add public init() to HTTP2ChannelInitializer * Use channel.pipeline.syncOperations when building HTTP1 child channel (#45) * Avoid flatSubmit in ByteBufferStreamer (#47) * Avoid eventLoop flatSubmit * Update comment * comment update * Add maxSize to `HBRequestBody.consumeBody` (#48) * Add maxSize to `consumeAll` * Add test for consumeBody(maxSize:on) * Fix consumeAll when called from a Task (#49) * Fix consumeAll when called from a Task * Add async HBRequestBody.consumeBody * Project template update * Swift 5.8 (#50) * Swift 5.8 * Remove #if compiler(>=5.5.2) * ByteBufferStreamer: Only create promises when consumes run out of input (#51) * Deprecate HBSendable (#52) * Deprecate HBSendable * Add some Sendable conformance I missed * Wrap HBByteBufferStreamer internals in NIOLoopBound (#53) * NIOLoopBoundBox HBByteBufferStreamer internal state Move all HBByteBufferStreamer state into an InternalState class and wrap that in a NIOLoopBoundBox * Remove public from internal function * Move backpressure checks into ByteBufferStreamer * onLoop -> runOnLoop * Swift 5.6 fixes * Deprecate consume(on:) * Sendable fixes * Catch invalid header error (#55) So it is easier to debug when invalid headers are written. * Make HBStaticStreamer Sendable with NIOLockedValueBox (#54) * Require Swift 5.7, macOS 10.15 * Server actor and strict concurrency (#60) * Make Server and test client Sendable Server is an actor ChannelInitializers are immutable TestClient APIs are all async/await * Changes to tests for async server/client * Sendable functions ByteBufferStreamer * Move responder to init * Make HBHTTPService a Service * Fixing sendable warnings * Remove unused type * Minor ordering issue causes crash * Merged serverNotRunning and serverShutdown errors * Added comments, improved fatalError message * Make a var a let * Remove -strict-concurrency * Include server channel in onServerRunning callback * Move hummingbird-core sources and tests * Update Package.swift * Delete hummingbird-core subfolder --------- Co-authored-by: PO Bengtsson <po@niffic.se>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.