Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Hyper Upgrade #1925

Open
1 of 26 tasks
jdisanti opened this issue Oct 31, 2022 · 3 comments
Open
1 of 26 tasks

Hyper Upgrade #1925

jdisanti opened this issue Oct 31, 2022 · 3 comments
Labels
client tracking Meta-issues to track overall progress

Comments

@jdisanti
Copy link
Collaborator

jdisanti commented Oct 31, 2022 •
edited by rcoh
Loading

This tracks progress on the upgrade to Hyper 1.0

Many of these changes are prototyped in this branch which can send an E2E request with RusTLS

Implementation Tasks

Upstream Changes

  • Backport capture_connection and poison to hyper-util
  • Create crate for 0.14 -> 1.x bridge. This allows us to use hyper-rustls and hyper-nativetls against the RC (or 1.0)

SDK changes

  • Backwards compatible solution for SegmentedBuf in ByteStream
  • hyper::body::channel() replacement for dvr
  • Careful upgrade + testing of AwsChunkedBody—the Frame API changes the flow here a little bit
  • Update all tests
  • Update Checksum Body
  • Add Spawn to AwsSmithyAsync—bridge this with Hyper Executor (you now must provide an executor to create a Hyper builder)
  • Event Stream Upgrade

Testing Tasks

  • Simple request/response with:
    • Native TLS
    • Rustls
  • Streaming request/response with:
    • Native TLS
    • Rustls
  • Checksum validating response (S3 with checksums)
  • Checksum calculating request (S3 with checksums)
  • Unidirectional event stream operation (e.g., Transcribe Streaming and S3 Select):
    • HTTP/1.1
    • h2
    • Native TLS
    • Rustls
  • Bidirectional event stream operation (Pokemon client/server test?):
    • h2
    • Native TLS
    • Rustls
@jdisanti
Copy link
Collaborator Author

jdisanti commented Dec 7, 2022 •
edited by rcoh
Loading

I spent some time working on this this week. I was able to get aws-smithy-http compiling with most tests passing, and some of aws-smithy-checksums (ended up stubbing out the checksum calculating body to move forward). My work is available on the hyper-1x-experimentation-dec-7-2022 branch. Findings below.

Further work that is unblocked right now:

  • Fix failing tests in aws-smithy-http after upgrade
  • Reimplement checksum calculating body in aws-smithy-checksums to support new Body trait

Prerequisites before continuing:

  • Need hyper-util to release with a Connect/Connection traits and connection pool
  • Need an upgraded TLS library (hyper-tls or hyper-rustls) to actually test the SDK. We can do this work ourselves if the previous prerequisite is completed.

Work that can be done after prerequisites:

  • Upgrade or wait for hyper-rustls to support hyper 1.x
  • Upgrade or wait for hyper-tls to support hyper 1.x
  • Fix dvr in aws-smithy-client (transition from hyper::body::channel() to http_body_util::StreamBody)
  • Continue investigation

@jdisanti
Copy link
Collaborator Author

jdisanti commented Dec 8, 2022

Note: On the Body impls, it should be safe to assume trailers come after data frames since hyper will error out otherwise.

@jdisanti jdisanti self-assigned this Dec 9, 2022
@jdisanti
Copy link
Collaborator Author

jdisanti commented Dec 9, 2022 •
edited
Loading

Findings so far:

  • Depending on what hyper-util ends up supporting, we may need to fully implement the HTTP versioning RFC to avoid needing to worry about ALPN (or it may be cheaper to do so).

Work in progress test matrix:

  • Simple request/response with:
    • Native TLS
    • Rustls
  • Streaming request/response with:
    • Native TLS
    • Rustls
  • Checksum validating response (S3 with checksums)
  • Checksum calculating request (S3 with checksums)
  • Unidirectional event stream operation (e.g., Transcribe Streaming and S3 Select):
    • HTTP/1.1
    • h2
    • Native TLS
    • Rustls
  • Bidirectional event stream operation (Pokemon client/server test?):
    • h2
    • Native TLS
    • Rustls

@jdisanti jdisanti changed the title Start upgrade to hyper 1.0 RC1 Start upgrade to hyper 1.0 RC2 Jan 10, 2023
@rcoh rcoh changed the title Start upgrade to hyper 1.0 RC2 Hyper Upgrade Apr 13, 2023
@rcoh rcoh added the tracking Meta-issues to track overall progress label Apr 13, 2023
@rcoh rcoh assigned rcoh and unassigned jdisanti Apr 13, 2023
@jdisanti jdisanti mentioned this issue Nov 28, 2023
Open
11 tasks
This was referenced Mar 4, 2024
Open
Open
@rcoh rcoh mentioned this issue Mar 5, 2024
Merged
2 tasks
jdisanti pushed a commit that referenced this issue Mar 7, 2024
@rcoh
Add Initial Support for Hyper 1.0 (#3461)
f68b5ee 
## Motivation and Context
- #1925 

## Description
This adds a minimal Hyper client, focusing on not exposing any unstable
APIs. For this reason, the `Client::Builder` customization API is not
exposed anymore. We do this because at some point in the future, we will
likely move away from the hyper-util based Client.

The code for this was lifted directly from the Hyper 0.14 implementation
but updated for new traits.

However, this does come with some new valuable pieces:
1. Support for aws-lc (no FIPS yet)
2. Support for providing a custom DNS resolver

## Testing
- E2E test with Hyper. A Canary should also be added
(awslabs/aws-sdk-rust#1089)

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
@jdisanti jdisanti added the client label Apr 5, 2024
@rcoh rcoh removed their assignment Apr 25, 2024
@Velfi Velfi mentioned this issue May 30, 2024
Merged
2 tasks
github-merge-queue bot pushed a commit that referenced this issue Jun 3, 2024
@Velfi
implement http-body 1.0 for PathBody (#3673)
9c1ae5a 
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
#1925 

## Description
<!--- Describe your changes in detail -->
Implements the v1 `http_body::Body` trait for `PathBody`. Part of the
ongoing hyper v1 upgrade.
This also moves a pre-1.0 impl into its own module.

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
I ported the tests too

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
client tracking Meta-issues to track overall progress
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jdisanti @rcoh