Implement Python runtime crate and shared socket #1399

crisidev · 2022-05-18T14:38:31Z

Motivation and Context

Continue on #1367

Description

Add the Python runtime crate.
Implement SharedSocket support.
Implement logging to tracing support.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

… comments

github-actions · 2022-05-18T17:36:24Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python

A new doc preview is ready to view.

Rust Wrk benchmark report:

Duration: 90 sec, Connections: 32, Threads: 2

Measurement	Deviation	Current	Old
Requests/sec	4.95%	39647.24	37777.96
Total requests	4.97%	3569791	3400702
Total errors	NaN%	0	0
Total successes	4.97%	3569791	3400702
Average latency ms	-3.49%	0.83	0.86
Minimum latency ms	50.00%	0.03	0.02
Maximum latency ms	-13.47%	16.77	19.38
Stdev latency ms	12.00%	0.56	0.5
Transfer Mb	4.97%	371.08	353.5
Connect errors	NaN%	0	0
Read errors	NaN%	0	0
Write errors	NaN%	0	0
Status errors (not 2xx/3xx)	NaN%	0	0
Timeout errors	NaN%	0	0

github-actions · 2022-05-18T22:04:47Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python

A new doc preview is ready to view.

Rust Wrk benchmark report:

Duration: 90 sec, Connections: 32, Threads: 2

Measurement	Deviation	Current	Old
Requests/sec	1.94%	84491.06	82885.18
Total requests	1.85%	7606034	7467652
Total errors	NaN%	0	0
Total successes	1.85%	7606034	7467652
Average latency ms	4.07%	1.28	1.23
Minimum latency ms	0.00%	0.02	0.02
Maximum latency ms	16.80%	28.02	23.99
Stdev latency ms	2.74%	2.25	2.19
Transfer Mb	1.85%	790.65	776.27
Connect errors	NaN%	0	0
Read errors	NaN%	0	0
Write errors	NaN%	0	0
Status errors (not 2xx/3xx)	NaN%	0	0
Timeout errors	NaN%	0	0

jdisanti · 2022-05-18T23:32:03Z

rust-runtime/aws-smithy-http-server-python/rustfmt.toml

+edition = "2021"
+max_width = 120
+# Prevent carriage returns
+newline_style = "Unix"


I'm kind of surprised we haven't had any issues with the aws-smithy-server crate failing lints where rustfmt formatted it properly according to this config (see https://github.com/awslabs/smithy-rs/blob/a0539e20b069a7de021c84521d8f3c7ba098ad6d/tools/ci-build/scripts/check-style-and-lints#L14-L13) since the default max_width is 100. Or maybe we have and I just haven't noticed.

I'm completely onboard with making these settings apply to all of our Rust crates if you want. We should probably be consistent about it though to keep CI simpler.

No need to block on this--just be aware that CI may give you some grief while it remains inconsistent.

I am happy to remove this and make it consistent. I am a fan of 120 chars, but I am ok also with 100 :)

I'm also a fan of 120, so either way

github-actions · 2022-05-19T10:05:23Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python

A new doc preview is ready to view.

Rust Wrk benchmark report:

Duration: 90 sec, Connections: 32, Threads: 2

Measurement	Deviation	Current	Old
Requests/sec	-8.25%	35159.31	38319.58
Total requests	-8.18%	3166884	3448918
Total errors	NaN%	0	0
Total successes	-8.18%	3166884	3448918
Average latency ms	5.49%	0.96	0.91
Minimum latency ms	50.00%	0.03	0.02
Maximum latency ms	11.70%	22.72	20.34
Stdev latency ms	-7.50%	0.74	0.8
Transfer Mb	-8.18%	329.2	358.52
Connect errors	NaN%	0	0
Read errors	NaN%	0	0
Write errors	NaN%	0	0
Status errors (not 2xx/3xx)	NaN%	0	0
Timeout errors	NaN%	0	0

github-actions · 2022-05-19T11:46:43Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python

A new doc preview is ready to view.

Rust Wrk benchmark report:

Duration: 90 sec, Connections: 32, Threads: 2

Measurement	Deviation	Current	Old
Requests/sec	10.25%	80181.3	72727.5
Total requests	10.28%	7223834	6550718
Total errors	NaN%	0	0
Total successes	10.28%	7223834	6550718
Average latency ms	75.00%	0.98	0.56
Minimum latency ms	0.00%	0.02	0.02
Maximum latency ms	36.52%	23.14	16.95
Stdev latency ms	118.18%	1.68	0.77
Transfer Mb	10.28%	750.92	680.95
Connect errors	NaN%	0	0
Read errors	NaN%	0	0
Write errors	NaN%	0	0
Status errors (not 2xx/3xx)	NaN%	0	0
Timeout errors	NaN%	0	0

github-actions · 2022-05-19T16:19:02Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python

A new doc preview is ready to view.

Rust Wrk benchmark report:

Duration: 90 sec, Connections: 32, Threads: 2

Measurement	Deviation	Current	Old
Requests/sec	-15.90%	64777.25	77027.07
Total requests	-15.87%	5836156	6937086
Total errors	NaN%	0	0
Total successes	-15.87%	5836156	6937086
Average latency ms	-7.78%	0.83	0.9
Minimum latency ms	0.00%	0.02	0.02
Maximum latency ms	14.96%	23.98	20.86
Stdev latency ms	-16.45%	1.27	1.52
Transfer Mb	-15.87%	606.67	721.11
Connect errors	NaN%	0	0
Read errors	NaN%	0	0
Write errors	NaN%	0	0
Status errors (not 2xx/3xx)	NaN%	0	0
Timeout errors	NaN%	0	0

github-actions · 2022-05-19T17:32:19Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python

A new doc preview is ready to view.

Rust Wrk benchmark report:

Duration: 90 sec, Connections: 32, Threads: 2

Measurement	Deviation	Current	Old
Requests/sec	-5.71%	43590.34	46228.9
Total requests	-5.64%	3926773	4161552
Total errors	NaN%	0	0
Total successes	-5.64%	3926773	4161552
Average latency ms	-4.90%	0.97	1.02
Minimum latency ms	0.00%	0.02	0.02
Maximum latency ms	4.49%	21.43	20.51
Stdev latency ms	-12.21%	1.15	1.31
Transfer Mb	-5.64%	408.19	432.6
Connect errors	NaN%	0	0
Read errors	NaN%	0	0
Write errors	NaN%	0	0
Status errors (not 2xx/3xx)	NaN%	0	0
Timeout errors	NaN%	0	0

github-actions · 2022-05-20T12:46:58Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python

A new doc preview is ready to view.

Rust Wrk benchmark report:

Duration: 90 sec, Connections: 32, Threads: 2

Measurement	Deviation	Current	Old
Requests/sec	-0.22%	84461.03	84649.15
Total requests	-0.20%	7606203	7621255
Total errors	NaN%	0	0
Total successes	-0.20%	7606203	7621255
Average latency ms	0.00%	1.28	1.28
Minimum latency ms	0.00%	0.02	0.02
Maximum latency ms	2.23%	25.71	25.15
Stdev latency ms	0.00%	2.25	2.25
Transfer Mb	-0.20%	790.67	792.23
Connect errors	NaN%	0	0
Read errors	NaN%	0	0
Write errors	NaN%	0	0
Status errors (not 2xx/3xx)	NaN%	0	0
Timeout errors	NaN%	0	0

david-perez · 2022-05-20T14:19:30Z

rust-runtime/aws-smithy-http-server-python/README.md

+Server libraries for smithy-rs generated servers, targeting pure Python business logic.
+
+<!-- anchor_start:footer -->
+This crate is part of the [AWS SDK for Rust](https://awslabs.github.io/aws-sdk-rust/) and the [smithy-rs](https://github.com/awslabs/smithy-rs) code generator. In most cases, it should not be used directly.


Not part of the AWS Rust SDK. We have the same phrase in aws-smithy-http-server.

rust-runtime/aws-smithy-http-server-python/src/lib.rs

david-perez · 2022-05-20T14:23:31Z

tools/sdk-lints/smithy-rs-maintainers.txt

@@ -3,3 +3,4 @@ velfi
 jdisanti
 crisidev
 david-perez
+82marbag


This should go in a separate PR.

Co-authored-by: david-perez <d@vidp.dev>

github-actions · 2022-05-20T16:01:41Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python

A new doc preview is ready to view.

Rust Wrk benchmark report:

Duration: 90 sec, Connections: 32, Threads: 2

Measurement	Deviation	Current	Old
Requests/sec	2.27%	58965.93	57659.89
Total requests	2.27%	5309140	5191508
Total errors	NaN%	0	0
Total successes	2.27%	5309140	5191508
Average latency ms	6.10%	0.87	0.82
Minimum latency ms	0.00%	0.02	0.02
Maximum latency ms	-12.32%	20.56	23.45
Stdev latency ms	15.32%	1.28	1.11
Transfer Mb	2.27%	551.89	539.66
Connect errors	NaN%	0	0
Read errors	NaN%	0	0
Write errors	NaN%	0	0
Status errors (not 2xx/3xx)	NaN%	0	0
Timeout errors	NaN%	0	0

github-actions · 2022-05-20T16:50:55Z

A new generated diff is ready to view.

AWS SDK (ignoring whitespace)
No codegen difference in the Server Test
No codegen difference in the Server Test Python

A new doc preview is ready to view.

Rust Wrk benchmark report:

Duration: 90 sec, Connections: 32, Threads: 2

Measurement	Deviation	Current	Old
Requests/sec	-1.86%	77174.97	78641.23
Total requests	-1.81%	6952974	7081302
Total errors	NaN%	0	0
Total successes	-1.81%	6952974	7081302
Average latency ms	-5.94%	0.95	1.01
Minimum latency ms	0.00%	0.02	0.02
Maximum latency ms	-0.19%	21.25	21.29
Stdev latency ms	-8.77%	1.56	1.71
Transfer Mb	-1.81%	722.77	736.1
Connect errors	NaN%	0	0
Read errors	NaN%	0	0
Write errors	NaN%	0	0
Status errors (not 2xx/3xx)	NaN%	0	0
Timeout errors	NaN%	0	0

crisidev added 17 commits May 5, 2022 12:18

Add Python subproject scaffolding

be8e740

Add codegen-server-python CI actions

d3025b0

Add CI script for python subproject

68b3423

Remove empty line from ci.mk

9d0ee50

Cleanup dependencies from Grandle build scripts of server-codegen

ce7360f

Configure codegen-server-test-python smithy-cli output directory. Fix…

5a20d5b

… comments

Add decriptions to codegen-server-test projects

fa5b446

Merge branch 'main' into oxipy

d77d479

Merge branch 'main' into oxipy

4a249c6

Fix python RustCodegenServerPlugin license header

aad9ed4

Fix multiple license headers

26c6d0b

Add shared socket implementation

d7e245f

Update shared socket to support both Ipv4 and Ipv6

9a06f0a

Move shared socket implementation in its own runtime crate

00f9f91

Add documentation to SharedSocket

7971e4b

Test SharedSocket try_clone()

67c117a

Merge branch 'main' into oxipy-shared-socket

678c482

crisidev requested a review from a team as a code owner May 18, 2022 14:38

crisidev added enhancement New feature or request server Rust server SDK labels May 18, 2022

crisidev added 2 commits May 18, 2022 15:39

Refactor SharedSocket new() documentation

f6eecad

Update Cargo.toml description

fdf9aa8

crisidev mentioned this pull request May 18, 2022

[WIP] Generate server implementation for pure Python business logic #1367

Open

34 tasks

crisidev added 4 commits May 18, 2022 16:09

Add README

2a0c08d

Add LICENCE file

f6fbd86

Add aws-smithy-http-server-python to the list of server crates

b1a99df

Add 82marbag to the list of maintainers

888654d

rcoh approved these changes May 18, 2022

View reviewed changes

crisidev requested review from david-perez and 82marbag May 18, 2022 15:41

crisidev added 2 commits May 18, 2022 17:54

Add python3-devel to the Docket image to allow building against PyO3

dc5690b

Little documentation fix

b409402

Run assert only on UNIX

180b318

jdisanti approved these changes May 18, 2022

View reviewed changes

crisidev added 2 commits May 19, 2022 10:22

Merge branch 'main' into oxipy-shared-socket

83376a7

Standardize format

f31b4e3

82marbag approved these changes May 19, 2022

View reviewed changes

Add tracing and python logging injection

c4b3477

Make all dependencies versions explicit

8474918

crisidev added 2 commits May 19, 2022 17:38

Change again versions to comply with CI

648a3b1

Change again versions to comply with CI

1b50330

crisidev requested a review from a team as a code owner May 19, 2022 16:43

crisidev added 2 commits May 20, 2022 12:58

Add tests and refactor logging.rs

b656190

Documentation updates

14f0018

david-perez approved these changes May 20, 2022

View reviewed changes

crisidev and others added 2 commits May 20, 2022 15:57

Update rust-runtime/aws-smithy-http-server-python/src/lib.rs

14a3563

Co-authored-by: david-perez <d@vidp.dev>

Update documentation

8398a72

crisidev enabled auto-merge (squash) May 20, 2022 15:24

Revert READMEs because of SDK lint

084a8be

crisidev merged commit ebabb98 into main May 20, 2022

crisidev deleted the oxipy-shared-socket branch May 20, 2022 17:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement Python runtime crate and shared socket #1399

Implement Python runtime crate and shared socket #1399

crisidev commented May 18, 2022 •

edited

Loading

github-actions bot commented May 18, 2022

github-actions bot commented May 18, 2022

jdisanti May 18, 2022

crisidev May 19, 2022

jdisanti May 19, 2022

github-actions bot commented May 19, 2022

github-actions bot commented May 19, 2022

github-actions bot commented May 19, 2022

github-actions bot commented May 19, 2022

github-actions bot commented May 20, 2022

david-perez May 20, 2022

david-perez May 20, 2022

github-actions bot commented May 20, 2022

github-actions bot commented May 20, 2022

Implement Python runtime crate and shared socket #1399

Implement Python runtime crate and shared socket #1399

Conversation

crisidev commented May 18, 2022 • edited Loading

Motivation and Context

Description

github-actions bot commented May 18, 2022

Rust Wrk benchmark report:

Duration: 90 sec, Connections: 32, Threads: 2

github-actions bot commented May 18, 2022

Rust Wrk benchmark report:

Duration: 90 sec, Connections: 32, Threads: 2

jdisanti May 18, 2022

Choose a reason for hiding this comment

crisidev May 19, 2022

Choose a reason for hiding this comment

jdisanti May 19, 2022

Choose a reason for hiding this comment

github-actions bot commented May 19, 2022

Rust Wrk benchmark report:

Duration: 90 sec, Connections: 32, Threads: 2

github-actions bot commented May 19, 2022

Rust Wrk benchmark report:

Duration: 90 sec, Connections: 32, Threads: 2

github-actions bot commented May 19, 2022

Rust Wrk benchmark report:

Duration: 90 sec, Connections: 32, Threads: 2

github-actions bot commented May 19, 2022

Rust Wrk benchmark report:

Duration: 90 sec, Connections: 32, Threads: 2

github-actions bot commented May 20, 2022

Rust Wrk benchmark report:

Duration: 90 sec, Connections: 32, Threads: 2

david-perez May 20, 2022

Choose a reason for hiding this comment

david-perez May 20, 2022

Choose a reason for hiding this comment

github-actions bot commented May 20, 2022

Rust Wrk benchmark report:

Duration: 90 sec, Connections: 32, Threads: 2

github-actions bot commented May 20, 2022

Rust Wrk benchmark report:

Duration: 90 sec, Connections: 32, Threads: 2

crisidev commented May 18, 2022 •

edited

Loading