Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTTP/2: Many specification compliancy issues #5761

Open
buchgr opened this issue Aug 29, 2016 · 10 comments
Open

HTTP/2: Many specification compliancy issues #5761

buchgr opened this issue Aug 29, 2016 · 10 comments

Comments

@buchgr
Copy link
Contributor

buchgr commented Aug 29, 2016

I ran h2spec against the hello world HTTP/2 server (4.1 branch, with removed HTTP upgrade logic). The result: 70 tests, 37 passed, 1 skipped, 32 failed.

From skimming over the list of errors, the failed tests seem to be mostly due to Netty responding with wrong error codes. Here is a detailed list of what went wrong.

cc: @Scottmitch @nmittler @louiscryan @ejona86

@normanmaurer
Copy link
Member

@buchgr thanks for having a look! I think I should write a maven plugin to do this as part of our build just as we do with the autobahn test suite (web sockets)

@buchgr
Copy link
Contributor Author

buchgr commented Aug 29, 2016

@normanmaurer ... and first fix all the errors 😜

@nmittler
Copy link
Member

@buchgr great, thanks for looking at this! Yes ... fixing errors sgtm 😜

@buchgr
Copy link
Contributor Author

buchgr commented Aug 29, 2016

Working on a fix. Most errors seem pretty straightfoward...

@normanmaurer
Copy link
Member

Thanks a lot!

Am 29.08.2016 um 16:01 schrieb Jakob Buchgraber notifications@github.com:

Working on a fix. Most errors seem pretty straightfoward...


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@carocean
Copy link

Everybody here, my platform in addition to use asynchronous circuit also used in the synchronous circuit, and I did not find netty synchronous architecture, thinking, I realized a framework, called net Rio, it to NiO based synchronization response mechanism, basic to meet the needs of our platform. The project was open source, hope you want to join

@Scottmitch
Copy link
Member

thanks for uncovering and patching @buchgr !

buchgr added a commit to buchgr/netty that referenced this issue Aug 31, 2016
Motivation:

h2spec [1] reported 32 issues [2] with Netty's HTTP/2 implementation.

Modifications:

Fixed 11 of those issues. Mostly wrong error codes or added missing validation
code in the frame reader.

Result:

11 fewer errors. h2spec now reports: 70 tests, 48 passed, 1 skipped, 21 failed

[1] https://github.com/summerwind/h2spec
[2] netty#5761
normanmaurer pushed a commit that referenced this issue Sep 1, 2016
Motivation:

h2spec [1] reported 32 issues [2] with Netty's HTTP/2 implementation.

Modifications:

Fixed 11 of those issues. Mostly wrong error codes or added missing validation
code in the frame reader.

Result:

11 fewer errors. h2spec now reports: 70 tests, 48 passed, 1 skipped, 21 failed

[1] https://github.com/summerwind/h2spec
[2] #5761
@buchgr
Copy link
Contributor Author

buchgr commented Sep 5, 2016

Running h2spec now: 70 tests, 48 passed, 1 skipped, 21 failed

When increasing the DEFAULT_MAX_HEADER_SIZE to 64KiB, 6 fewer tests fail. Specfically those in 6.10 CONTINUATION, as this test sends large HEADERS / CONTINUATION frames. It's totally legit for a HTTP/2 implementation to not support such large HEADERS though #5774 (comment) ...

@chhsiao90
Copy link
Contributor

May I ask that if I could help on this issue?
I will pick up some simple case to fix and create issue and PR for that.
It seems that most failed test are occurred at http2 validation.
Thanks!

@normanmaurer
Copy link
Member

@chhsiao90 sure just open PRs, we love contributions

liuzhengyang pushed a commit to liuzhengyang/netty that referenced this issue Sep 10, 2017
Motivation:

h2spec [1] reported 32 issues [2] with Netty's HTTP/2 implementation.

Modifications:

Fixed 11 of those issues. Mostly wrong error codes or added missing validation
code in the frame reader.

Result:

11 fewer errors. h2spec now reports: 70 tests, 48 passed, 1 skipped, 21 failed

[1] https://github.com/summerwind/h2spec
[2] netty#5761
normanmaurer pushed a commit to madgnome/netty that referenced this issue Oct 25, 2019
Motivation:

Netty HTTP/2 implementation is not 100% compliant to the spec. This
commit improves the compliance regarding headers validation,
in particular pseudo-headers and connection ones.

According to the spec:
   All HTTP/2 requests MUST include exactly one valid value for the
   ":method", ":scheme", and ":path" pseudo-header fields, unless it is
   a CONNECT request (Section 8.3).  An HTTP request that omits
   mandatory pseudo-header fields is malformed (Section 8.1.2.6).

Modifications:

- Introduce Http2HeadersValidator class capable of validating HTTP/2
headers
- Invoke validation from DefaultHttp2ConnectionDecoder#onHeadersRead
- Modify tests to use valid headers when required
- Modify HttpConversionUtil#toHttp2Headers to not add :scheme and
:path header on CONNECT method in order to conform to the spec

Result:

- Initial requests without :method, :path, :scheme will fail
- Initial requests with multiple values for :method, :path, :scheme
will fail
- Initial requests with an empty :path fail
- Requests with connection-specific header field will fail
- Requests with TE header different than "trailers" will fail
-
- Fixes 8.1.2.2 tests from h2spec netty#5761
- Fixes 8.1.2.3 tests from h2spec netty#5761
normanmaurer pushed a commit that referenced this issue Oct 27, 2019
Motivation:

Netty HTTP/2 implementation is not 100% compliant to the spec. This
commit improves the compliance regarding headers validation,
in particular pseudo-headers and connection ones.

According to the spec:
   All HTTP/2 requests MUST include exactly one valid value for the
   ":method", ":scheme", and ":path" pseudo-header fields, unless it is
   a CONNECT request (Section 8.3).  An HTTP request that omits
   mandatory pseudo-header fields is malformed (Section 8.1.2.6).

Modifications:

- Introduce Http2HeadersValidator class capable of validating HTTP/2
headers
- Invoke validation from DefaultHttp2ConnectionDecoder#onHeadersRead
- Modify tests to use valid headers when required
- Modify HttpConversionUtil#toHttp2Headers to not add :scheme and
:path header on CONNECT method in order to conform to the spec

Result:

- Initial requests without :method, :path, :scheme will fail
- Initial requests with multiple values for :method, :path, :scheme
will fail
- Initial requests with an empty :path fail
- Requests with connection-specific header field will fail
- Requests with TE header different than "trailers" will fail
-
- Fixes 8.1.2.2 tests from h2spec #5761
- Fixes 8.1.2.3 tests from h2spec #5761
normanmaurer pushed a commit that referenced this issue Oct 27, 2019
Motivation:

Netty HTTP/2 implementation is not 100% compliant to the spec. This
commit improves the compliance regarding headers validation,
in particular pseudo-headers and connection ones.

According to the spec:
   All HTTP/2 requests MUST include exactly one valid value for the
   ":method", ":scheme", and ":path" pseudo-header fields, unless it is
   a CONNECT request (Section 8.3).  An HTTP request that omits
   mandatory pseudo-header fields is malformed (Section 8.1.2.6).

Modifications:

- Introduce Http2HeadersValidator class capable of validating HTTP/2
headers
- Invoke validation from DefaultHttp2ConnectionDecoder#onHeadersRead
- Modify tests to use valid headers when required
- Modify HttpConversionUtil#toHttp2Headers to not add :scheme and
:path header on CONNECT method in order to conform to the spec

Result:

- Initial requests without :method, :path, :scheme will fail
- Initial requests with multiple values for :method, :path, :scheme
will fail
- Initial requests with an empty :path fail
- Requests with connection-specific header field will fail
- Requests with TE header different than "trailers" will fail
-
- Fixes 8.1.2.2 tests from h2spec #5761
- Fixes 8.1.2.3 tests from h2spec #5761
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants