Static table only QPACK

[kachayev]

Motivation:

QPACK is headers encoding algorithms essential for H3 protocol.

Modifications:

• Key additions are QpackEncoder, QpackDecoder and QpackStaticTable, API is shown in the test case.
• I made a small Python script to parse static table entries from Appendix A of the spec and generate Java code, didn't include the script itself though.
• Logic of name/value and name-only indexing is implemented the same way static table works for HPACK with only caveats for 0-based indexing used in QPACK. I'm not convinced this is the faster approach though, I want to play around with different structures & search methods to test out performance. Also, encoder calls static table twice to get name/value or name-only matches. It seems like this could be optimized by returning masked index to differentiate 2 cases.
• Http3Headers, DefaultHttp3Headers are copy-pasted from HTTP/2 codec (corresponding classes). Not sure if we even need H3 headers to be defined separately or cleanup HTTP/2 implementation to have less dependencies on itself, e.g. HTTP/2-specific exceptions.
• HuffmanEncoder and HuffmanDecoder are copy-pastes as well. Those classes are package-private in HTTP/2 code and decoder uses HTTP/2-related exception. It would be nice to move both into utils or common instead of keep 2 separate copies. Same goes for CharSequenceMap.
• Http3Exception is some sort of a placeholder for future errors processing layer. Not sure if we want to follow HTTP/2 codec model though.
• Headers validation for QpackDecoder is not implemented yet. Once again, it would be nice if we can cleanup HTTP/2 implementation to be able to re-use blocks of it (including headers sink and header validation logic).
• Neither encoder nor decoder support dynamic table control messages (I will add it later when working on a dynamic table support).
• Literal names and values are always huffman compressed which might be not optimal in all cases. I will look into making the same optimization HPACK is doing when deciding between sending raw or compressed value.
• HPACK decoder implemented using state transition flow (I assume to reduce function calls overhead, right?). I've implemented decoder using functions as it was much easier to keep track of the logic. I would rather prefer to make sure that the implementation is, in fact, correct before working on optimizations. Plus, having good benchmarks is also must have before optimizing.

[normanmaurer]

First quic review

[normanmaurer]

let's update the validateName methods signature to include throws declaration so we don't need to use this hack

[kachayev]

Errors/exceptions handling definitely needs some love here. So far, it's mostly copy from HTTP/2

[kachayev]

This one turned out to be a pretty deep hole: it relies on an interface from codec. I'm going to just leave it "as is" for now. We can change it later

[kachayev]

@normanmaurer I pushed quite a significant change on how static table lookup works. See QpackStaticTable::findFieldIndex function. Rather than resolving name -> index twice (as http2 codec HPACK is doing), the function "memorizes" name index and return it when exact match is not found. Masked in a way that caller could differentiate field index from name index.

[normanmaurer]

@kachayev looks great... let me know when you think its ready for a merge. We can do a followup later when we want to improve it / find bugs.

[kachayev]

@normanmaurer I will do another pass tomorrow, mainly to cleanup errors handling and validation. And will open issues for enhancements that I've mentioned in the very beginning. Also, I'm going to make a PR for codec-http2 with the same trick to avoid double hash map lookup.

[normanmaurer]

Sounds good 👌

[normanmaurer]

@kachayev wdyt about moving the Huffman stuff + CharSequenceMap into the codec package (just like the other compression stuff) so we could reuse it from h2 and h3.

[kachayev]

@normanmaurer Totally agree. I've mentioned this earlier

HuffmanEncoder and HuffmanDecoder are copy-pastes as well. Those classes are package-private in HTTP/2 code and decoder uses HTTP/2-related exception. It would be nice to move both into utils or common instead of keep 2 separate copies. Same goes for CharSequenceMap.

Decoder needs to be cleanup a bit as it throws HTTP/2 connection error, - this logic should stay in codec-http2. I can work on PR

[kachayev]

@normanmaurer I think it's good so far... Errors handling (e.g. validation exceptions you've mentioned) and general logic of Http3Exception would be much easier to work on when the rest of the code is shaped. E.g. some decoder problems should shutdown the stream, some the entire connection. I think we should first identify all use cases and then adjust implementation where necessary

[normanmaurer]

Just to clarify... This could be just the "standard" HuffmanDecoder that would be shared with http2 in the future, correct @kachayev ?

[kachayev]

Yes, exactly. The only thing we will need to change:

netty-incubator-codec-http3/src/main/java/io/netty/incubator/codec/http3/QpackHuffmanDecoder.java

Line 4649 in 8110ecc

private static final Http3Exception BAD_ENCODING = new Http3Exception("QPACK - Bad Encoding");

[kachayev]

When moving to codec package, this exception should be changed to something fairly generic. Meaning that the code should wrap decoding into try/catch to produce meaningful error information "locally"