Decompression could be optimized (would be 5x to 20x faster)

[rflauer]

Hi,

LZW decompression could be optimized to be somewhat faster by not using the classic build-a-stack-then-reverse-it technique but rather using references in the read buffer.

I have implemented this in a GIF reader a while ago. If that's of interest I could try working on a patch.

Cheers,

Robert.

[vapier]

feel free to send a PR

[rflauer]

I did some preliminary investigation; this is a bit trickier for the general case than for the GIF case (because a code may refer to a position that is no longer in the current buffer whereas for a GIF image you can keep the whole image in memory) but I think it can be done.

[andrew-aladev]

I've investigated this question. I am not able to read this idea in ncompress, it is not readable, but we have LZW AB instead:

prev_code = prefixes[code]
last_symbol = terminators[code]

We are able to find previous code for current code, we are able to find last symbol for current code. We can see that this solution targets the least possible amount of memory. It can be named as "reverted trie on unidirectional linked list". And it requires you to write data in opposite direction, of course.

It requires only ((2 ** n_bits) - 257) * (sizeof(code_t) + 2). 261 KB for 16 bit codes.

If we want to write output in same direction we have to implement "trie on bidirectional linked list" ot "trie on bidirectional sparse array" or "bidirectional hash map". It is possible but I don't think that it will speed up decompression.

I think that penalty of maintaining any bidirectional structure will be so much more than some buffer remapping.