runGetState uses toChunks and fromChunks

[kolmodin]

This quickly gets expensive if called multiple times, and can build up a huge stack if called multiple times, like so:

countTrades :: BL.ByteString -> Int
countTrades input = stepper (0, input) where
  stepper (!count, !buffer)
    | BL.null buffer = count
    | otherwise      =
        let (trade, rest, _) = runGetState getTrade buffer 0
        in stepper (count+1, rest)

Code from http://stackoverflow.com/questions/9567040/poor-performance-parsing-binary-file-in-haskell/9573661#9573661

[tibbe]

Why does this build up a huge stack?

[kolmodin]

runGetState takes the Lazy ByteString, and turns it into a list of chunks.
The leftover data that did not get consumed gets converted back into a Lazy ByteString. When that happens, empty chunks gets filtered out. The filter is not strict though.
When this gets repeated many times, there are lots of filters applied to the same lazy bytestring. These multiple filters can eat the whole stack.
The patch above solves it, but needs to be used for more functions in the module before we close this issue.