Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Add a newline, get Partial instead of Some #28

Closed
solidsnack opened this Issue · 2 comments

2 participants

@solidsnack

I have an encountered an infelicity in the incremental parser, where input
that could be returned is delayed unnecessarily. Please find below a GHCi
session demonstrating the problem.

- Prelude
> import Data.ByteString.Char8 (ByteString)
- Prelude Data.ByteString.Char8
> :set -XOverloadedStrings 
- Prelude Data.ByteString.Char8
> import qualified Data.Csv.Incremental as Cassava
- Prelude Data.ByteString.Char8 Cassava
> let Cassava.Partial f = Cassava.decode False :: Cassava.Parser [ByteString]
f :: ByteString -> Cassava.Parser [ByteString]
- Prelude Data.ByteString.Char8 Cassava
> f "a"
Partial <function>
it :: Cassava.Parser [ByteString]
- Prelude Data.ByteString.Char8 Cassava
> f "a\nb"
Some [Right ["a"]] <function>
it :: Cassava.Parser [ByteString]
- Prelude Data.ByteString.Char8 Cassava
> f "a\nb\n"
Partial <function>
it :: Cassava.Parser [ByteString]
- Prelude Data.ByteString.Char8 Cassava
> f "a\nb\nc"
Some [Right ["a"],Right ["b"]] <function>
it :: Cassava.Parser [ByteString]

As you can see, when a\nb\n is presented to the parser, it retains the
entire input, even though it could parse a before.

In my application, I/O is interleaved with parsing on a fixed schedule, with
records flushed to a database at each tick. Cassava is used to group them, so
that only whole tuples are wrapped with COPY ... before being sent to
Postgres. Thus there are a few large transaction per second instead of
thousands and thousands (not really possible with Postgres on the hardware
that I have).

In testing, this problem seems to affect at most 20 rows at a time.

@tibbe
Owner

Thanks for the report. I'll take a look.

@tibbe tibbe closed this in a4f3e28
@tibbe
Owner

Released as 0.2.1.1. Please try it out to make sure it works for you. I tested it on your specific example and that worked fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.