New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: Token stream #560

Open
wants to merge 6 commits into
base: master
from

Conversation

Projects
None yet
2 participants
@phadej
Copy link
Contributor

phadej commented Jul 11, 2017

While working on the sax-like stream based JSON parser, I noticed that parsing Value with it is faster than with attoparsec. It started as a separate library, but as it makes isolated (and quite internal) part of aeson a bit faster, I made this PR. If you think this doesn't below into aeson, I'm ok with making it into own library (exposing few internals jsstring_ and scientific would be useful though).

Notes:

Differences:

  • slightly better error reporting: #558
> decodeValue "{ 1 "
Left "Expecting record key or '}', got token TkError \"Expecting record key or '}', got 1 \""
  • slightly better performance
  • cannot separate prime-less and primeless versions (decode & decode')

Further work

  • Write version consuming strict ByteString.
  • Should this be default or not?
  • It's possible to make a variant of TokenStream which rules out invalid streams. I think it's possible to have an encoding such for TokenStream without TkError (which can be enforced with a GADT parameter, but I don't see a need to do so), one could write total TokenStream -> Value function.
    This would require GADTs,, DataKinds and TypeFamilies. This encoding might be useful for hand-written "fancy" TokenStream consumers.
  • Build a parser consuming TokenStream (the list version). First iteration could be done using parsec. Probably for little more performance, one would need specialized version at the end.
    • parsing records a little problem:
      - tag-encoding of sum (one would need to retain the pointer to the point where object tokens started). Similarly monadic parsers are problematic
      - record's unordered nature is a problem too, but not impossible
    • as a con, we can avoid building hashmaps and vectors when we don't need to (e.g. parsing Map!)
    • makes possible to write parsers taking advantage of the order of keys in JSON!
    • makes it simpler to write record parser disallowing keys (will fail earlier)
    • skipping parser is trivial (decode and decode' separating might not be needed!)

Benchmarks

my machine is surprisingly noisy right now:

benchmarking decode/en/aeson/lazy
time                 1.832 ms   (1.799 ms .. 1.865 ms)
                     0.998 R²   (0.997 R² .. 0.999 R²)
mean                 1.817 ms   (1.803 ms .. 1.839 ms)
std dev              57.44 μs   (40.03 μs .. 85.58 μs)
variance introduced by outliers: 18% (moderately inflated)

benchmarking decode/en/aeson/strict
time                 1.819 ms   (1.800 ms .. 1.839 ms)
                     0.998 R²   (0.998 R² .. 0.999 R²)
mean                 1.855 ms   (1.835 ms .. 1.887 ms)
std dev              77.86 μs   (52.22 μs .. 110.1 μs)
variance introduced by outliers: 28% (moderately inflated)

benchmarking decode/en/aeson/stricter
time                 1.947 ms   (1.927 ms .. 1.967 ms)
                     0.999 R²   (0.998 R² .. 0.999 R²)
mean                 1.956 ms   (1.944 ms .. 1.975 ms)
std dev              51.37 μs   (32.60 μs .. 78.53 μs)
variance introduced by outliers: 13% (moderately inflated)

benchmarking decode/en/aeson/hackage
time                 2.109 ms   (2.079 ms .. 2.142 ms)
                     0.998 R²   (0.997 R² .. 0.999 R²)
mean                 2.129 ms   (2.112 ms .. 2.159 ms)
std dev              73.19 μs   (45.28 μs .. 119.2 μs)
variance introduced by outliers: 20% (moderately inflated)

benchmarking decode/en/aeson/hackage'
time                 1.921 ms   (1.905 ms .. 1.931 ms)
                     0.999 R²   (0.998 R² .. 1.000 R²)
mean                 1.942 ms   (1.934 ms .. 1.993 ms)
std dev              48.40 μs   (11.99 μs .. 120.8 μs)
variance introduced by outliers: 11% (moderately inflated)

benchmarking decode/en/aeson/parser
time                 2.092 ms   (2.069 ms .. 2.119 ms)
                     0.998 R²   (0.997 R² .. 0.999 R²)
mean                 2.109 ms   (2.095 ms .. 2.135 ms)
std dev              60.11 μs   (34.63 μs .. 93.45 μs)
variance introduced by outliers: 15% (moderately inflated)

benchmarking decode/en/aeson/stream
time                 1.983 ms   (1.803 ms .. 2.198 ms)
                     0.966 R²   (0.945 R² .. 0.999 R²)
mean                 1.857 ms   (1.828 ms .. 1.936 ms)
std dev              144.3 μs   (67.70 μs .. 300.8 μs)
variance introduced by outliers: 57% (severely inflated)

benchmarking decode/en/json
time                 4.208 ms   (2.040 ms .. 6.774 ms)
                     0.194 R²   (0.044 R² .. 0.421 R²)
mean                 10.07 ms   (8.596 ms .. 12.12 ms)
std dev              4.890 ms   (3.838 ms .. 6.278 ms)
variance introduced by outliers: 97% (severely inflated)

screenshot from 2017-07-11 19-25-50

Legend (numbers in parentheses point which benches are virtually the same)

  • lazy: decode using stream (1)
  • strict: decode' using stream
  • stricter: decodeStrict using attoparsec (3)
  • hackage: decode using attoparsec from aeson 1.2.1.0 (cffi: false) (2)
  • hackage' decodeStrict as previous (3)
  • parser: decodeWith from Parser (2)
  • stream: decodeWith from Stream (1)
  • json: using http://hackage.haskell.org/package/json
@phadej

This comment has been minimized.

Copy link
Contributor Author

phadej commented Jul 11, 2017

Forget this, I accidentally enabled cffi: true.


Added few bang patterns, now the speedup is noticeable (json-stream has C code, aeson is pure haskell, cffi: false)

screenshot from 2017-07-11 20-47-13

benchmarking decode/en/aeson/lazy
time                 1.137 ms   (1.124 ms .. 1.150 ms)
                     0.999 R²   (0.998 R² .. 0.999 R²)
mean                 1.144 ms   (1.135 ms .. 1.156 ms)
std dev              33.15 μs   (23.02 μs .. 47.10 μs)
variance introduced by outliers: 18% (moderately inflated)

benchmarking decode/en/aeson/strict
time                 1.156 ms   (1.147 ms .. 1.167 ms)
                     0.999 R²   (0.998 R² .. 0.999 R²)
mean                 1.165 ms   (1.157 ms .. 1.183 ms)
std dev              38.79 μs   (21.75 μs .. 67.99 μs)
variance introduced by outliers: 22% (moderately inflated)

benchmarking decode/en/aeson/stricter
time                 1.291 ms   (1.278 ms .. 1.302 ms)
                     0.999 R²   (0.998 R² .. 0.999 R²)
mean                 1.294 ms   (1.286 ms .. 1.312 ms)
std dev              38.92 μs   (21.37 μs .. 66.90 μs)
variance introduced by outliers: 19% (moderately inflated)

benchmarking decode/en/aeson/hackage
time                 2.079 ms   (2.053 ms .. 2.105 ms)
                     0.998 R²   (0.997 R² .. 0.999 R²)
mean                 2.101 ms   (2.081 ms .. 2.155 ms)
std dev              108.0 μs   (46.01 μs .. 207.6 μs)
variance introduced by outliers: 36% (moderately inflated)

benchmarking decode/en/aeson/hackage'
time                 1.906 ms   (1.893 ms .. 1.915 ms)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 1.923 ms   (1.915 ms .. 1.951 ms)
std dev              43.34 μs   (8.642 μs .. 90.59 μs)

benchmarking decode/en/aeson/parser
time                 1.470 ms   (1.451 ms .. 1.490 ms)
                     0.998 R²   (0.997 R² .. 0.999 R²)
mean                 1.475 ms   (1.465 ms .. 1.497 ms)
std dev              47.67 μs   (29.14 μs .. 81.88 μs)
variance introduced by outliers: 20% (moderately inflated)

benchmarking decode/en/aeson/stream
time                 1.244 ms   (1.189 ms .. 1.314 ms)
                     0.985 R²   (0.981 R² .. 0.994 R²)
mean                 1.200 ms   (1.180 ms .. 1.227 ms)
std dev              78.37 μs   (52.47 μs .. 103.4 μs)
variance introduced by outliers: 52% (severely inflated)

benchmarking decode/en/json-stream
time                 1.244 ms   (1.234 ms .. 1.251 ms)
                     0.999 R²   (0.997 R² .. 1.000 R²)
mean                 1.259 ms   (1.253 ms .. 1.276 ms)
std dev              30.39 μs   (8.533 μs .. 57.82 μs)
variance introduced by outliers: 13% (moderately inflated)

benchmarking decode/en/json
time                 4.803 ms   (4.737 ms .. 4.862 ms)
                     0.999 R²   (0.998 R² .. 0.999 R²)
mean                 4.821 ms   (4.796 ms .. 4.868 ms)
std dev              99.23 μs   (67.59 μs .. 155.7 μs)

@phadej phadej force-pushed the phadej:token-stream branch from bfd74fc to 97c710a Jul 12, 2017

@phadej phadej referenced this pull request Jul 12, 2017

Merged

Benchmarks fix #562 #563

@phadej phadej force-pushed the phadej:token-stream branch from 97c710a to bd5e567 Jul 12, 2017

@phadej

This comment has been minimized.

Copy link
Contributor Author

phadej commented Jul 12, 2017

TokenStream approach is useful for stream things, maximum residency of stream processing is smaller by magnitude (while processing only 438k file)

The aeson-count-laureates benchmark is too manual, but that's further work to make high-level parsing via TokenStream

./aeson-count-laureates tokenStream +RTS -s
910
      50,834,544 bytes allocated in the heap
          32,376 bytes copied during GC
          85,440 bytes maximum residency (3 sample(s))
          92,992 bytes maximum slop
               2 MB total memory in use (0 MB lost due to fragmentation)

                                     Tot time (elapsed)  Avg pause  Max pause
  Gen  0        94 colls,     0 par    0.000s   0.000s     0.0000s    0.0000s
  Gen  1         3 colls,     0 par    0.000s   0.000s     0.0001s    0.0002s

  INIT    time    0.000s  (  0.000s elapsed)
  MUT     time    0.008s  (  0.009s elapsed)
  GC      time    0.000s  (  0.000s elapsed)
  EXIT    time    0.000s  (  0.000s elapsed)
  Total   time    0.044s  (  0.009s elapsed)

  %GC     time       0.0%  (5.1% elapsed)

  Alloc rate    6,354,318,000 bytes per MUT second

  Productivity 100.0% of total user, 94.2% of total elapsed

./aeson-count-laureates parseJSON +RTS -s
910
      53,038,280 bytes allocated in the heap
      12,378,080 bytes copied during GC
       3,034,968 bytes maximum residency (5 sample(s))
          69,280 bytes maximum slop
               7 MB total memory in use (0 MB lost due to fragmentation)

                                     Tot time (elapsed)  Avg pause  Max pause
  Gen  0        96 colls,     0 par    0.004s   0.004s     0.0000s    0.0001s
  Gen  1         5 colls,     0 par    0.004s   0.005s     0.0010s    0.0024s

  INIT    time    0.000s  (  0.000s elapsed)
  MUT     time    0.008s  (  0.009s elapsed)
  GC      time    0.008s  (  0.009s elapsed)
  EXIT    time    0.000s  (  0.000s elapsed)
  Total   time    0.052s  (  0.018s elapsed)

  %GC     time      15.4%  (49.1% elapsed)

  Alloc rate    6,629,785,000 bytes per MUT second

  Productivity  84.6% of total user, 50.6% of total elapsed

@phadej phadej force-pushed the phadej:token-stream branch from 38753dd to 632eb2c Jul 12, 2017

@phadej

This comment has been minimized.

Copy link
Contributor Author

phadej commented Jul 12, 2017

For fancy encoding of token stream see https://gist.github.com/phadej/31bdb72c815766edde1eaf3efe8cf0ff, that would allow making safe skip as in aeson-count-laureates benchmark.

I think it's a good exercise for type-hackery, but aeson isn't a good place for it.

@phadej phadej referenced this pull request Jul 13, 2017

Open

RFC: Object parser #564

0 of 3 tasks complete

@phadej phadej changed the title WIP: Token stream RFC: Token stream Jul 13, 2017

@phadej phadej force-pushed the phadej:token-stream branch from 632eb2c to ebdf34b Jul 14, 2017

-------------------------------------------------------------------------------

newtype TokenParser = TokenParser
{ runTokenParser :: [TokenParser] -> BSL.ByteString -> TokenStream }

This comment has been minimized.

@phadej

phadej Jul 14, 2017

Author Contributor

turning [TokenParser] into strict list (both elements and spine) doesn't seem to have effect on performance.

@Lysxia Lysxia referenced this pull request Nov 5, 2017

Closed

Object is unordered #368

@awalterschulze

This comment has been minimized.

Copy link

awalterschulze commented Nov 5, 2017

I would like to use this order preserving decoder in aeson or as a separate library for my encoding agnostic validation language and use TokenStream to create an instance of Tree.

data Label
    = String Text
    | Number Scientific
    | Bool Bool

class Tree a where
    getLabel :: a -> Label
    getChildren :: a -> [a]

But maybe this isn't the right place for this comment. I just want to say that I am looking forward to using this.

@awalterschulze awalterschulze referenced this pull request Nov 5, 2017

Open

Consider aeson #6

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment