Skip to content
This repository
branch: master

Jul 11, 2014

  1. Damian Gryski

    Merge pull request #61 from oalders/master

    Fixes typos in Sereal::Encoder Pod.
    dgryski authored
  2. Olaf Alders

    Fixes typos in Sereal::Encoder Pod.

    oalders authored

Jul 10, 2014

  1. Maxim Vuets

    Go: Document the default ZlibCompressor.Level value

    mvuets authored
  2. Maxim Vuets

    Go: Remove ZlibNoCompression const

    mvuets authored

Jul 07, 2014

  1. Ivan Kruglov

    merge.go: fix data race

    ikruglov authored
  2. Ivan Kruglov

    update description of tag MANY in sereal_spec.pod

    ikruglov authored

Jul 06, 2014

  1. Ivan Kruglov

    merge.go: short description

    Sereal's merger is capabale of merging two Sereal documents
    into a single one without deserialization/serialization
    cycle.
    
    Available options:
    
    TopLevelElement
    TopLevelElement allows a user to choose what container will be used at top level.
    
    DedupeStrings
    If enabled, merger will deduplicate all strings it meets.
    Otherwise, only hash key and class names will be deduplicated
    
    KeepFlat bool
    If enabled, KeepFlat keeps flat structture of final document.
    Specifically, consider two arrays [A,B,C] and [D,E,F]:
    - when KeepFlat == false, the result of merging is [[A,B,C],[D,E,F]]
    - when KeepFlat == true, the result is [A,B,C,D,E,F]
      This mode is relevant only to top level elements
    
    The work is not done yet, but the tool is good enough to
    be available to wider audience.
    
    TODO:
    * write tests
    * add header support
    * memory optimizations
    ikruglov authored
  2. Ivan Kruglov

    go: setup ZlibDefaultCompression as default compression for zlib

    ikruglov authored
  3. Ivan Kruglov

    merge.go: some code cleanup

    ikruglov authored
  4. Ivan Kruglov

    merge.go: implement DedupStrings feature

    If enabled, merger will deduplicate all strings it meets.
    Otherwise, only hash key and class names will be deduplicated
    ikruglov authored
  5. Ivan Kruglov

    merge.go: merger's output can be compressed by snappy or zlib

    ikruglov authored
  6. Ivan Kruglov

    merge.go: fix broken tests

    ikruglov authored
  7. Ivan Kruglov

    merge.go: implement KeepFlat feature

    If enabled, KeepFlat keeps flat structture of final document.
    Specifically, consider two arrays [A,B,C] and [D,E,F]:
    - when KeepFlat == false, the result of merging is [[A,B,C],[D,E,F]]
    - when KeepFlat == true, the result is [A,B,C,D,E,F]
      This mode is relevant only to top level elements
    ikruglov authored
  8. Ivan Kruglov

    merge.go: had to return back to usage of append() instead of copy()

    Basically, my assumption was that I can preallocate space in merger's
    buffer by size of appending item and safely work with the memory knowing
    that I have enough space. My validation was that when I merge a document
    I always copy less or equal amount of memory (due to string deduplication).
    In other words, merger's buffer grows slower then the document is processed.
    
    This's wrong assumption. The issue is varints:
    * consider merger's buffer which contain string "TEST" (A)
    * consider a Sereal document with string "TEST" (B) and a copytag (C) refering to
    * so, now, when I merge the document, B will be replaced by new copytag
      refering to A. Also, copytag C will refer to A as well
    * if merger's buffer is big enough, new varint representation of C's
      offset can be bigger then the original one
    
    P.S. At some point in the past I switched from append() to copy()
    because benchmarks showed that copy() is faster by ~8%
    ikruglov authored
  9. Ivan Kruglov

    merge.go: add support of all tags

    ikruglov authored
  10. Ivan Kruglov

    merge.go: never try to dedup short strings (0 or 1 char long)

    ikruglov authored
  11. Ivan Kruglov

    merge.go: add sanity checks

    ikruglov authored
  12. Ivan Kruglov

    merge.go: fix buf with overwriting PAD tags

    ikruglov authored
  13. Ivan Kruglov

    merge.go: can read compressed documents

    ikruglov authored
  14. Ivan Kruglov

    merge.go: can create v2 and v3 documents

    ikruglov authored
  15. Ivan Kruglov

    merge.go: more optimization led to ~8% speedup

    * preallocate memory
    * assign 1-byte values directly
    * call copy() instead of append() to copy bigger chunks
    ikruglov authored
  16. Ivan Kruglov

    merge.go: set default version to v2 (some issue with v3)

    ikruglov authored
  17. Ivan Kruglov

    merge.go: unroll recursion

    ikruglov authored
  18. Ivan Kruglov

    merge.go: optimize string creation

    ikruglov authored
  19. Ivan Kruglov

    merge.go: preallocate memory (not sure about impact)

    ikruglov authored
  20. Ivan Kruglov

    merge.go: improve benchmark

    ikruglov authored
  21. Ivan Kruglov

    merge.go: fix couple mistakes

    ikruglov authored
  22. Ivan Kruglov

    merge.go: remove needless initialization

    ikruglov authored
  23. Ivan Kruglov

    merge.go: add support of different Sereal versions

    ikruglov authored
  24. Ivan Kruglov

    merge.go: let user decide about top level element (TopLevelArray|TopL…

    …evelArrayRef)
    ikruglov authored
  25. Ivan Kruglov

    merge.go: cosmetic changes + better names

    ikruglov authored
  26. Ivan Kruglov

    merge.go: optimize access to track hash table

    ikruglov authored
  27. Ivan Kruglov

    merger_test: go fmt knows better how to format code

    ikruglov authored
  28. Ivan Kruglov

    merger_test: simple benchmark for profiling

    ikruglov authored ikruglov committed
  29. Ivan Kruglov

    first workable code (still not all tags supported)

    ikruglov authored ikruglov committed
Something went wrong with that request. Please try again.