fast-serialization-of-numpy-arrays-with-bloscpack.json

{
  "alias": "video/3060/fast-serialization-of-numpy-arrays-with-bloscpack",
  "category": "PyData Berlin 2014",
  "copyright_text": "http://creativecommons.org/licenses/by/3.0/",
  "description": "Bloscpack [1] is a reference implementation and file-format for fast\nserialization of numerical data. It features lightweight, chunked and\ncompressed storage, based on the extremely fast Blosc [2] metacodec and\nsupports serialization of Numpy arrays out-of-the-box. Recently, Blosc\n-- being the metacodec that it is -- has received support for using the\npopular and widely used Snappy [3], LZ4 [4], and ZLib [5] codecs, and\nso, now Bloscpack supports serializing Numpy arrays easily with those\ncodecs! In this talk I will present recent benchmarks of Bloscpack\nperformance on a variety of artificial and real-world datasets with a\nspecial focus on the newly available codecs. In these benchmarks I will\ncompare Bloscpack, both performance and usability wise, to alternatives\nsuch as Numpy's native offerings (NPZ and NPY), HDF5/PyTables [6], and\nif time permits, to novel bleeding edge solutions. Lastly I will argue\nthat compressed and chunked storage format such as Bloscpack can be and\nsomewhat already is a useful substrate on which to build more powerful\napplications such as online analytical processing engines and\ndistributed computing frameworks. [1]:\nhttps://github.com/Blosc/bloscpack [2]:\nhttps://github.com/Blosc/c-blosc/ [3]: http://code.google.com/p/snappy/\n[4]: http://code.google.com/p/lz4/ [5]: http://www.zlib.net/ [6]:\nhttp://www.pytables.org/moin\n",
  "duration": null,
  "id": 3060,
  "language": "eng",
  "quality_notes": "",
  "recorded": "2014-07-27",
  "related_urls": [
    "http://code.google.com/p/lz4/",
    "http://code.google.com/p/snappy/",
    "http://www.pytables.org/moin",
    "http://www.zlib.net/",
    "https://github.com/Blosc/bloscpack",
    "https://github.com/Blosc/c-blosc/"
  ],
  "slug": "fast-serialization-of-numpy-arrays-with-bloscpack",
  "speakers": [
    "Valentin Haenel"
  ],
  "summary": "",
  "tags": [],
  "thumbnail_url": "https://i.ytimg.com/vi/TZdqeEd7iTM/hqdefault.jpg",
  "title": "Fast Serialization of Numpy Arrays with Bloscpack",
  "videos": [
    {
      "type": "mp4",
      "url": "http://video.ep14.c3voc.de/20226.mp4"
    },
    {
      "length": 0,
      "type": "youtube",
      "url": "https://www.youtube.com/watch?v=TZdqeEd7iTM"
    }
  ]
}