Skip to content
Postgres extension for parsing json using simdjson library
C++
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
expected
sql
.gitignore
LICENSE
Makefile
README.md
pg_simdjson--0.1.sql
pg_simdjson.control
pg_simdjson.cpp
pg_simdjson.h
pg_simdjson.png
simdjson.cpp
simdjson.h

README.md

pg_simdjson

A prototype for an extension, that generates jsonb via simdjson library.

=# select simdjson_parse('{"a": {"b": 1, "c": 2, "d": "value"}}');
            simdjson_parse
---------------------------------------
 {"a": {"b": 1, "c": 2, "d": "value"}}
 (1 row)

All the tests from the original JSONB implementation are passing, but there is an obvious difference in reported error messages.

Here are results of parsing json with native PostgreSQL implementation and simdjson. Test was performed in the benchmark environment on a pinned cpu without hyperthreading and intel turbo with a dataset containing from 1 to 200 documents, each document is about 1.6 kB. Plotted graph here represents parsing query latency, i.e. the whole processing time that includes not only json parsing, but also everything to actually perform a query.

benchmark

The idea behind this is to see how this extension behaves in a relatively real situation. Actual json parsing part with simdjson takes just a small fraction of time, e.g. for 200 documents it's between 256 – 511 microseconds. This corresponds to approximately 1.3 GB/s and somehow close to the numbers, reported in the simdjson repository.

You can’t perform that action at this time.