Making Postgres and Elasticsearch work together like it's 2019
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github update template Aug 13, 2018
build update copyright per issue #333 Feb 3, 2019
src update copyright per issue #333 Feb 3, 2019
.gitignore reduce overhead of generating ZDB's MVCC visibility query clause by d… Jan 26, 2019
.travis.yml travis<-->slack Jan 26, 2019
AGGREGATIONS.md Pusing a new ZomboDB that supports Postgres 10, doesn't require an El… Aug 13, 2018
CAT-API.md doc update Aug 27, 2018
CMakeLists.txt Issue 296 (#310) Nov 13, 2018
CONFIGURATION-SETTINGS.md Resolve Issue 273 (#292) Oct 11, 2018
CREATE-INDEX.md Issue 296 (#310) Nov 13, 2018
INDEX-MANAGEMENT.md Pusing a new ZomboDB that supports Postgres 10, doesn't require an El… Aug 13, 2018
INSTALL.md Fix typo (#318) Dec 6, 2018
LICENSE.md update copyright per issue #333 Feb 3, 2019
LLAPI.md Pusing a new ZomboDB that supports Postgres 10, doesn't require an El… Aug 13, 2018
Makefile update copyright per issue #333 Feb 3, 2019
QUERY-DSL.md working on issue #324 (#325) Jan 3, 2019
README.md update copyright per issue #333 Feb 3, 2019
SCORING-HIGHLIGHTING.md Issue 296 (#310) Nov 13, 2018
SQL-FUNCTIONS.md Pusing a new ZomboDB that supports Postgres 10, doesn't require an El… Aug 13, 2018
TABLE-SAMPLERS.md Pusing a new ZomboDB that supports Postgres 10, doesn't require an El… Aug 13, 2018
THINGS-TO-KNOW.md Issue 296 (#310) Nov 13, 2018
TUTORIAL-data.dmp fix issue #275: restore tutorial data Aug 14, 2018
TUTORIAL.md fix PG version Aug 14, 2018
TYPE-MAPPING.md working on issue #324 (#325) Jan 3, 2019
VACUUM.md Pusing a new ZomboDB that supports Postgres 10, doesn't require an El… Aug 13, 2018
logo.png Pusing a new ZomboDB that supports Postgres 10, doesn't require an El… Aug 13, 2018
update-versions.sh Pusing a new ZomboDB that supports Postgres 10, doesn't require an El… Aug 13, 2018
zombodb.control another issue #333 change Feb 3, 2019

README.md

logo

Making Postgres and Elasticsearch work together like it's 2019

Build Status

ZomboDB brings powerful text-search and analytics features to Postgres by using Elasticsearch as an index type. Its comprehensive query language and SQL functions enable new and creative ways to query your relational data.

From a technical perspective, ZomboDB is a 100% native Postgres extension that implements Postgres' Index Access Method API. As a native Postgres index type, ZomboDB allows you to CREATE INDEX ... USING zombodb on your existing Postgres tables. At that point, ZomboDB takes over and fully manages the remote Elasticsearch index and guarantees transactionally-correct text-search query results.

ZomboDB is fully compatible with all of Postgres' query plan types and most SQL commands such as CREATE INDEX, COPY, INSERT, UPDATE, DELETE, SELECT, ALTER, DROP, REINDEX, (auto)VACUUM, etc.

It doesn’t matter if you’re using an Elasticsearch cloud provider or managing your own cluster -- ZomboDB communicates with Elasticsearch via its RESTful APIs so you’re covered either way.

ZomboDB allows you to use the power and scalability of Elasticsearch directly from Postgres. You don’t have to manage transactions between Postgres and Elasticsearch, asynchronous indexing pipelines, complex reindexing processes, or multiple data-access code paths -- ZomboDB does it all for you.

Quick Links

Features

Current Limitations

  • Only one ZomboDB index per table
  • ZomboDB indexes with predicates (ie, partial indexes) are not supported
  • CREATE INDEX CONCURRENTLY is not supported

These limitations may be addressed in future versions of ZomboDB.

System Requirements

Product Version
Postgres 10.x
Elasticsearch 5.6.x, 6.x
libcurl >=7.28.0

Downloading

Please visit https://www.zombodb.com/releases/ to download.

If you want to integrate with a CI/CD system you can intuit the pattern for versions from the Postgres extension download links, but it'll be something like:

https://www.zombodb.com/releases/VERSION/zombodb_trusty_pg10-VERSION_amd64.deb

Note: you'll need to use the one that's for your Postgres + Linux distro combination -- the example above is for Postgres 10 on Ubuntu Trusty.

Quick Overview

Note that this is just a quick overview. Please read the getting started tutorial for more details.

Create the extension:

CREATE EXTENSION zombodb;

Create a table:

CREATE TABLE products (
    id SERIAL8 NOT NULL PRIMARY KEY,
    name text NOT NULL,
    keywords varchar(64)[],
    short_summary text,
    long_description zdb.fulltext, 
    price bigint,
    inventory_count integer,
    discontinued boolean default false,
    availability_date date
);

-- insert some data

Create a ZomboDB index:

CREATE INDEX idxproducts 
          ON products 
       USING zombodb ((products.*)) 
        WITH (url='localhost:9200/');

Query it:

SELECT * 
  FROM products 
 WHERE products ==> '(keywords:(sports OR box) OR long_description:"wooden away"~5) AND price:[1000 TO 20000]';

Contact Information

Older Versions

This version of ZomboDB only supports Postgres 10 and doesn't require an Elasticsearch plugin.

Previous versions that support Postgres <=9.5, and require an Elasticsearch plugin, are still available, from these branches:

Commerical support for all versions of ZomboDB is available by contacting ZomboDB, LLC.

History

The name is an homage to zombo.com and its long history of continuous self-affirmation.

This version of ZomboDB was graciously sponsored by Impira.

Historically, ZomboDB began in 2013 by Technology Concepts & Design, Inc as a closed-source effort to provide transaction safe text-search on top of Postgres tables. While Postgres' "tsearch" features are useful, they're not necessarily adequate for 200 column-wide tables with 100M rows, each containing large text content.

Initially designed on-top of Postgres' Foreign Data Wrapper API, ZomboDB quickly evolved into an index type so that queries are MVCC-safe and standard SQL can be used to query and manage indices.

Elasticsearch was chosen as the backing search index because of its horizontal scaling abilities, performance, and general ease of use.

ZomboDB was open-sourced in July 2015 and has since been used in numerous production systems of various sizes and complexity.

License

Copyright 2018-2019 ZomboDB, LLC

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.