@dennwc dennwc released this Jun 3, 2018 · 3 commits to master since this release

This release fixes few bugs and adds label support for the schema lib.


  • Schema lib now allows adding a label to all written quads (1a0c7cb)
  • Fixed a bug with KV backend that allowed to write duplicate quads in the same TX (#675)
  • All() query for nodes now honors a query limit (#718). Previous behavior can be achieved by passing ?limit=-1 parameter.

As always, we expect Go users to pin a specific Cayley version tag with dep.


on Github

dennwc: graphql: update dependency and test filter with multiple values
dennwc: schema: allow to specify quads label on write
dennwc: bolt: do not insert duplicate quads found in the same tx; fixes #675
dennwc: set a correct limit in integration tests
dennwc: cli: assume stdout as an output when no arguments are specified
dennwc: gizmo: All should honor a query limit; fixes #718
robfig: cayley-single.yml: update version tag to latest


@dennwc dennwc released this Apr 23, 2018 · 10 commits to master since this release

This release brings Elasticsearch v6.x support, a transition to dep and few bug fixes.


  • Cayley now supports ElasticSearch v6.x (#711)
  • SaveOptional is exposed in Gizmo (#702)
  • Transition to dep for dependency management (#704, #703)
  • Bug fixes (#710, #709, #708, #707)

As always, we expect Go users to pin a specific Cayley version tag with dep.


on Github

dennwc: quad: do not un-quote strings in StringToValue; fixes #710
dennwc: add CODEOWNERS
dennwc: elastic: support v6; fix #711
skydan: Fix dep ensure and downloading with "follow redirects" option
mguentner: hello_bolt: replace TempFile with TempDir
mguentner: hello_schema: replace TempFile with TempDir
dennwc: gizmo: expose SaveOptional; resolves #702
dennwc: build and test for go 1.10
dennwc: switch from glide to dep
glaslos: replace badge png with scaling svg


@dennwc dennwc released this Mar 5, 2018 · 22 commits to master since this release

This minor release brings few bug fixes, faster CockroachDB backend, configurable schema package and new GraphQL features.

It will be the last feature release before switching to v0.8 branch. Only bug fixes will be backported to v0.7 moving forward.


  • Faster CockroachDB transactions (@dsymonds).
  • Support for un-nesting objects and expand-all for GraphQL (#686, #692).
  • Configurable schema package (#694).
  • Fixes for json-ld, graphviz data formats (@jtorvald).
  • Fixes for Recursive iterator and optimization of And.

As always, we expect Go users to pin a specific Cayley version tag with a dependency manager.


dennwc: schema: allow to customize behavior; resolves #694
dennwc: graphql: support un-nesting objects into parent; resolves #686
dsymonds: Switch CockroachDB driver from lib/pq to jackc/pgx.
dennwc: graphql: support expand all; resolves #692
dennwc: properly close sub-iterator in recursive; fix #693
dsymonds: Rewrite CockroachDB SQL construction to be much faster.
dsymonds: Bump cockroachdb Docker tag for testing to the latest version, v1.1.5.
dennwc: iterator: do not add empty iterator to And on failed optimization
jtorvald: jsonld: added a DocumentLoader to NewReaderFromMap; fix NPE (#689)
dennwc: dot: correctly escape slashes; additional file extension

v0.7.1 -- Wildcards, CouchDB, fixes and performance

@dennwc dennwc released this Jan 30, 2018 · 35 commits to master since this release

This minor release brings multiple bug fixes, query performance improvements, wildcards query and new CouchDB backend support.

As noted in v0.7.0 release, support for legacy bolt1, leveldb1, mongo1 backends was dropped. New backend implementation are still available under bolt, leveldb and mongo names.


As always, we expect Go users to pin a specific Cayley version tag with a dependency manager.


dennwc: nosql: iterator size should not be negative in case of error
dennwc: dock: expose container port on darwin properly
dennwc: nosql: if store has only 32 bits, store a copy of int field
dennwc: graphtest: test max int roundtrips and comparisons
elliott5: ouch: new couchdb/pouchdb backend
dennwc: cli: fix database close on init
dennwc: iterator: correctly limit depth for recursive; fix #685
dennwc: gizmo: always initialize context; fixes #674
dennwc: graph: unwrap implementation to enable optional interfaces
dennwc: sql: copy tags when cloning an iterator
dennwc: allow to specify quad file address for non-persistent backends; fix #677
dennwc: iterator: correctly clone error iterator and make a distinct type for it
dennwc: gizmo: clarify Count docs
jtorvald: All negative values should result in unlimited depth (#679)
jtorvald: Javascript error from sigma was not visible
dennwc: path: test comparisons with wildcards
dennwc: nosql: support regexp and wildcards
dennwc: sql: do better job optimizing filters
dennwc: sql: optimize regular expressions
dennwc: shape,gizmo: support wildcard value filter
dennwc: iterator: get some rough size estimates for comparisons
dennwc: iterator: calculate real size for LinksTo(Fixed)
dennwc: all: add benchmarks
dennwc: kv: fix typo in bloom filter init; fix #675
dennwc: cli: allow to pass file name as argument for load and dump
dennwc: nosql: do not process invalid deltas
dennwc: sql: do not decrement nodes from non-existent quads; return delta errors
dennwc: kv: fix decrement of node refs from non-existent quads
dennwc: test all writer parameters for each backend
dennwc: remove test flag for backends without nodes gc
dennwc: bump version to 0.7.1 and drop legacy backends
gauravtiwari: Adds locations for looking up static assets (#673)

v0.7.0 -- Meta backends, ElasticSearch, Shapes, and new CLI

@dennwc dennwc released this Dec 24, 2017 · 76 commits to master since this release

Release covers nearly 8 months of work, mostly on implementing meta-backends, new query optimizer, performance improvements and bug fixes.

Note: From now on we plan to release point releases on regular basis to push fixes and new features faster.

Storage format for backends received an upgrade bringing backward incompatible changes. Check migration guide for more details. It only affects the database format, HTTP API remains backward-compatible.

This release defines new internal interfaces (meta-backends) for working with various database kinds. Currently three of them are defined: KV, NoSQL, SQL. Each of them significantly simplifies a process of implementing a new backend, as well as allows us to lower maintenance overhead by sharing most optimizations, read and write paths, etc. This also means that optimization work on one backend will be automatically applied to other backends of the same kind in the future.

Part of this effort was an implementation of new query representation that we call "shapes". All optimizations like query rewrites and reordering are now done on a generic level, without any backend-specific code. It will allow us to save query stats, make more complex optimizations and even store query templates in the database, or split it to be executed in distributed manner by multiple backends. Since new representation can be manipulated more easily, most backends will now rewrite complex query types into their native query language (SQL, for example).


  • ElasticSearch support
  • New query optimizer (aka query shapes)
  • New command line interface
  • KV meta-backend + port of Bolt and LevelDB to it
  • NoSQL meta-backend, + port of Mongo and ElasticSearch to it
  • "Raw" value type is deprecated
  • Exposed node delete API for HTTP
  • Consistent behavior for all backends on node delete
  • New query types in Gizmo: Labels, SaveInPredicates, SaveOutPredicates, see docs
  • Has traversal now supports value filters
  • FollowRecursive can be limited to a maximal depth
  • GraphQL now supports quad labels
  • SQL query optimizer improvements
  • SQL backends now has their own names: postgres, mysql, cockroach (no need to specify flavor)
  • Significant performance improvements
  • Smaller Docker image
  • Minor UI improvements

Breaking changes

  • Bolt, LevelDB and Mongo backends were rewritten and database format has changed. Old implementations are still available under bolt1, leveldb1 and mongo1 names. Old backends will be removed in next minor release. c266392
  • SQL table structure has changed 31b0f48, #617
  • Cayley now ignores duplicated quads by default 9cb0eef
  • Gremlin is deprecated and dropped in favor of Gizmo #519
  • Count in gizmo is now a 'final' instead of traversal and returns result #576

Breaking changes (Go)

As always, we expect users to pin a specific Cayley version tag with a dependency manager and sync with following changes in the code:

  • Cayley uses Go version 1.9+
  • Bolt, LevelDB and Mongo backends were rewritten and database format has changed. Old implementations are still available under old paths, but will be dropped in next minor release. Users are encouraged to update imports to kv/bolt, kv/leveldb and nosql/mongo.
  • SQL flavors moved to their own packages and needs to be imported separately (sql/postgres for example). sql backend name will be deprecated in next major release. 48e7946
  • "Raw" value type (quad.Raw and quad.MakeRaw) is deprecated and replaced with a helper to preserve at least some compatibility. It's strongly recommended to switch to quad.String or IRI and quad.Make. 41bf496
  • Iterator optimizations for backends are mostly disabled. It is recommended to build queries with new graph/shape package and run optimizations on it.
  • Removed unused methods from QuadStore: Horizon, FixedIterator, Type d994e2a, b63faea, #631
  • Removed graph.PrimaryKey type d994e2a
  • graph.Value is now type-safe e4ab526
  • Removed ID and Timestamp fields from graph.Delta - they will be assigned by server 524c8e2
  • RemoveNode type changed to quad.Value to match other methods #607
  • Quad writers produced by graph.NewWriter and NewRemover now requires Flush to be called a420e62
  • Iterator now accepts context for Next, NextPath and Contains f3e814f
  • Describe method was removed from Iterator and replaced by String method on iterator and graph.DescribeIterator b491f41
  • Iterator type is now a string instead of enum and registration is not necessary anymore


dennwc: http: expose node delete api; resolves #622
dennwc: deprecate quad.Raw type
dennwc: code-wide cleanup
dennwc: iterator: remove unused equality function for fixed
dennwc: graph: remove unused horizon function
dennwc: memstore: use correct size for all iterator
dennwc: add context argument to iterators and kv
dennwc: kv: track nodes reference count; fixes #617
dennwc: sql: track nodes reference count and simplify dialects; fixes #617
dennwc: memstore: remove nodes if all their references are gone; addresses #617
dennwc: iterator: returns correct subiterator slice for empty And; fixes #659
dennwc: nosql: use base64 instead of hex encoding for hashes
dennwc: update Go version to 1.9
dennwc: update integration test, make it a part of suite; move paths as well
dennwc: fix tests for predicate save query; fix node tokens for bolt
dennwc: nosql: use shapes optimizer for queries
dennwc: shape: allow to simplify shapes; add helpers for building node filters
dennwc: elastic: port to nosql layer; resolves #653
dennwc: nosql: store each typed value in separate field; support batch inserts
dennwc: separate mongo logic and define a generic nosql layer; resolves #652
dennwc: iterator: replace Describe method with a helper
dennwc: graphql: support @label directive; resolves #614
dennwc: path/gizmo: support Has with filters; resolves #650
dennwc: bolt,leveldb: swap names to a new kv backend implementations
michaelqiu94: ElasticSearch as Supported Backend (#634)
dennwc: path,gizmo: allow to save predicates without moving from nodes; fix #591
dennwc: http: allow to serve API without web files; fix #503
dennwc: ui: format json results; resolve #454
dennwc: sql: optimize intersection, value lookup and comparison
dennwc: sql: fix params propagation from sub-queries
dennwc: sql: rewrite query builder to shapes; fix #563
dennwc: shape: optimize multiple Pages; push Fixed values from Intersect into NodesFrom
dennwc: propagate sub-iterator errors from And properly
dennwc: stream ops in node delete; return error if node is missing; fix #608
dennwc: add a single package that imports all supported backends
derekrliang: Add optional max recursion depth to FollowRecursive (#639)
derekrliang: Fix query shape error when query has no result data links (#641)
derekrliang: Add running local unit test documentation (#640)
dennwc: bolt,leveldb: port to kv layer
dennwc: kv: add in-memory implementation
dennwc: integration: open and close db for each test set; use table benchmarks
dennwc: kv: support for index scans and indexes on multiple directions
dennwc: kv: allow to customize indexes
dennwc: kv: batch get requests and simplify bucket interface
Yannic: Reduce Docker image size (#637)
dennwc: sql: separate packages for flavors (#633)
Yannic: Remove unused Type function (#631)
dennwc: docs: add http api definitions (swagger v3)
dennwc: iterator: remove duplicated TagResults code and  type registration
dennwc: github: simplify issue template header, add slack link
dennwc: tag values in recursive iterator correctly
dennwc: shape: new query optimizer
dennwc: memstore: iterator optimizations
dennwc: graph: rewrite pk; remove horizon and ts from delta; rewrite memstore
dennwc: kv: registration, scan by prefix, support for flat kv
dennwc: integration: allow to load data to remote backends
dennwc: gizmo: fix #620 and bump goja version
dennwc: graph: helper for writing quads to tx (useful for schema)
dennwc: writer: add a direct constructor for single replication
Jason Kingsbury: make bloom filter buffer before initialisation
dennwc: cli: read query timeout from config file; fix #613
dennwc: graphql: allow full IRI charset in names; resolve #616
dennwc: schema: allow to limit depth; fix #609
joostverdoorn: Set CORS headers on gephi stream
dennwc: http: allow to specify query limit; fix #612
joostverdoorn: Add labels morphism and Gizmo API (#610)
dennwc: http: do not add .md extension if it's already present; fix #597
dennwc: schema: return an error on write if required field is not set; fix #602
dennwc: accept quad.Value instead of graph.Value for RemoveNode; fix #607
dennwc: print warning on set verbosity for glog; fix #599
dennwc: flush writer in schema example; fix #606
harlantwood: Remove "angry highlighting" from JSON examples
dennwc: generic kv backend based on bolt
h4ck3rm1k3: query from stdin (#601)
chrispassas: Fix example startup command
dennwc: make graph.Value safe again
h4ck3rm1k3: adding help to repl
dennwc: schema: handle id in anonymous struct fields
h4ck3rm1k3: Update sidebar.tmpl (#595)
dennwc: schema: allow to load objects from path
dennwc: cli: implement query command
barakmich: conversion check
h4ck3rm1k3: Update transaction.go
dennwc: graph: batch quads automatically if caller uses single AddQuad on writer
gkontos: gae: fix add of previously deleted quads (#580)
MarioAriasGa: Glog issues (#579)
barakmich: bolt2: use bloom filter to speed insertion
barakmich: bolt2: working queries
barakmich: graph: Add primitive proto
barakmich: Fix glide story; ignore docker deps, `glide.lock` is now sane
dennwc: gizmo: make Count a final instead of traversal
dennwc: fix contains on recursive iterator
dennwc: check for nil result from NameOf; fix #574
dennwc: ignore duplicate quads by default
dennwc: docs: add example configs for kubernetes; resolves #327
dennwc: do not load test dataset in container
dennwc: golang client for data import/export; add tests for v2 api
dennwc: http: support queries in api v2; expose api v2 as a library
dennwc: http: get rid of custom handler type
dennwc: http: use optional interface to create qs for request
dennwc: bind to localhost by default
owulveryck: The -host option was missing the port argument
mikaelcabot: cockroach: schema changes within transactions are restricted (#567)
dennwc: gizmo: generate docs from Go function comments; resolve #520 (#560)
dennwc: cli: register glog flags; make conv an alias for convert; resolve #561
oren: Update README.md - add MySQL and CockroachDB
dennwc: deprecate gremlin in favor of gizmo; resolve #519
dennwc: cli: deduplication tool for nodes with same properties
dennwc: cayley: switch to cobra/viper cli

v0.6.1 -- GraphQL, Gizmo, SQL flavors, Recursive and Gephi

@dennwc dennwc released this Apr 12, 2017 · 268 commits to master since this release

This release marks community build number three. It includes all the pending features that can be merged before v0.7, which will be a major release to introduce Reification, and may cause breaking changes.

The storage format was not changed for this release. Existing databases should work properly without any migration needed.

Cayley now supports multiple SQL flavors, including MySQL and CockroachDB.

To avoid further confusion, our Gremlin-style query language was renamed to Gizmo to emphasize that it's not fully Gremlin-compliant. We also switched to the Goja JavaScript VM for better performance and ES5.1 support. Error handling was also improved for our query language.

This release introduces an experimental GraphQL implementation. Note that it only shares a similar syntax and does not support schema yet. It's a preview feature, so the implementation or API may change in following releases.

The import and export formatter system is now modular and can be easily extended. Cayley now supports JSON-LD, Graphviz (aka DOT; export only) and custom binary formats for better import speed. The HTTP API was updated (v2) to support the new formatter system and allows for the streaming of large amounts of data in single request.

The recursive iterator is finally available via the Go path library and Gizmo. It allows traversal of a path recursively to potentially unlimited depth.

A new Schema library was implemented to allow developers to save and load complex Go objects with a single method call.


  • Support for MySQL and CockroachDB.
  • Gizmo query language with faster ES5.1-complaint VM (supersedes Gremlin).
  • Experimental GraphQL-like query language.
  • Modular import/export formats system.
  • Support for JSON-LD, Graphviz (DOT) and Cayley-specific binary format.
  • HTTP API v2 with better import/export performance for large data files.
  • Recursive iterator support (FollowRecursive).
  • Schema lib (aka ORM or Object Persistence Layer).
  • IRI prefix registration.

Full changelog:

dennwc: gizmo: expose recursive traversal and update docs
dennwc: path: FollowRecursive traverses quad.Values
barakmich: add FollowRecursive to the path lib
barakmich: graph/iterator: Add Recursive iterator
dennwc: graphql: close iterator properly; fixes #545
dennwc: tests: require.NoError instead of Nil
dennwc: gae: fix nop iterator optimization
dennwc: sql: append WHERE only if constraints are present
dennwc: http: CORS, API to get supported formats and RO mode fix
dennwc: quads: add json stream format
dennwc: mql: fix NPE
dennwc: gephi: allow to filter quads in raw mode
dennwc: gephi: inline primitive values automatically; inline schema:url
dennwc: gephi: initial support for graph stream
jtorvald: Updated documentation based on Issue #349 (#541)
caosiyang: docs: fix a typo in Quickstart-As-Application.md (#542)
jtorvald: Issue #286 init db and config documentation (#538)
Yannic: graph: cleanup and more robust options handling (#536)
dennwc: initial graphql support
dennwc: cli: fix loading of data for non-persistent backend
jtorvald: docs: replace broken MQL link, add quick-start cli usage example (#535)
mikaelcabot: CockroachDB transaction retries (#534)
dennwc: bolt: make bolt concurrent-safe as it should be
dennwc: gizmo/gremlin: fix tags for Both traversal; fix #532
dennwc: fix #530 and write errors as plain json
dennwc: quad: support for graphviz format
dennwc: memstore: close iterator properly
dennwc: memstore: do not hash values in QuadDirection
dennwc: memstore: no recursion in Next
dennwc: http: use formats - implement write api v2; resolves #176
dennwc: mongo: allow to pass full urls; fix #526
dennwc: add missing build tags; find relative path of data files automatically
dennwc: sql: support flavors: mysql, cockroachdb
dennwc: mongo: remove nodes with no references; fix #517
dennwc: gizmo: allow to register custom IRI prefixes; resolves #505
barakmich: revert proto GetNativeValue and check for crashing nils on describe
barakmich: Fill literal strings in MQL the same way we do results in Gremlin
dennwc: gremlin: fix limit in ForEach; fixes #518
barakmich: Fix non-persistent load, modernize MQL test
barakmich: Fix -v option and debug error
barakmich: Tidy up Optional iterator, and allow MQL to support JS-like string syntax
dennwc: switch to goja (gizmo)
barakmich: Fix dependencies, make Docker an optional dependency, add build tags for docker and appengine tests
mzats: Updated examples so that they work in repl/http. (#499)
barakmich: Remove IsNode() from the graph.Value definition.
dennwc: schema: move well-known types package
dennwc: graph: refactor interfaces
dennwc: add example for schema library
dennwc: ported schema lib
dennwc: use vocabulary constants for value types, allow to shorten and expand IRIs
dennwc: register known rdf vocabularies, add constants
joostverdoorn: Add missing gojsonld dependency to glide (#502)
dennwc: bolt/leveldb: stop complaining about existing quads even if ignore is set
dennwc: http/repl: handle errors correctly
dennwc: nquads: remove duplicated code
dennwc: integrate quad formats, replace exporter, use quad helpers in tests
dennwc: quad: readers, writers and formats.
dennwc: Update minimal Go version
dwhitena: Ability to pass the Mongo Backend an externally managed session. (#486)
pbsladek: Update quickstart for 0.6.0 (#487)
dennwc: add github issue template
dennwc: sql: fix sqlNodeIterator size
dennwc: fix count iterator
dennwc: path: test both optimized and unoptimized case; few more test for gremlin
kaneshin: Fix minor typos (#483)
barakmich: Add arm64 support

v0.6.0 -- Types

@barakmich barakmich released this Oct 7, 2016 · 379 commits to master since this release

Current release marks community build number two, introducing major change to data layout and bringing new features to Gremlin.

Storage format changed slightly. We highly recommend to do a clean import of the database with a new binary.

Cayley now honors types of nquad values. IRIs (<name>) were converted to untyped strings (name), so it was possible for nodes with different types to be loaded as a single node. Now Cayley preserves type information, making these nodes separate. Queries that used IRIs without <> quotes will break because of this change. Gremlin now requires IRI it to be in "<follows>" form and the path lib expect these values to be of quad.IRI Go type. The same stands for Strings (aka RDF literals) and BNodes (aka blank nodes, _:name).

Data loader now recognizes few basic typed literals like "10"^^<http://schema.org/Integer> and converts them into corresponding native type (Int for this example). It may also affect existing queries. Gremlin will automatically convert basic types into corresponding RDF typed literals.


  • Typed values support for all backends. Includes types: String, IRI, BNode, Int, Float, Bool, Time.
  • New traversals in path lib and Gremlin: HasReverse, Skip, Limit, Count, Unique, Comparison/Filter, Regex.
  • More consistent backends behavior thanks to unified tests.
  • Updated docs and added examples.
  • Updated Docker image, and added example for it.
  • Many, many bugfixes and some optimizations.

Full changelog:

gabrys: Fix link to GremlinAPI.md (#477)
dennwc: expose Unique via path lib and gremlin; implement and expose Count iterator; resolve #240
dennwc: gremlin: add regex support, make it harder to abuse regexps; add docs explaining why
guycook: Implement regex filter as iterator (#474)
dennwc: bolt,leveldb: fix AllQuads iterator not skipping deleted quads, add test
dennwc: iterator: add variadic arguments to Add, Or and Fixed constructors
dennwc: query: Refactor interfaces (#471)
dennwc: fix lru panic
wayeast: Clone receiver paths (#466)
kenshaw: appengine: updating dependency paths and adding documentation (#464)
dennwc: gremlin: handle string slice in via parameter
kenshaw: appengine: add build constraint (#463)
dennwc: Glide migration (#460)
dwhitena: user/pass functionality for mongo backend
allanino: Fix Docker building (#456)
barakmich: gremlin: don't require a session to last as long as the timeout
kris-runzer: graph: fix Is*(err error) checkers to properly compare against *DeltaError, add tests
dennwc: optimize by default in Iterate helper
dennwc: gremlin: clone path objects
neonstalwart: simplify the signature of NextLogOut (manual merge of #264)
dennwc: graph: add iteration helpers, update examples; resolves #446
dennwc: http: fix json quads decoding
dennwc: gremlin: fix ToArray without arguments
dennwc: iterator: promote Next to main interface; make NoNext an optional interface
oren: Example of BoltDB, update examples/README.md
robertmeta: Just better quickstart + examples creation
dennwc: query: check type assertions, recover properly in gremlin; fixes #450
josephschorr: Add Clone() to Path for easier reuse of Path segments (#444)
dennwc: Wire Skip and Limit to Gremlin. Also fixed #254.
dennwc: postgres: store hashes as binary data instead of hex strings
dennwc: postgres: prepare each statement in ApplyDeltas
dennwc: postgres: generate distinct node insert statements only
dennwc: cquads: convert TypedString values to native types automatically on loading
dennwc: gremlin: allow to specify multiple conditions in Filter
dennwc: mongo: optimize comparison iterator
dennwc: Test for typed value comparison and optimization
dennwc: gremlin: expose value comparison iterator via path lib, add tests
dennwc: gremlin: support for typed values
dennwc: gremlin: replace JS-based graph object with reflect-based (rewritten, now it's easier extend)
dennwc: comparison: compare only values with the same type, add a test for mixed set
dennwc: mongo: use native value types where possible, fix indexing, remove recursion in LinksTo, remove unnecessary roundtrip
dennwc: postgres: new table structure, store typed values natively; fix iterators; separate LRU cache
dennwc: Update value comparison iterator. Stop graph.Value madness.
dennwc: cquads: recognize TypedString and LangString in parser
dennwc: Add a test for Raw-to-typed values exchange. Fix memstore not using hashes in map.
dennwc: GAE (untyped). More tests. Fixed handling of size and conflicts in postgres.
dennwc: quad: add support for few basic types (int, float, bool, time)
dennwc: Unify tests for QuadStore. Run Mongo/Postgres in Docker automatically.
dennwc: leveldb: use protobuf instead of json, migration code
dennwc: bolt: migration code
dennwc: add typed fields to protobufs
dennwc: graph,query,exporter,path: updated to use typed values
dennwc: quad: directions are typed
kortschak: graph: provide more meaningful errors for bad actions (#443)
josephschorr: Don't iterate over all quads unless necessary (#440)
kortschak: clog: fix comment typos and add package comment (#442)
dennwc: Implement Skip and Limit iterators (#369)
EntilZha: Delete node API (#158)
dennwc: Updated Dockerfile (fixed #357)
dennwc: fix Unique iterator for non-comparable tokens; fixes #437
kortschak: clog: make nil Logger safe (#436)
robertmeta: Sweep through docs update/cleanup
oren: remove duplicate in readme (#435)
dennwc: fix postgres error on import, fixes #429
dennwc: Added HasReverse function to path lib (#351)
drzippie: Added sync.RWMutex to mongo cache  (#380)
robertmeta: Update docs (README, Locations, Quickstart-As-Lib, Quickstart-As-Application, HACKING, FAQ, Container)
robertmeta Rerouting CONTRIBUTING to the discourse, updating CONTRIBUTORS to remove mention of CLA
robertmeta: Moving Hacking from wiki to project
dennwc: separate log package, resolves #221

v0.5.0 -- SQL, Labels, and Community

@barakmich barakmich released this Jul 3, 2016 · 547 commits to master since this release

We've spun out of the google namespace, so this is community build number one. There's some good chatter in IRC and places, and expect some notes in the mailing list soon. The project isn't dead, it was just resting.

Now that we can add contributors, it's time to tag and build a release as a baseline. There may be some turbulence as some longstanding changes get merged over the next version or two, so before we do that, here are some binaries for people to try out the latest.


  • Postgres/SQL backend support
  • Add support for LabelContext() to query the Label field across Quads
  • SaveOptional()
  • Many, many bugfixes

Full changelog:

barakmich: Add SaveOptional() to save predicates if they exist, and add the
dennwc: path: Added SaveOptional
dennwc: Fix interface conversion in Gremlin
barakmich: fix build on 1.6 -- more accurate interface conversion
josephschorr: Make call to `Sprintf` lazy
barakmich: query/gremlin, graph/path: Add support for querying with labels
Quentin-M: Fix NPE in SQL initialization
jzelinskie: dockerfile: fix go install command
jzelinskie: initial Dockerfile
barakmich: graph: make quadstore init functions idempotent
barakmich: Update README to mention Postgres and remove unused trello board
jzelinskie: add Triple function
Quentin-M: Fix SLOW DELETEs
Quentin-M: Fix a deadlock behavior with DELETE transactions
Quentin-M: Ensure that quads in a transaction are applied in the desired order.
Quentin-M: Use a Size() estimate for SQL Quadstore when noSizes is enabled.
barakmich: sql: Combine AND(Fixed, SQL) and fix Save() roundtrips
barakmich: graph/sql: Use an IN clause when optimizing a fixed iterator
barakmich: query/gremlin: fix building Or iterator, add test
barakmich: query/gremlin: Refactor in terms of graph/path
Quentin-M: Fix NPE in SQL Quadstore
Quentin-M: Improve Transaction: deduplicate quads, allow adding/removing a quad in the same tx
barakmich: Fix corners after causing a couple merges
ds--: Adds exporting capabilities
barakmich: Add migration tool and use protos in Bolt store
barakmich: graph: Postgres backend
sayden: Typo fix when building cayley from scratch
sayden: Order of parameters when using :a and :d are switched
Kavec: Refactor for readability
kortschak: graph/bolt: don't panic on unitialised store
barakmich: Fixes memstore transaction semantics
Quentin-M: Make the SQL connection fail-fast
Quentin-M: Fix NPEs in SQL Next and Contains
barakmich: Roll the Go version forward on Travis builds (new stable)
barakmich: Add check for multiple all iterators
kortschak: internal: move {config,db,http} into internal
skynet: Update README.md
neonstalwart: add string comparison to value comparison iterator
tmlbl: Small REPL Improvements
igm: Fixed multiplier for position of "label" in case of "po" key entry
cayleydb: Update README.md
barakmich: Docs update based on @KamiQuasi #280
lytics: Use new import path for go-uuid
jzelinskie: travis.yml: bump appengine dependency
barakmich: Add transaction (a list of deltas) as a primitive for applying sets o…
nii236: Widened visualisation box to 100%
panamafrancis: Added myself to the A+C
barakmich: Add Save and Has API functions
Quentin-M: Allow opening a database (bolt, leveldb, mongo) through Go API
barakmich: Clean up Linkage definition
allonhadaya: In out predicates
barakmich: Fix bug where first Bolt iteration ignores deletion
barakmich: Fix describe in Or iterator
neonstalwart: collapse 2 iterations into 1 in memstore
op: Fix nil pointer panic when meta bucket is missing
barakmich: Fix LevelDB deleted-triple iterator stack overflow bug
neonstalwart: move cmp to where it's used
kortschak: graph: remove redundant type
barakmich: Go API
heyalexej: update leveldb link and fix some https links in README
andrew-d: Refactor packages
andrew-d: Fix TravisCI tests
andrew-d: Remove deprecated ResultTree API
Dabaez: Update README.md
barakmich: Implement the Unique iterator

Mongo speedups, Loading fixes, Except/Not Operator

@barakmich barakmich released this Apr 26, 2015 · 743 commits to master since this release

Point release, to get a number of speedups and fixes out to the binary distribution channel.

Roadmap for 0.5 includes a number of major open PRs.

Notable features:

  • #236 Much faster Mongo support
  • #180 No Mongo empty values
  • #153 Gremlin Except() operator
  • #184 Fix Google Appengine Support
  • #235 Repeatable builds with Godep

Full changelist:

    @jf87: added example n-quad file for gremlin api description
    @barakmich: Mongo Indexed LinksTo
    @jzelinskie: store dependencies' versions with godep
    @barakmich: fix decompressing for http
    @andrew-d: Add myself to AUTHORS + CONTRIBUTORS
    @andrew-d: Add Err() and Close() methods to Iterators
    @barakmich: Fix PrimaryKey Marshalling and add Bolt test
    @alexwlchan: A couple of small spelling fixes in the docs
    @barakmich: Rewrite keys into concrete types, remove key package
    @barakmich: Clean up a little lint and some shadowed variables
    @LAlbertalli: Add options to ignore duplicate or missing quad
    @kortschak: Make query/... interfaces more idiomatic
    @barakmich: bench: Add some grunty Not queries to the integration test (derived from...
    @buley: Misspellings in Gremlin and Config docs
    @mataevs: Except/Not Operator for Gremlin.
    @barakmich: Minor doc fixes
    @kortschak: Provide informative logging for dups/absences
    @bkendall: Prevent empty values in cache (using mongo)
    @kortschak: Make graph.Delta zero state invalid for use
    @kortschak: Ensure callers get errors when they happen
    @varadharajan: Allocate 82 bytes instead of 62 bytes in LevelDB's CreateKeyFor method
    @bcleenders: Prevents repl panic, uses primary key generation for all writers                              
    @panamafrancis: Implementation of the PrimaryKey type
    @barakmich: mathutil is back in line
    @barakmich: temporary travis fix
    @kortschak: Fix leveldb change skew
    @barakmich: Remove the news in the README, add Trello Link
    @kortschak: Issue160
    @kortschak: Don't retain results where the value is empty
    @kortschak: Ensure we don't examine empty token
    @kortschak: Replace DebugString with a well defined type returned by Describe
    @kortschak: Delinting, deadcode removal and vetting                                                                  

Bolt, Speedups and Fixes

@barakmich barakmich released this Aug 26, 2014 · 917 commits to master since this release

Worth noting: you may have to reload your data in this version, as the database format has changed (hence the minor version bump). This is part of working with early software.

On the plus side, you'll notice quite a few benefits, including the new features.

For example, the memory benchmarks:

benchmark                                   old ns/op        new ns/op     delta     mult
BenchmarkNamePredicate                      917680           1072457       +16.87%   +1.17x
BenchmarkLargeSetsNoIntersection            176704664        48692651      -72.44%   -3.63x
BenchmarkVeryLargeSetsSmallIntersection     32566448297      154770005     -99.52%   -208.33x
BenchmarkHelplessContainsChecker            16041341849      40699233      -99.75%   -400.00x
BenchmarkNetAndSpeed                        71710442         16658337      -76.77%   -4.30x
BenchmarkKeanuAndNet                        13858313         12192151      -12.02%   -1.14x
BenchmarkKeanuAndSpeed                      16161303         15231229      -5.75%    -1.06x
BenchmarkKeanuOther                         3748121471       60429073      -98.39%   -62.11x
BenchmarkKeanuBullockOther                  300796920802     99996852      -99.97%   -3333.33x

If you think those look good, they also get better with bigger backends. The big wins come from optimized iteration, which benefits all backends.

New in this release:

  • Bolt backend for persistence
  • Log-structure persistence
  • Materialize iterator and related speedups
  • Better memory backend (based on cznic/b)
  • Better Gremlin session handling
  • Underscore.js available in Gremlin

Shortlist of Bugfixes:
#9 Use boltdb in the backend
#70 Write-log and as_of_time and replication preparation
#93 Simplify the Nexter Interface
#107 by default
#106 #108 Use cquads in the web UI loader
#114 Materializer regression sparking odd results
#117 Data-race when handling many simultaneous queries
#125 Underscore.js available in Gremlin environment
#104 #129 Better REPL support (history, etc)
#141 Parser conformance for IRIRef
#145 Gremlin workers