The long anticipated release brings a lot of bug fixes and some performance improvements, especially for KV backends.
- New SQLite backend (#751).
- KV and NoSQL backends migrated to Hidalgo.
- Initial import to KV backends should be much faster now (#803).
- A new
Order()iterator and corresponding method for Gizmo (#796).
- Set of changes to Gizmo to make it more Gremlin-compatible (#822, #849, #859).
- Gizmo can now emit JSON-LD via HTTP API if appropriate
Acceptheader is set (#821).
- A lot of bug fixes for queries, backends, etc. See the full changelog for details.
- Switched to Go modules.
- Quads package split to it's own repository.
query.Sessioninterface changed to be more user-friendly.
Note: Cayley haven't reached 1.0 yet, thus you should pin a specific version in
go.mod to avoid breaking changes. We try hard to minimize those.
This release brings multiple stability fixes and adds initial support for cyclic objects to the schema lib.
- Fix bug when deleted a reinserted quad in KV backends (@3pCode, #747).
Optionaliterator not propagating own tags (@phyrwork, #739).
- Update BoomFilter to avoid panic (@hypirion, #744).
- schema: Support loading and writing objects containing loops (#745, #746).
- schema: Fix recursive ID lookup in anonymous fields (@phyrwork, #733).
Resolveriterator (@phyrwork, #741).
BatchQuadStoreinterface (@phyrwork, #742).
As always, we expect Go users to pin a specific Cayley version tag with dep.
dennwc: iterator: do not drop an error in resolver phyrwork: Implement Resolver iterator HaraldNordgren: Bump Go versions and use '.x' to always get latest minor versions 3pCode: update test to check all return codes, update variable name. 3pCode: Update authors and contributors 3pCode: Fix bug when deleted a reinserted quad. cristaloleg: Use type-switch dennwc: schema: support loading objects with loops dennwc: schema: split schema package into multiple files phyrwork: Add RefsOf to the BatchQuadStore interface dennwc: schema: support writing structs that contain loops; fix #731 phyrwork: Fix mock store ValueOf implementation never returning nil hypirion: Update dependency revision on BoomFilters phyrwork: Fix Optional iterator not propagating own tags s_guozhuming: run tests for a specific backend implementations s_guozhuming: pull image from remote repository if there is not image at local machine barakmich: change order of dep grabbing barakmich: Update pouchdb version barakmich: fix goja dep, update travis lynic: Correct db name using mongo backend (#723) phyrwork: Fix Config.idFor() recursive ID lookup in anonymous fields (fixes #732) tooolbox: Begin the Glossary (#728) jtorvald: Added unsigned integer types (#725)
This release fixes few bugs and adds label support for the schema lib.
- Schema lib now allows adding a label to all written quads (1a0c7cb)
- Fixed a bug with KV backend that allowed to write duplicate quads in the same TX (#675)
All()query for nodes now honors a query limit (#718). Previous behavior can be achieved by passing
As always, we expect Go users to pin a specific Cayley version tag with dep.
dennwc: graphql: update dependency and test filter with multiple values dennwc: schema: allow to specify quads label on write dennwc: bolt: do not insert duplicate quads found in the same tx; fixes #675 dennwc: set a correct limit in integration tests dennwc: cli: assume stdout as an output when no arguments are specified dennwc: gizmo: All should honor a query limit; fixes #718 robfig: cayley-single.yml: update version tag to latest
This release brings Elasticsearch v6.x support, a transition to dep and few bug fixes.
- Cayley now supports ElasticSearch v6.x (#711)
SaveOptionalis exposed in Gizmo (#702)
- Transition to dep for dependency management (#704, #703)
- Bug fixes (#710, #709, #708, #707)
As always, we expect Go users to pin a specific Cayley version tag with dep.
dennwc: quad: do not un-quote strings in StringToValue; fixes #710 dennwc: add CODEOWNERS dennwc: elastic: support v6; fix #711 skydan: Fix dep ensure and downloading with "follow redirects" option mguentner: hello_bolt: replace TempFile with TempDir mguentner: hello_schema: replace TempFile with TempDir dennwc: gizmo: expose SaveOptional; resolves #702 dennwc: build and test for go 1.10 dennwc: switch from glide to dep glaslos: replace badge png with scaling svg
This minor release brings few bug fixes, faster CockroachDB backend, configurable schema package and new GraphQL features.
It will be the last feature release before switching to v0.8 branch. Only bug fixes will be backported to v0.7 moving forward.
- Faster CockroachDB transactions (@dsymonds).
- Support for un-nesting objects and expand-all for GraphQL (#686, #692).
- Configurable schema package (#694).
- Fixes for json-ld, graphviz data formats (@jtorvald).
- Fixes for Recursive iterator and optimization of And.
As always, we expect Go users to pin a specific Cayley version tag with a dependency manager.
dennwc: schema: allow to customize behavior; resolves #694 dennwc: graphql: support un-nesting objects into parent; resolves #686 dsymonds: Switch CockroachDB driver from lib/pq to jackc/pgx. dennwc: graphql: support expand all; resolves #692 dennwc: properly close sub-iterator in recursive; fix #693 dsymonds: Rewrite CockroachDB SQL construction to be much faster. dsymonds: Bump cockroachdb Docker tag for testing to the latest version, v1.1.5. dennwc: iterator: do not add empty iterator to And on failed optimization jtorvald: jsonld: added a DocumentLoader to NewReaderFromMap; fix NPE (#689) dennwc: dot: correctly escape slashes; additional file extension
v0.7.1 -- Wildcards, CouchDB, fixes and performance
This minor release brings multiple bug fixes, query performance improvements, wildcards query and new CouchDB backend support.
As noted in v0.7.0 release, support for legacy
mongo1 backends was dropped. New backend implementation are still available under
- CouchDB support (@elliott5)
- PouchDB support via GopherJS (@elliott5)
- Support for wildcard queries (#676)
- Multiple fixes for
- Better location of UI assets (@gauravtiwari)
- KV and SQL backends will now track node ref counters properly.
- Various fixes and performance improvements (1f53d04, 463642e, 2d6d563; thx to @jtorvald)
As always, we expect Go users to pin a specific Cayley version tag with a dependency manager.
v0.7.0 -- Meta backends, ElasticSearch, Shapes, and new CLI
Release covers nearly 8 months of work, mostly on implementing meta-backends, new query optimizer, performance improvements and bug fixes.
Note: From now on we plan to release point releases on regular basis to push fixes and new features faster.
Storage format for backends received an upgrade bringing backward incompatible changes. Check migration guide for more details. It only affects the database format, HTTP API remains backward-compatible.
This release defines new internal interfaces (meta-backends) for working with various database kinds. Currently three of them are defined: KV, NoSQL, SQL. Each of them significantly simplifies a process of implementing a new backend, as well as allows us to lower maintenance overhead by sharing most optimizations, read and write paths, etc. This also means that optimization work on one backend will be automatically applied to other backends of the same kind in the future.
Part of this effort was an implementation of new query representation that we call "shapes". All optimizations like query rewrites and reordering are now done on a generic level, without any backend-specific code. It will allow us to save query stats, make more complex optimizations and even store query templates in the database, or split it to be executed in distributed manner by multiple backends. Since new representation can be manipulated more easily, most backends will now rewrite complex query types into their native query language (SQL, for example).
- ElasticSearch support
- New query optimizer (aka query shapes)
- New command line interface
- KV meta-backend + port of Bolt and LevelDB to it
- NoSQL meta-backend, + port of Mongo and ElasticSearch to it
- "Raw" value type is deprecated
- Exposed node delete API for HTTP
- Consistent behavior for all backends on node delete
- New query types in Gizmo:
SaveOutPredicates, see docs
Hastraversal now supports value filters
FollowRecursivecan be limited to a maximal depth
- GraphQL now supports quad labels
- SQL query optimizer improvements
- SQL backends now has their own names:
cockroach(no need to specify
- Significant performance improvements
- Smaller Docker image
- Minor UI improvements
- Bolt, LevelDB and Mongo backends were rewritten and database format has changed. Old implementations are still available under
mongo1names. Old backends will be removed in next minor release. c266392
- SQL table structure has changed 31b0f48, #617
- Cayley now ignores duplicated quads by default 9cb0eef
- Gremlin is deprecated and dropped in favor of Gizmo #519
- Count in gizmo is now a 'final' instead of traversal and returns result #576
Breaking changes (Go)
As always, we expect users to pin a specific Cayley version tag with a dependency manager and sync with following changes in the code:
- Cayley uses Go version 1.9+
- Bolt, LevelDB and Mongo backends were rewritten and database format has changed. Old implementations are still available under old paths, but will be dropped in next minor release. Users are encouraged to update imports to
- SQL flavors moved to their own packages and needs to be imported separately (
sqlbackend name will be deprecated in next major release. 48e7946
- "Raw" value type (
quad.MakeRaw) is deprecated and replaced with a helper to preserve at least some compatibility. It's strongly recommended to switch to
- Iterator optimizations for backends are mostly disabled. It is recommended to build queries with new
graph/shapepackage and run optimizations on it.
- Removed unused methods from
Typed994e2a, b63faea, #631
graph.Valueis now type-safe e4ab526
graph.Delta- they will be assigned by server 524c8e2
RemoveNodetype changed to
quad.Valueto match other methods #607
- Quad writers produced by
Flushto be called a420e62
Iteratornow accepts context for
Describemethod was removed from
Iteratorand replaced by
Stringmethod on iterator and
- Iterator type is now a string instead of enum and registration is not necessary anymore
dennwc: http: expose node delete api; resolves #622 dennwc: deprecate quad.Raw type dennwc: code-wide cleanup dennwc: iterator: remove unused equality function for fixed dennwc: graph: remove unused horizon function dennwc: memstore: use correct size for all iterator dennwc: add context argument to iterators and kv dennwc: kv: track nodes reference count; fixes #617 dennwc: sql: track nodes reference count and simplify dialects; fixes #617 dennwc: memstore: remove nodes if all their references are gone; addresses #617 dennwc: iterator: returns correct subiterator slice for empty And; fixes #659 dennwc: nosql: use base64 instead of hex encoding for hashes dennwc: update Go version to 1.9 dennwc: update integration test, make it a part of suite; move paths as well dennwc: fix tests for predicate save query; fix node tokens for bolt dennwc: nosql: use shapes optimizer for queries dennwc: shape: allow to simplify shapes; add helpers for building node filters dennwc: elastic: port to nosql layer; resolves #653 dennwc: nosql: store each typed value in separate field; support batch inserts dennwc: separate mongo logic and define a generic nosql layer; resolves #652 dennwc: iterator: replace Describe method with a helper dennwc: graphql: support @label directive; resolves #614 dennwc: path/gizmo: support Has with filters; resolves #650 dennwc: bolt,leveldb: swap names to a new kv backend implementations michaelqiu94: ElasticSearch as Supported Backend (#634) dennwc: path,gizmo: allow to save predicates without moving from nodes; fix #591 dennwc: http: allow to serve API without web files; fix #503 dennwc: ui: format json results; resolve #454 dennwc: sql: optimize intersection, value lookup and comparison dennwc: sql: fix params propagation from sub-queries dennwc: sql: rewrite query builder to shapes; fix #563 dennwc: shape: optimize multiple Pages; push Fixed values from Intersect into NodesFrom dennwc: propagate sub-iterator errors from And properly dennwc: stream ops in node delete; return error if node is missing; fix #608 dennwc: add a single package that imports all supported backends derekrliang: Add optional max recursion depth to FollowRecursive (#639) derekrliang: Fix query shape error when query has no result data links (#641) derekrliang: Add running local unit test documentation (#640) dennwc: bolt,leveldb: port to kv layer dennwc: kv: add in-memory implementation dennwc: integration: open and close db for each test set; use table benchmarks dennwc: kv: support for index scans and indexes on multiple directions dennwc: kv: allow to customize indexes dennwc: kv: batch get requests and simplify bucket interface Yannic: Reduce Docker image size (#637) dennwc: sql: separate packages for flavors (#633) Yannic: Remove unused Type function (#631) dennwc: docs: add http api definitions (swagger v3) dennwc: iterator: remove duplicated TagResults code and type registration dennwc: github: simplify issue template header, add slack link dennwc: tag values in recursive iterator correctly dennwc: shape: new query optimizer dennwc: memstore: iterator optimizations dennwc: graph: rewrite pk; remove horizon and ts from delta; rewrite memstore dennwc: kv: registration, scan by prefix, support for flat kv dennwc: integration: allow to load data to remote backends dennwc: gizmo: fix #620 and bump goja version dennwc: graph: helper for writing quads to tx (useful for schema) dennwc: writer: add a direct constructor for single replication Jason Kingsbury: make bloom filter buffer before initialisation dennwc: cli: read query timeout from config file; fix #613 dennwc: graphql: allow full IRI charset in names; resolve #616 dennwc: schema: allow to limit depth; fix #609 joostverdoorn: Set CORS headers on gephi stream dennwc: http: allow to specify query limit; fix #612 joostverdoorn: Add labels morphism and Gizmo API (#610) dennwc: http: do not add .md extension if it's already present; fix #597 dennwc: schema: return an error on write if required field is not set; fix #602 dennwc: accept quad.Value instead of graph.Value for RemoveNode; fix #607 dennwc: print warning on set verbosity for glog; fix #599 dennwc: flush writer in schema example; fix #606 harlantwood: Remove "angry highlighting" from JSON examples dennwc: generic kv backend based on bolt h4ck3rm1k3: query from stdin (#601) chrispassas: Fix example startup command dennwc: make graph.Value safe again h4ck3rm1k3: adding help to repl dennwc: schema: handle id in anonymous struct fields h4ck3rm1k3: Update sidebar.tmpl (#595) dennwc: schema: allow to load objects from path dennwc: cli: implement query command barakmich: conversion check h4ck3rm1k3: Update transaction.go dennwc: graph: batch quads automatically if caller uses single AddQuad on writer gkontos: gae: fix add of ...
v0.6.1 -- GraphQL, Gizmo, SQL flavors, Recursive and Gephi
This release marks community build number three. It includes all the pending features that can be merged before v0.7, which will be a major release to introduce Reification, and may cause breaking changes.
The storage format was not changed for this release. Existing databases should work properly without any migration needed.
Cayley now supports multiple SQL flavors, including MySQL and CockroachDB.
This release introduces an experimental GraphQL implementation. Note that it only shares a similar syntax and does not support schema yet. It's a preview feature, so the implementation or API may change in following releases.
The import and export formatter system is now modular and can be easily extended. Cayley now supports JSON-LD, Graphviz (aka DOT; export only) and custom binary formats for better import speed. The HTTP API was updated (v2) to support the new formatter system and allows for the streaming of large amounts of data in single request.
The recursive iterator is finally available via the Go path library and Gizmo. It allows traversal of a path recursively to potentially unlimited depth.
A new Schema library was implemented to allow developers to save and load complex Go objects with a single method call.
- Support for MySQL and CockroachDB.
- Gizmo query language with faster ES5.1-complaint VM (supersedes Gremlin).
- Experimental GraphQL-like query language.
- Modular import/export formats system.
- Support for JSON-LD, Graphviz (DOT) and Cayley-specific binary format.
- HTTP API v2 with better import/export performance for large data files.
- Recursive iterator support (FollowRecursive).
- Schema lib (aka ORM or Object Persistence Layer).
- IRI prefix registration.
dennwc: gizmo: expose recursive traversal and update docs dennwc: path: FollowRecursive traverses quad.Values barakmich: add FollowRecursive to the path lib barakmich: graph/iterator: Add Recursive iterator dennwc: graphql: close iterator properly; fixes #545 dennwc: tests: require.NoError instead of Nil dennwc: gae: fix nop iterator optimization dennwc: sql: append WHERE only if constraints are present dennwc: http: CORS, API to get supported formats and RO mode fix dennwc: quads: add json stream format dennwc: mql: fix NPE dennwc: gephi: allow to filter quads in raw mode dennwc: gephi: inline primitive values automatically; inline schema:url dennwc: gephi: initial support for graph stream jtorvald: Updated documentation based on Issue #349 (#541) caosiyang: docs: fix a typo in Quickstart-As-Application.md (#542) jtorvald: Issue #286 init db and config documentation (#538) Yannic: graph: cleanup and more robust options handling (#536) dennwc: initial graphql support dennwc: cli: fix loading of data for non-persistent backend jtorvald: docs: replace broken MQL link, add quick-start cli usage example (#535) mikaelcabot: CockroachDB transaction retries (#534) dennwc: bolt: make bolt concurrent-safe as it should be dennwc: gizmo/gremlin: fix tags for Both traversal; fix #532 dennwc: fix #530 and write errors as plain json dennwc: quad: support for graphviz format dennwc: memstore: close iterator properly dennwc: memstore: do not hash values in QuadDirection dennwc: memstore: no recursion in Next dennwc: http: use formats - implement write api v2; resolves #176 dennwc: mongo: allow to pass full urls; fix #526 dennwc: add missing build tags; find relative path of data files automatically dennwc: sql: support flavors: mysql, cockroachdb dennwc: mongo: remove nodes with no references; fix #517 dennwc: gizmo: allow to register custom IRI prefixes; resolves #505 barakmich: revert proto GetNativeValue and check for crashing nils on describe barakmich: Fill literal strings in MQL the same way we do results in Gremlin dennwc: gremlin: fix limit in ForEach; fixes #518 barakmich: Fix non-persistent load, modernize MQL test barakmich: Fix -v option and debug error barakmich: Tidy up Optional iterator, and allow MQL to support JS-like string syntax dennwc: switch to goja (gizmo) barakmich: Fix dependencies, make Docker an optional dependency, add build tags for docker and appengine tests mzats: Updated examples so that they work in repl/http. (#499) barakmich: Remove IsNode() from the graph.Value definition. dennwc: schema: move well-known types package dennwc: graph: refactor interfaces dennwc: add example for schema library dennwc: ported schema lib dennwc: use vocabulary constants for value types, allow to shorten and expand IRIs dennwc: register known rdf vocabularies, add constants joostverdoorn: Add missing gojsonld dependency to glide (#502) dennwc: bolt/leveldb: stop complaining about existing quads even if ignore is set dennwc: http/repl: handle errors correctly dennwc: nquads: remove duplicated code dennwc: integrate quad formats, replace exporter, use quad helpers in tests dennwc: quad: readers, writers and formats. dennwc: Update minimal Go version dwhitena: Ability to pass the Mongo Backend an externally managed session. (#486) pbsladek: Update quickstart for 0.6.0 (#487) dennwc: add github issue template dennwc: sql: fix sqlNodeIterator size dennwc: fix count iterator dennwc: path: test both optimized and unoptimized case; few more test for gremlin kaneshin: Fix minor typos (#483) barakmich: Add arm64 support
v0.6.0 -- Types
Current release marks community build number two, introducing major change to data layout and bringing new features to Gremlin.
Storage format changed slightly. We highly recommend to do a clean import of the database with a new binary.
Cayley now honors types of nquad values. IRIs (
<name>) were converted to untyped strings (
name), so it was possible for nodes with different types to be loaded as a single node. Now Cayley preserves type information, making these nodes separate. Queries that used IRIs without
<> quotes will break because of this change. Gremlin now requires IRI it to be in
"<follows>" form and the path lib expect these values to be of
quad.IRI Go type. The same stands for Strings (aka RDF literals) and BNodes (aka blank nodes,
Data loader now recognizes few basic typed literals like
"10"^^<http://schema.org/Integer> and converts them into corresponding native type (
Int for this example). It may also affect existing queries. Gremlin will automatically convert basic types into corresponding RDF typed literals.
- Typed values support for all backends. Includes types: String, IRI, BNode, Int, Float, Bool, Time.
- New traversals in path lib and Gremlin: HasReverse, Skip, Limit, Count, Unique, Comparison/Filter, Regex.
- More consistent backends behavior thanks to unified tests.
- Updated docs and added examples.
- Updated Docker image, and added example for it.
- Many, many bugfixes and some optimizations.
gabrys: Fix link to GremlinAPI.md (#477) dennwc: expose Unique via path lib and gremlin; implement and expose Count iterator; resolve #240 dennwc: gremlin: add regex support, make it harder to abuse regexps; add docs explaining why guycook: Implement regex filter as iterator (#474) dennwc: bolt,leveldb: fix AllQuads iterator not skipping deleted quads, add test dennwc: iterator: add variadic arguments to Add, Or and Fixed constructors dennwc: query: Refactor interfaces (#471) dennwc: fix lru panic wayeast: Clone receiver paths (#466) kenshaw: appengine: updating dependency paths and adding documentation (#464) dennwc: gremlin: handle string slice in via parameter kenshaw: appengine: add build constraint (#463) dennwc: Glide migration (#460) dwhitena: user/pass functionality for mongo backend allanino: Fix Docker building (#456) barakmich: gremlin: don't require a session to last as long as the timeout kris-runzer: graph: fix Is*(err error) checkers to properly compare against *DeltaError, add tests dennwc: optimize by default in Iterate helper dennwc: gremlin: clone path objects neonstalwart: simplify the signature of NextLogOut (manual merge of #264) dennwc: graph: add iteration helpers, update examples; resolves #446 dennwc: http: fix json quads decoding dennwc: gremlin: fix ToArray without arguments dennwc: iterator: promote Next to main interface; make NoNext an optional interface oren: Example of BoltDB, update examples/README.md robertmeta: Just better quickstart + examples creation dennwc: query: check type assertions, recover properly in gremlin; fixes #450 josephschorr: Add Clone() to Path for easier reuse of Path segments (#444) dennwc: Wire Skip and Limit to Gremlin. Also fixed #254. dennwc: postgres: store hashes as binary data instead of hex strings dennwc: postgres: prepare each statement in ApplyDeltas dennwc: postgres: generate distinct node insert statements only dennwc: cquads: convert TypedString values to native types automatically on loading dennwc: gremlin: allow to specify multiple conditions in Filter dennwc: mongo: optimize comparison iterator dennwc: Test for typed value comparison and optimization dennwc: gremlin: expose value comparison iterator via path lib, add tests dennwc: gremlin: support for typed values dennwc: gremlin: replace JS-based graph object with reflect-based (rewritten, now it's easier extend) dennwc: comparison: compare only values with the same type, add a test for mixed set dennwc: mongo: use native value types where possible, fix indexing, remove recursion in LinksTo, remove unnecessary roundtrip dennwc: postgres: new table structure, store typed values natively; fix iterators; separate LRU cache dennwc: Update value comparison iterator. Stop graph.Value madness. dennwc: cquads: recognize TypedString and LangString in parser dennwc: Add a test for Raw-to-typed values exchange. Fix memstore not using hashes in map. dennwc: GAE (untyped). More tests. Fixed handling of size and conflicts in postgres. dennwc: quad: add support for few basic types (int, float, bool, time) dennwc: Unify tests for QuadStore. Run Mongo/Postgres in Docker automatically. dennwc: leveldb: use protobuf instead of json, migration code dennwc: bolt: migration code dennwc: add typed fields to protobufs dennwc: graph,query,exporter,path: updated to use typed values dennwc: quad: directions are typed kortschak: graph: provide more meaningful errors for bad actions (#443) josephschorr: Don't iterate over all quads unless necessary (#440) kortschak: clog: fix comment typos and add package comment (#442) dennwc: Implement Skip and Limit iterators (#369) EntilZha: Delete node API (#158) dennwc: Updated Dockerfile (fixed #357) dennwc: fix Unique iterator for non-comparable tokens; fixes #437 kortschak: clog: make nil Logger safe (#436) robertmeta: Sweep through docs update/cleanup oren: remove duplicate in readme (#435) dennwc: fix postgres error on import, fixes #429 dennwc: Added HasReverse function to path lib (#351) drzippie: Added sync.RWMutex to mongo cache (#380) robertmeta: Update docs (README, Locations, Quickstart-As-Lib, Quickstart-As-Application, HACKING, FAQ, Container) robertmeta Rerouting CONTRIBUTING to the discourse, updating CONTRIBUTORS to remove mention of CLA robertmeta: Moving Hacking from wiki to project dennwc: separate log package, resolves #221