Assets 3

Dear Grakn Community, we've got a New Year's gift for you: Grakn Core and KGMS 1.4.3 has been unleashed!!!

2018 has been a tremendous year for us at Grakn Labs. Well done to the Grakn Warriors on this massive engineering leaps we've taken this year! And as always, thank you so much to our beloved community for helping us get here!

The focus of this release is to enable us to upgrade Grakn KGMS (previously 1.2) to use the latest Grakn Core 1.4.2, and leverage all the architectural and performance improvements that we've introduced to the core stack. Before we can do so, we had to upgrade Grakn Core one more time, from 1.4.2 to 1.4.3.

Grakn Core 1.4.3

This is a minor release for Grakn core, as the major work on Grakn Core is currently undergoing for the 1.5 release, which will be released several weeks from now. The focus of this 1.4.3 release for Grakn Core is to fix issues that prohibit us from upgrading Grakn KGMS to 1.4. Nevertheless, they also beneficial for Grakn Core in general.

The following are improvements we made in Grakn Core 1.4.3:

  1. Bootup: When you create a new copy of a GraknConfig instance, the copy will no longer share the underlying data with the original instance. This is the correct and expected behaviour. We also provide a warning if cassandra-pidfile system property isn't set, and the data-dir parameter in grakn.properties is now correctly applied to Cassandra's cdc_raw_directory and hints_directory.
  2. Keyspace: A KeyspaceStore can now be properly closed with closeStore().
  3. Sessions: Adds closedSessions() which can properly close sessions opened by EngineGraknTxFactory.
  4. Compute Query: GraknComputerImpl now handles ExecutionException in a cleaner manner. Before, the method will throw a RuntimeException which will wrap ExecutionException which wrap the actual Exception. Now, the RuntimeException will wrap the actual Exception directly without any intermediary ExecutionException. We also removed an obsolete log message "Hadoop graph ignores parameter address."
  5. Post-processing: Refactored AttributeDeduplicatorDaemon so that it can be injected into the Grakn Core and KGMS Server. AttributeDeduplicator.deduplicate() also no longer opens its own Transaction instance, making the caller responsible for providing one. This means that the caller has full control over how to tune the transaction parameter.

Read more on Grakn Core and KGMS 1.4.3 release through our community announcement post. We wish you a grand 2019 and lots of fun using Grakn KGMS on Google Cloud and Amazon AWS with the latest Grakn Core features!

@marco-scoppetta marco-scoppetta released this Oct 4, 2018 · 188 commits to master since this release

Assets 3

Dear Grakn Community, Grakn Core 1.4.2 is now available!

(This is a patch release for Grakn Core 1.4.0)

When we released Grakn 1.4.0 with Windows support last week, we broke the distribution for a few Linux distros. ;) Thanks to some of you in the community, this was uncovered immediately - thank you! We've fixed the issues now and Grakn should be back to normal - running on all Linux distributions.

There were other improvements that went into the codebase since the last release, but this patch release focused on the following issues.

  1. Fixes #4381: User experienced an issue connecting to Grakn from some Linux distributions
  2. Fixes #4420: User is unable to add client-java to his/her project due to a missing pom.xml

Enjoy Grakn Core 1.4.2 on OSX, Windows, and Linux! Stay tuned for 1.5 coming out soon with major architectural (and thus, performance) improvements!

Assets 6

Our beloved Grakn Community, Grakn Core 1.4 is finally released! And, it comes with the very much anticipated: Grakn Workbase!

Grakn Workbase 1.4

Well done to our Grakn Warriors (kuple!) and, as always, thank you so much to our amazing community for helping us improve Grakn every day of the journey!

As we promised in the previous release, we started making smaller faster releases. This release includes 4 key improvements - but major improvements nonetheless, focusing on performance, reliability and usability.

Engine post-processing now happens in real-time

  1. Grakn Engine (aka. server) no longer has a periodic batch postprocessing tasks running in the background. Engine's postprocessing task now runs in real-time! This makes Engine so much faster, and allowed us to also implement the following point.
  2. Grakn Engine no longer depends on Redis! One big system less from the server's dependency, making Grakn much lighter, more stable and reliable.

Grakn now runs on Windows

All our Windows-based community members, we heard you, we did not forget about you! We fought the battle: upgraded Cassandra from 2.17 to 3.11.3, and rearchitected Grakn to bring it to you on Windows! You can download it from our download centre as usual, and choose "Windows" as the OS.

Graql reasons faster

We made Graql's reasoning engine much faster by adding more improvements in rules pruning and in the way Reasoner handles attributes.

  1. Using inferred types when calculating the structural hash of relations
  2. Fixed the vicious circle of attribute playing role with a rule
  3. Attaching disconnected resource during inference no longer throws an exception
  4. Prune rules based on attributes of the head relation

In addition to the above, the following issue in Graql was also fixed.

  1. When an attribute plays a role, other roleplayers no longer (incorrectly) become owners.

And finally, Workbase is born into this world!

There has been a lot of work put into our new "Knowledge IDE" - starting with the visualiser. Why? Performance, and performance! And usability, of course. This first release is already available for Linux, Mac and Windows. You can download it directly from the Grakn Core 1.4 release page on GitHub (the download centre page for Workbase may take a few days to be updated).

What's next?

Read more on our community announcement post!

@lolski lolski released this Aug 7, 2018 · 339 commits to master since this release

Assets 4

Grakn Core 1.3 has been unleashed!

Finally, the long-awaited 1.3 release has arrived! Kudos to our Grakn team (kuple-kuple!), and thank you to everyone in our community for helping us improve Grakn every day!

There is an incredible amount of work that has gone into this PR. So much so, that it has taught us the lesson to make much smaller releases and to do so more frequently. Below is the list of changes in this release. We’ve got quite exciting news to share, we hope you enjoy reading about it and we look forward to seeing you upgrade to it!

New Language Drivers

The changes and improvements in this category are actually very extensive for each component, but we'll keep the summary at a high level because the modules have either been heavily re-written or are completely new. And, all Grakn Client Drivers are released under the Apache 2 license!

  1. New client driver protocol (client-protocol): we now provide access to Grakn via a GRPC (Google’s Remote Procedure Call) framework, which allows us to provide Grakn drivers directly in more native languages! Please reach out to us if you’d like to collaborate on writing a Grakn driver in a new programming language!
  2. New Java client driver (client-java): we’ve upgraded our Java client driver to implement the latest GRPC protocol and fully support all Graql queries in a Java client.
  3. New NodeJS client driver (client-nodejs): we’ve upgraded our NodeJS client driver to implement the latest GRPC protocol and fully support all Graql queries in a NodeJS client.
  4. New Python client Driver (client-python): we’ve upgraded our Python client driver to implement the new GRPC protocol and fully support all Graql queries in a Python client.

Engine

(This module will soon be renamed to "Server".)

  1. New RPC Server, implemented using GRPC (Google’s Remote Procedure Call) framework - it's all about performance and control! So far, we've implemented 2 "services" for any Graql driver (in any language) to interact with Grakn remotely:

    • Session Service: provides the full API of Grakn Transactions. This includes the Graql query API, as well as the all of the methods in the Concept API.
    • Keyspace Service: provides Keyspace management functionalities - create, retrieve, and delete. Only retrieve and delete have been implemented in this release - create will follow soon.
  2. Improved boot-up process:

    • No longer fails to start because it could not find the PID.
    • Properly exits with code 1 on failure.
    • Boots up even when located on a path which has whitespaces.
    • Terminates immediately when there is a bootup failure.
    • Outputs stderr into the log file whenever an error occurs.
    • The grakn-bootup module has been removed, and the codebase merged into grakn-engine.

Graql: usability and performance

The following are the medium to large changes and improvements in this release. There have been a lot of smaller improvements and bug fixes that may not be included in this list.

  1. Refinement of the Graql compute queries

    We have an improved Graql compute query grammar (Graql.g4:23-75) that produces a more generic and expressive language for analytics, whilst making it simpler to use. The new Graql compute query now has the following behaviour:

    • A simple structure of: compute <method> <conditions>? ;
    • Uses commas (,) properly instead of semicolons (;), and
    • Uses square brackets ([ and ]) to provide a list of arguments.
    • Improved error messages for Graql compute queries.

    Thus, analytics queries will look like: compute cluster in [person, marriage], using connected-component, where [start = "V123"];

  2. Refinement of the Graql compute path query

    Graql compute path (that returns one shortest path) and compute paths (that returns all shortest paths) are now merged into just compute path that returns all shortest paths. This change is also followed by:

    • Improved performance by streaming the results of compute path (which is a list of paths), both on server-side and client-side queries.
  3. Graql reasoning

    • Improved performance by considering rule resolvability and query atomicity when planning resolution.
    • Improved performance for queries with missing type labels.
    • Added transaction-level rule cache to improve performance with large rule bases.
    • Added extra rule pruning step to improve performance with large rule bases.
    • Improved query planning to guarantee production of complete plans.
    • Improved validation of resolution plans.
    • Corrected behaviour of isa! patterns.
    • Optimised propagation of partial substitutions within queries with neq predicates.
    • Fixed query cache inconsistencies involving disconnected id predicates and surplus variable mappings.
    • Corrected behaviour of rules that attach a role player to an existing and/or new relations.
    • Improved correctness of atom decomposition for conjunctions.
    • Removed the materialise option for inferred answers as it unnecessarily cluttered the code, and the functionality can be accomplished straight-forwardly with the current API.
    • Improved resolution logging to be more meaningful.
  4. Disambiguation of equality comparison operators

    • $x = $y requires $x and $y to be the same concept
    • $x != $y requires $x and $y to be different concepts
    • $x == $y requires $x and $y to have the same value
    • $x !== $y requires $x and $y to have different values
    • As a result, the val keyword became redundant, thus it has been removed.
  5. Concept API

    • A lot of method renaming to be simpler, and more aligned with Graql terminology.
    • Lightened the dependencies by removing TinkerPop.
  6. Unification of Graql Query Answers

    • All Graql queries now return a List of Answers, where Answer could be one of 6 types of common data structures: Value, ConceptMap, ConceptList, ConceptSet, ConceptSetMeasure, and AnswerGroup. This includes queries that previously returned a single or void value, such as delete and aggregate queries.
    • Given the above change, all Graql queries are now streamable.
    • A delete query now returns the list of ConceptIds that it has deleted.
    • Several aggregate queries are removed because they do not fit with any data structure, and do not deliver a strong and unique use case: ask, select, and as aggregate. They have plenty of workarounds too.
  7. Graql correctness and reliability

    • Graql queries that perform match over both instance and schema concepts now work correctly.
    • Graql queries of the form match(...).get(var("x")) (single var) no longer crash on remote client-java.
    • Improved various error message and exception handling.
    • Upgraded TinkerPop to 3.2.9 and removed the dependency on our custom TinkerPop 3.2.5-grakn-snapshot.
    • Fixed delete queries to work remotely, and with an arbitrary number of given variables.
    • match-insert queries now execute inserts before streaming results.
    • Improved exception logging for execution of remote Graql queries via GRPC.
    • match-get queries containing a relationship with roleplayers that share a role no longer break on remote client-java.
    • Graql aggregate count queries now take in a variable in which it computes count on a distinct set of results.
    • Graql aggregate and compute queries that perform a numerical computation no longer return a just a value. Instead, they return a list containing the value. Therefore, when there are no instances of the aggregated attribute type, we no longer return a null value, but rather an empty list. I.e. match $x isa person, has age $y; aggregate min $y; will return an empty list {}, rather than 0, if there are no instances of an age connected to a person in the graph.

Visualisation Dashboard

There hasn't been a lot of work going into Grakn Visualisation Dashboard. That's because we're putting all our effort on getting Grakn Workbase together - and it will be released very soon!

  1. Added a command option to the main menu to display a window that lists all possible commands/shortcuts.

  2. Added a button in the node settings panel to reset attribute labels of a concept.

  3. Added a relationship settings to allow users to configure whether to autoload role players and also to limit them.

Continuous Integration

This may be more important/relevant to the Grakn developers than Grakn users right now, but as we prepare our codebase to be contributor-friendly, cleaning up our Continuous Integration framework is one of the first important steps.

  1. All Grakn tests are now tested on CircleCI, which can be found at: circleci.com/gh/graknlabs.

  2. More bootup, distribution and release integration tests have been added.

  3. Notifications of GitHub Pull Requests and CI test run status can also be found in #github channel of the Grakn Community Slack (you can join here if you haven't: grakn.ai/slack).

@lolski lolski released this Mar 29, 2018 · 513 commits to master since this release

Assets 4

Core

  • Introduced a new, experimental version of the Java Driver based on the gRPC protocol
  • Support auto-completing roles and rules in shell

Graql, Reasoning, Analytics

  • Provide "as" syntax for overriding roles
  • Add new syntax for cluster and centrality query
  • Support direct-isa syntax in the Java API and Graql
  • Changes the match ... insert behaviour to project on the match results
  • Improved reasoning explanation

Documentation

  • Added 'Getting started on Google Cloud' documentation
  • Added docs for deleting relationships and attributes
  • Fixed header sizes in ddl.md to match dml.md
  • Fixed incorrect Graql and Java examples in the docs
  • Added git clone and build instructions to README.md
  • Academy: update term to be more consistent
  • Fixed anchor jump on documentation
  • Added EULA
  • Fixed syntax error in one of the examples

Bugfixes

  • Fixed an issue with the analytics query when host is set to anything other than 'localhost'
  • Skip post processing if the system keyspace isn't yet ready
  • Adjustable storage and log directory
  • Disabled the janus cache
  • Optimise query planner: infer relationship type from schema
  • Fixed a validation bug which can prevent a valid transactions from being committed.
  • Visual tweak: Updated the component name which is shown during bootup, from Grakn to Engine
  • Fixed incorrect error message regarding datatypes
  • Removed the ability of overriding GRAKN_HOME environment variable
  • Fixed an issue in the dashboard to show long results properly on multiple lines

@lolski lolski released this Feb 16, 2018 · 682 commits to master since this release

Assets 4

Core

  • Switched to AGPL License
  • Revamping the architecture of post-processing, resulting in faster processing and shutdown step

Analytics

  • Introducing a new analytics method, k-core

Graql

  • Relationships can no longer be committed without a roleplayer
  • When $x relates $y, infer $y sub role

Documentation

  • Documentation for the undefine Graql query
  • Grakn Academy no longer requires a VM
  • New documentation for the client library grakn-python and grakn-haskell
  • Updated path query documentation

Bugfixes

  • Better error logging in case of fatal error
  • graql prints error directly to stdout rather than to grakn.log
  • Fixed a bug where Redis would stop working when host is set to anything other than localhost
  • Make analytics count return the same result as aggregate count
  • Fixed a bug in answer consumption
  • Fixed some query cache inconsistencies
  • Fixed several REST API bugs
  • Fixed issue with consuming answers to bodies of atomic rules
  • Make behaviour of atomic states consistent w.r.t. type inference

@fppt fppt released this Dec 14, 2017 · 809 commits to master since this release

Assets 2

Key Features

Core API

Core java api providing the following functionality:

  • Transactional Java API supporting read, write, and batch transactions.
  • Supports building N-ary relationships.
  • Automatic reification of binary relationships when expanding them.
  • Mutable and extendable schema after loading data.
  • Creation of Graql queries

Graql

A high-level declarative query language for Grakn:

  • Data definition language formed of define and undefine queries
  • Data manipulation language formed of get, aggregate, insert and delete queries
  • compute queries supporting graph analytics
  • Real-time, query-based reasoning based on rules with lazy, iterable execution
  • Full support for general recursive queries

Graql Analytics

Built-in distributed OLAP queries, including:

  • Various statistics methods, such as min, max, sum, std, mean and median
  • path query that can find one or all shortest paths between two entities
  • degree query for computing degree, a commonly used centrality measure that is suitable for knowledge base
  • cluster query for identifying connected components in the knowledge base

Grakn Engine

Server providing:

  • REST API for querying and mutating knowledge base
  • Clustering and scaling
  • Bulk loading functionality

Dashboard

Web based knowledge-base visualiser:

  • Ability to query knowledge base
  • Traverse visual representation of query results
  • Visualise detailed information about every concept in the KB
  • Get visual explanation of inferred relationships

Migration

Graql based templating language enabling automatic migration from various sources:

  • Automated migration from JSON
  • Automated migration from SQL
  • Automated migration from CSV
  • Automated migration from XML

Change Log Since 0.18

  • Bug fixes to Core API and Graql
  • Improved error messages
  • Performance improvements
  • Docker support temporarily unavailable. We will be addressing this in the next release.

@fppt fppt released this Nov 27, 2017 · 1086 commits to master since this release

Assets 2

Core

  • Significant performance improvements to reasoning engine
  • Reasoning enabled by default
  • General bug fixes and improvements

Graql

  • All queries will now result in executing rules unless explicitly stated to not do so.

Migration

  • Larger files are now streamed which means batch loading can now support larger file migrations.

@fppt fppt released this Nov 10, 2017 · 1101 commits to master since this release

Assets 2

Hot Fix

This release fixes an issue in the startup script which would prevent Grakn from starting smoothly on certain systems

Below are the features of the original 0.17.0 release:

Setup Changes

  • Grakn Server now started with following command: grakn server start
  • Graql console started with following command: graql console
  • Migration started using following command: graql migrate

Grakn Academy

Introducing Grakn Academy. A full online course which teaches you all the basics of Grakn. From Grakn zero to Grakn hero.

Graql

  • Resource renamed to Attribute
  • ResourceType renamed to AttributeType
  • Relation renamed to Relationship
  • RelationType renamed to RelationshipType
  • Types and Schema Concepts are no inserted with define. Data continues to be inserted with insert. For example:
define person sub entity;
insert $s isa person;
  • Rules are now part of the schema. For example defining a rule is now:
define my-rule sub rule when {...} then {...};
  • Match queries now known as get queries must now end with get. For example:
match $x isa person; get; 
match $x has name $xn; ($x, mother: $y); $y has name $yn; get $xn, $yn; 
  • Introduced as syntax for more easily accessing links between entities and attributes. This is to make data provenance easier. For example:
match $x isa person has name "Bob" as $r; get $r; 
  • Optimised queries for getting attributes of entity
  • Optimised queries which involve traversing subs
  • Improved error messages on Graql shell
  • Several reasoning optimisations and bug fixes

Core

  • GraknGraph renamed to GraknTx
  • Optimised insertions
  • Bug fixes when creating keyspaces
  • Changed to OpenJDK 8. Oracle JDK no longer required
  • Core API returns Streams rather than Collections
  • Updated Persistence layer
  • Batched commit log submissions in order to improve performance
  • Faster transaction and session initialisation using cached properties.
  • Remove redundant validation leading to write improvements

Analytics

  • Analytics now takes into account optimised attribute structures

Engine

  • Improved internal queuing system
  • More user notifications during migration processes
  • Fixed memory leak in task queue
  • Fixed several migration failures

Dashboard

  • Several Bug Fixes
  • Changed to Yarn to manage dependencies

@fppt fppt released this Nov 2, 2017 · 1123 commits to master since this release

Assets 2

Setup Changes

  • Grakn Server now started with following command: grakn server start
  • Graql console started with following command: graql console
  • Migration started using following command: graql migrate

Grakn Academy

Introducing Grakn Academy. A full online course which teaches you all the basics of Grakn. From Grakn zero to Grakn hero.

Graql

  • Resource renamed to Attribute
  • ResourceType renamed to AttributeType
  • Relation renamed to Relationship
  • RelationType renamed to RelationshipType
  • Types and Schema Concepts are no inserted with define. Data continues to be inserted with insert. For example:
define person sub entity;
insert $s isa person;
  • Rules are now part of the schema. For example defining a rule is now:
define my-rule sub rule when {...} then {...};
  • Match queries now known as get queries must now end with get. For example:
match $x isa person; get; 
match $x has name $xn; ($x, mother: $y); $y has name $yn; get $xn, $yn; 
  • Introduced as syntax for more easily accessing links between entities and attributes. This is to make data provenance easier. For example:
match $x isa person has name "Bob" as $r; get $r; 
  • Optimised queries for getting attributes of entity
  • Optimised queries which involve traversing subs
  • Improved error messages on Graql shell
  • Several reasoning optimisations and bug fixes

Core

  • GraknGraph renamed to GraknTx
  • Optimised insertions
  • Bug fixes when creating keyspaces
  • Changed to OpenJDK 8. Oracle JDK no longer required
  • Core API returns Streams rather than Collections
  • Updated Persistence layer
  • Batched commit log submissions in order to improve performance
  • Faster transaction and session initialisation using cached properties.
  • Remove redundant validation leading to write improvements

Analytics

  • Analytics now takes into account optimised attribute structures

Engine

  • Improved internal queuing system
  • More user notifications during migration processes
  • Fixed memory leak in task queue
  • Fixed several migration failures

Dashboard

  • Several Bug Fixes
  • Changed to Yarn to manage dependencies