ProConcepts Knowledge Graph
This ProConcepts covers the KnowledgeGraph API. Knowledge graphs, also known as graph networks, consist of nodes (or "entities") linked between them with relationships. Knowledge graphs support both relational GDB (sql) queries and openCypher graph network queries. The classes and methods discussed here are delivered through the ArcGIS.Core, ArcGIS.Desktop.Mapping and ArcGIS.Desktop.KnowledgeGraph assemblies.
Language: C#
Subject: Knowledge Graph
Contributor: ArcGIS Pro SDK Team <arcgisprosdk@esri.com>
Organization: Esri, http://www.esri.com
Date: 03/22/2024
ArcGIS Pro: 3.3
Visual Studio: 2022
- Background
- Overview
- KnowledgeGraph Datastore
- KnowledgeGraph Layer
- KnowledgeGraph Link Charts
- KnowledgeGraph Graph Data Model
- KnowledgeGraph Graph Queries and Text Search
- Additional Reading
A knowledge graph, also known as a semantic network or graph network, is a representation of a network consisting of nodes (i.e. "entities") and the links or relationships between them (ideally in a "human-understandable" form). Knowledge graphs are usually implemented to bring together diverse sets of information, much of it non-spatial, to help users extract knowledge from the knowledge graph data. Knowledge graphs are also very efficient at discovering related content. As they store relationships together with the entity data, knowledge graphs can follow the path of a given relationship very quickly - much quicker than relational systems where the relationships have to be derived (usually from shared primary/foreign key values). Knowledge graphs can also acquire and integrate adjacent information through additional relationships to derive new knowledge (for the user).
Knowledge graphs are stored in graph databases (like Neo4j) and visualized as a graph structure, prompting the term knowledge “graph.” They can be made up of datasets from many different sources and each can differ in structure. The knowledge graph data model defines the types of entities and relationships that can exist in the knowledge graph along with their properties. Entities in the model represent real-world objects, concepts, or events such as a harbor, a test plan, and/or a scheduled maintenance activity or repair. Relationships in the model express how entities are associated with each other as well as the correct context. Having the correct context, for example, allows the knowledge graph to determine the difference between, say, "Apple", the brand, and "apple", the fruit. A knowledge graph allows you to discover how parts of the system are connected, which factors in the system have the biggest impact, and which hidden connections have more influence than expected. Entities with a spatial location can be connected with other entities that do not. Additionally, a special type of entity, called "Document", can be added to an ArcGIS knowledge graph. "Documents" can provide additional context for an entity (or a relationship in which it participates) as well as provide authoritative sources for the facts stored in the properties of entities and relationships. Documents can be pictures, presentations, text or Adobe Acrobat PDF files, websites, and so on.
In the ArcGIS Platform, knowledge graphs are stored in a "Graph Store". A graph store is the database that the portal's ArcGIS Knowledge Server uses to store the entities and relationships that compose the knowledge graph. Knowledge graphs can be stored as hosted knowledge graphs that are entirely managed by ArcGIS Knowledge in an Enterprise portal. Alternatively, knowledge graphs can be stored in a NoSQL user-managed data store in Neo4j. A user-managed data store must be registered with a portal's ArcGIS Knowledge Server site. There will be one registered NoSQL data store for each knowledge graph (requires ArcGIS Pro 3.0 or later). As content is added to the knowledge graph, users can explore the relationships between entities in the investigation, document their findings, create maps and link charts, and use different forms of analysis to improve their understanding of the system. Consult Get started with ArcGIS Knowledge in the ArcGIS Pro documentation for more information.
An ArcGIS knowledge graph also supports both a (traditional) relational (gdb) model and a graph model for its entities, relationships, and documents. Within the relational gdb model, entities and relationships with a spatial location are represented as features in feature classes (and feature layers) and entities and relationships with no spatial location are represented as rows in tables. Knowledge graph feature classes and tables support all the standard geodatabase functionality, such as search, select, and edit sessions.
At Pro 3.2, the public API provides the following capabilities:
- Access to the KnowledgeGraph gdb data store and gdb data model (i.e. "relational") and support for relational queries.
- KnowledgeGraph queries using openCypher query language to find subsets of the entities, relationships, documents, and provenance it contains and identify how different entities are connected.
- KnowledgeGraph text searches against indexed entity and relationship types.
- Access to the KnowledgeGraph data model and metadata to describe the different entity and relationship types, their associated properties and provenance (provenance describes the origin of the information used to create entities and relationships).
- Adding knowledge graphs to a map or scene to create a knowledge graph layer.
At Pro 3.3, the second release of the Knowledge Graph API, the following capabilities have been added:
- Bind parameters on knowledge graph queries
- Creating and retrieving knowledge graph layer ID sets
- Knowledge graph layer creation enhancements using ID sets
- Create and append to link charts
- Changing the link chart layout algorithm
The KnowledgeGraph datastore and associated core data model objects can be found in the ArcGIS.Core.Data.Knowledge namespace within the ArcGIS.Core assembly. Accessing a KnowledgeGraph datastore follows the same pattern as other supported relational data stores, namely:
- Creation of a
KnowledgeGraphdatastore connection using a connection property object - KnowledgeGraphConnectionProperties. TheKnowledgeGraphConnectionPropertiesshould point to the URI of the KnowledgeGraph service being connected to or "opened". - Access and retrieval of all contained datasets (there is one dataset per spatial and non-spatial entity and relationship type). Use the standard
T OpenDataset<T>(string name)method on the (knowledge graph) data store where "T" is either aFeatureClassorTable. - Access and retrieval of dataset definitions via the standard
IReadOnlyList<T> GetDefinition<T>()andIReadOnlyList<T> GetDefinitions<T>()data store methods where "T" can be eitherFeatureClassDefinitionorTableDefinition.
** Only feature classes, feature class definitions, tables, and table definitions are supported in a knowledge graph
KnowledgeGraph datastores also have a SpatialReference accessed via a GetSpatialReference method. All feature classes within a knowledge graph must share a common coordinate system.
The following snippet shows creating a connection to a KnowledgeGraph datastore and retrieving all of its datasets and dataset definitions:
string URL = @"https://acme.com/server/rest/services/Hosted/Acme_KG/KnowledgeGraphServer";
var kg_props =
new KnowledgeGraphConnectionProperties(new Uri(URL));
using(var kg = new KnowledgeGraph(kg_props)) //connect
{
var fc_names = new List<string>();
var tbl_names = new List<string>();
int c = 0;
//get "relational" definitions and datasets - only feature classes, tables,
//and associated definitions are supported
System.Diagnostics.Debug.WriteLine("\r\nFeatureClassDefinitions:");
var fc_defs = kg.GetDefinitions<FeatureClassDefinition>();
foreach (var fc_def in fc_defs)
{
System.Diagnostics.Debug.WriteLine(
$" FeatureClassDefinition[{c++}] {fc_def.GetName()}, {fc_def.GetAliasName()}");
fc_names.Add(fc_def.GetName());
}
System.Diagnostics.Debug.WriteLine("\r\nFeature classes:");
c = 0;
foreach (var fc_name in fc_names)
{
using (var fc = kg.OpenDataset<FeatureClass>(fc_name))
System.Diagnostics.Debug.WriteLine(
$" FeatureClass[{c++}] {fc.GetName()}");
}
System.Diagnostics.Debug.WriteLine("\r\nTableDefinitions:");
c = 0;
var tbl_defs = kg.GetDefinitions<TableDefinition>();
foreach (var tbl_def in tbl_defs)
{
System.Diagnostics.Debug.WriteLine(
$" TableDefinitions[{c++}] {tbl_def.GetName()}, {tbl_def.GetAliasName()}");
tbl_names.Add(tbl_def.GetName());
}
System.Diagnostics.Debug.WriteLine("\r\nTables:");
c = 0;
foreach (var tbl_name in tbl_names)
{
using(var tbl = kg.OpenDataset<Table>(tbl_name))
System.Diagnostics.Debug.WriteLine(
$" Table[{c++}] {tbl.GetName()}");
}
}A KnowledgeGraphLayer, KG layer, is a composite layer with a knowledge graph as its data source. Knowledge graphs can be added to either a map, scene, or link chart. In a map, the KG layer will contain one child feature layer per entity and relationship type with a spatial column ("shape") and one child standalone table for each entity and relationship type that does not. In a link chart, the KG layer will contain one child aggregation layer (a "LinkChartFeatureLayer") per relate and entity type. Regardless of the type of child layers, all child feature layers of the KG layer share the same spatial reference and can be manipulated via the UI and API with certain restrictions (see next paragraph). Their behavior and appearance can be controlled by modifying their properties and symbology respectively.
With regards to restrictions, KG layer child layers (whether feature layers and standalone tables in a map or aggregation sub-layers within a link chart) can only be added as children of their parent KG layer during KG layer creation. You cannot "re-parent" a KG child layers or standalone tables (for example, by attempting to move them into a different group layer or add them as child content of the map container directly). In addition, other layers can not be "moved" into a KG layer - either via code or interactively on the UI (eg via drag/drop). Essentially the KG layer is a fixed container. Sub-layers within link charts are explained in more detail here: KnowledgeGraphLayer in Link Chart.
Additionally, within a map or scene, attempting to create a feature layer or standalone table using an entity or relationship type feature class or table as its source will fail with an ArgumentException and LayerFactory.Instance.CanCreateLayer | CanCreateStandaloneTable will return false. Feature layers and standalone tables sourced on KG datasets can only be created as children of the KG layer itself.
There are also a few differences between the way KG layers behave on a map vs on a link chart. Mostly it is to do with how empty entity and relate types are represented. As you will see in the discussions for KnowledgeGraph Layer ID Sets and KnowledgeGraph Link Charts, KG layers within maps cannot be empty. They must always contain at least one child feature layer layer or table that has content. KG layers within maps do not contain empty layers or standalone tables (unless modified after the fact by a query definition, for example) in the TOC. Empty child layers/tables are simply not added to the KG layer when it is created. KG layers within link charts, however, always contain all their child layers - one per entity and relate type - whether they are empty (i.e. are displaying no records) or not. Additional content can be "appended" to the KG layer over its lifetime. This usually leads to a different looking TOC for the same KG layer when it is added to a map vs added to a link chart - even if its underlying id set is the same.
The specifics of creating KG layers and id sets for KG Layers are explained further in the following sections.
As with all other layer types, there is a KnowledgeGraphLayerCreationParams class to use with the LayerFactory.Instance.CreateLayer method to add a KnowledgeGraph layer to a map. The KnowledgeGraphLayerCreationParams has constructors for specifying the KnowledgeGraph with either its server URI or knowledge graph datastore. Set the templated type on the LayerFactory.Instance.CreateLayer<T> call to be KnowledgeGraphLayer. By default, the layer will contain all the data in the KnowledgeGraph.
Note that using the LayerFactory.Instance.CreateLayer method to add a KnowledgeGraph layer when the hosting map is a link chart will throw an exception. Instead use a MapFactory.Instance.CreateLinkChart method to add a KnowledgeGraph layer to a new link chart. Refer to Creating a Link Chart.
The following code shows how to create a knowledge graph layer in a map:
string URL = @"https://acme.com/server/rest/services/Hosted/Acme_KG/KnowledgeGraphServer";
var map = MapView.Active.Map;
//Use a KnowledgeGraphLayerCreationParams with a KnowledgeGraph
QueuedTask.Run(() => {
var kg_props = new KnowledgeGraphConnectionProperties(new Uri(URL));
using(var kg = new KnowledgeGraph(kg_props)) {
//Can also use the URI of the KG service as the parameter...
// var kg_params = new KnowledgeGraphLayercreationParams(new Uri(URL))
var kg_params = new KnowledgeGraphLayercreationParams(kg) {
Name = "Acme KnowledgeGraph"
}
//By default, this creates a KG layer with a child feature layer/standalone table
//per entity and relate type (excluding provenance)
var kg_layer = LayerFactory.Instance.CreateLayer<KnowledgeGraphLayer>(kg_params, map);To create a KG layer containing just a subset of the entity and relate types, first create an "id set" or KnowledgeGraphLayerIDSet - explained in detail in the KnowledgeGraph Layer ID Sets section. KG layer id sets are similar to "Selection Sets" - they contain a list of the entity and relate types to be added to the KG layer along with a list of ids for each type (in the id set). The KnowledgeGraphLayer's component child layer and table content in the map will be limited to just those types and records specified in the id set. For example:
QueuedTask.Run(() => {
//create a KG layer with just two types - one entity type and one relate type in this case
using(var kg = ....) {
//Get the names of the types to be added as a subset to the (new) KG Layer
var kg_datamodel = kg.GetDataModel();
var first_entity = kg_datamodel.GetEntityTypes().Keys.First();//arbitrarily first entity type name
var first_relate = kg_datamodel.GetRelationshipTypes().Keys.First();//arbitrarily first relate type name
//create the dictionary for the entries
var dict = new Dictionary<string, List<long>>();
//Each entry consists of the type name and corresponding lists of ids
//Empty or null list means "add all records" - saves having to retrieve all the ids just to add "all records"
dict.Add(first_entity, new List<long>());//Empty list means all records
dict.Add(first_relate, null);//Null list means all records
//or specific records - however the ids are obtained
//dict.Add(entity_or_relate_name, new List<long>() { 1, 5, 18, 36, 78});
//Create the id set...
var idSet = KnowledgeGraphLayerIDSet.FromDictionary(kg, dict);
//Create the layer params and assign the id set
var kg_params = new KnowledgeGraphLayerCreationParams(kg) {
Name = "KG_With_ID_Set",
IsVisible = false,
IDSet = idSet //assign to the "IDSet" property
};
//KG Layer will be created in the map containing just the "child" types present in the id set
var kg_layer = LayerFactory.Instance.CreateLayer<KnowledgeGraphLayer>(
kg_params, map);
//Note: A KG Layer within a link chart always includes an entry for all its child content
//whether it was specified as part of its id set or not. Attempting to create a feature layer (or standalone table) sourced with a child entity/relate type will throw an exception. KG child feature layers and standalone tables can only be added as a child component of a KG layer in a map as part of the KG layer creation. For example:
QueuedTask.Run(() => {
//Feature class and/or standalone tables representing KG entity and
//relate types can only be added to a map (or link chart) as a child
//of a KnowledgeGraph layer (as part of the KnowledgeGraphLayer creation)....
//For example:
using(var kg = ....) {
var fc = kg.OpenDataset<FeatureClass>("Some_Entity_Or_Relate_Name");
try
{
//Attempting to create a feature layer containing the returned KG fc
//is NOT ALLOWED - can only be a child of a KG layer
var fl_params_w_kg = new FeatureLayerCreationParams(fc);
//CanCreateLayer will return false
if (!(LayerFactory.Instance.CanCreateLayer<FeatureLayer>(
fl_params_w_kg, map)))
{
System.Diagnostics.Debug.WriteLine(
$"Cannot create a feature layer for {fc.GetName()}");
return;
}
//Attempting to call "CreateLayer" will throw an exception - same is true
//for standalone tables
LayerFactory.Instance.CreateLayer<FeatureLayer>(fl_params_w_kg, map);
}
catch (Exception ex)
{
System.Diagnostics.Debug.WriteLine(ex.ToString());
}
//To add only the specific entity or relate type, use a KG layer create params and id set...
var dict = new Dictionary<string, List<long>>();
dict.Add(fc.GetName(), new List<long>());//default to all records
var kg_params = new KnowledgeGraphLayerCreationParams(kg)
{
Name = $"KG_With_Just_{fc.GetName()}",
IsVisible = false,
IDSet = KnowledgeGraphLayerIDSet.FromDictionary(kg, dict)
};
//KG layer will contain just the specified child feature layer in this case
var kg_layer = LayerFactory.Instance.CreateLayer<KnowledgeGraphLayer>(
kg_params, map);
...To retrieve the currently connected KnowledgeGraph datastore from the knowledge graph (composite) layer use the GetDataStore method:
var kg_layer = MapView.Active.Map.GetLayersAsFlattenedList()
.OfType<ArcGIS.Desktop.Mapping.KnowledgeGraphLayer>()
.FirstOrDefault();
QueuedTask.Run(() => {
//use the KG layer...
using(var kg = kg_layer.GetDatastore()) {
...Refer to Creating a Link Chart for how to create a KG layer within a link chart.
All KG layers (regardless of whether they are a contained in a map or a link chart) include an id set. The id set, or KnowledgeGraphLayerIDSet, acts as a filter and contains a set of entries, one for each entity and/or relate type that contains the records to be included in the KG layer. The id set is not unlike a selection set in this regard.
ID sets contain both a list of object ids and a list of the underlying entity and relate uuids (ie "guids"). In the special case of NoSQL user-managed knowledge graphs in Neo4j, where there may be no persisted object ids, the object ids will be created synthetically by the underlying Geodatabase workspace/datastore. As a result, object ids retrieved from id sets of layers in NoSQL user-managed graphs, which do not store an object id internally, can change from session to session. Uuids, for all graph types, are always permanent and never change. Object ids can be extracted from an id set via a KnowledgeGraphLayerIDSet.ToOIDDictionary call and uuids ("guid" strings) can be extracted via a KnowledgeGraphLayerIDSet.ToUIDDictionary call.
You can retrieve the id set for a KG layer using the GetIDSet method:
QueuedTask.Run(() =>
{
var idSet = kgLayer.GetIDSet();
// is the set empty?
var isEmpty = idSet.IsEmpty;
// get the count of named object types
var countNamedObjects = idSet.NamedObjectTypeCount;
// does it contain the entity "Species";
var contains = idSet.Contains("Species");
// get the idSet as a dictionary of namedObjectType and oids
var oidDict = idSet.ToOIDDictionary();
var speciesOIDs = oidDict["Species"];
// alternatively get the idSet as a dictionary of
// namedObjectTypes and uuids
var uuidDict = idSet.ToUIDDictionary();
var speciesUuids = uuidDict["Species"];
});Id sets can created in one of three ways:
- Using pre-sets from the KnowledgeGraphFilterType enum with KnowledgeGraphLayerIDSet.FromKnowledgeGraph
- Using a dictionary containing the list of the type names and their corresponding id values (of the records to be included) that is converted to an id set via one of the KnowledgeGraphLayerIDSet.FromDictionary methods.
- From a SelectionSet directly using the static KnowledgeGraphLayerIDSet.FromSelectionSet method.
Three examples follow. First, using the presets. In the following code an id set that contains all the entity data in the knowledge graph is being created with KnowledgeGraphFilterType.AllEntities and KnowledgeGraphLayerIDSet.FromKnowledgeGraph:
string url =
@"https://acme.server.com/server/rest/services/Hosted/AcmeKnowledgeGraph/KnowledgeGraphServer";
QueuedTask.Run(() =>
{
try
{
using (var kg = new KnowledgeGraph(new KnowledgeGraphConnectionProperties(new Uri(url))))
{
//or use AllRelationships for just relationship or AllNamedObjects for all content to be included
var idSet = KnowledgeGraphLayerIDSet.FromKnowledgeGraph(kg, KnowledgeGraphFilterType.AllEntities);
//Todo - use the id set to create a KnowledgeGraphLayer in a map or a new link chart
};
}
catch (Exception ex) {
System.Diagnostics.Debug.WriteLine(ex.ToString());
}
});Second, an id set is being constructed using a dictionary. The dictionary contains the explicit list of types and corresponding records to be included and is converted to an id set using KnowledgeGraphLayerIDSet.FromDictionary(...). Specifying a null or empty list for a named type means include every record for that named type rather than having to explicitly specify all the records in the list. Here the type names are hardcoded but could, instead, be retrieved from a query or selection:
var dict = new Dictionary<string, List<long>>();
dict.Add("person", new List<long>()); //Empty list means all records
dict.Add("made_call", null); //null list means all records
// or specific records - however the ids are obtained
dict.Add("phone_call", new List<long>() { 1, 5, 18, 36, 78});
var idSet = KnowledgeGraphLayerIDSet.FromDictionary(kg, dict);Third, the final way of constructing an id set is from a SelectionSet using the static KnowledgeGraphLayerIDSet.FromSelectionSet method. If the selection set does not contain any selections for KG entity and/or relate types then KnowledgeGraphLayerIDSet.FromSelectionSet will return null.
QueuedTask.Run(() =>
{
// get the selection set
var sSet = map.GetSelection();
// translate to an IDSet
// if the selectionset does not contain any KG entity or relationship records
// then idSet will be null
var idSet = KnowledgeGraphLayerIDSet.FromSelectionSet(sSet);
// if there was no KG data in the selection set then don't proceed
if (idSet == null)
return;
//etc
});Id sets are used to define a KG's layer content in three distinct scenarios:
- Creating a KG layer on a map
- Creating a KG layer on a new link chart
- Appending data to an existing KG layer in a link chart
Firstly, when creating a KG layer on a map, use the IDSet property on the KnowledgeGraphLayerCreationParams object. See Creating a KnowledgeGraph Layer for a Map for discussion and snippets. By default the IDSet property is set to retrieve all data from the knowledge graph. Modify this property to populate the KG layer with a subset of the knowledge graph data.
Note: You cannot assign a null or empty id set to the KnowledgeGraphLayerCreationParams.IDSet parameter: Attempting to use a KnowledgeGraphLayerCreationParams where the IDSet parameter has been nulled out or assigned an empty id set will cause an exception to be thrown when it is used to create a new KG layer with LayerFactory.Instance.CreateLayer<T>(...) and LayerFactory.Instance.CanCreateLayer will return false. Empty KG layers cannot be added to a map.
Secondly, to create a KG layer on a new link chart; specify the id set as a parameter to one of the MapFactory.CreateLinkChart methods. Refer to Creating a Link Chart for discussion on how to create a KG layer within a link chart.
Specifying a null or empty id set to the MapFactory.CreateLinkChart method is valid. It will create a KG layer containing child layers for each of the entity and relate types in the knowledge graph. But these child layers will be empty "placeholder" layers - they will not contain any content.
Finally, to append data to an existing KG layer in a link chart; specify the id set as a parameter to the AppendToLinkChart method. Refer to Appending data to a Link Chart.
As with creating a KG layer on a map, you cannot pass a null or empty id set to the AppendToLinkChart method. Attempting to do so will cause an exception to be thrown. Use the CanAppendToLinkChart method prior to calling AppendToLinkChart.
In all of the above situations, the named types in an id set entry are case-sensitive. A KnowledgeGraphLayerException exception will be thrown when the id set is used if there are named types specified in the id set that do not match the named types in the knowledge graph. A KnowledgeGraphLayerException exception, if thrown, will contain a list of any named object type names that were invalid.
// running on QueuedTask
var dict = new Dictionary<string, List<long>>();
dict.Add("person", new List<long>()); //Empty list means all records
dict.Add("made_call", null); //null list means all records
// or specific records - however the ids are obtained
dict.Add("phone_call", new List<long>() { 1, 5, 18, 36, 78 });
// make the id set
var idSet = KnowledgeGraphLayerIDSet.FromDictionary(kg, dict);
try
{
//Create the link chart and show it
var linkChart = MapFactory.Instance.CreateLinkChart(
"KG With ID Set", kg, idSet);
FrameworkApplication.Panes.CreateMapPaneAsync(linkChart);
}
catch (KnowledgeGraphLayerException e)
{
// get the invalid named types
// remember that the named types are case-sensitive
var invalidNamedTypes = e.InvalidNamedTypes;
// do something with the invalid named types
// for example - log or return to caller to show message to user
}
Empty Lists
Empty id lists within an id set are interpreted differently when creating a KG layer vs retrieving an id set from an existing KG layer in a map or link chart.
-
When creating a KG layer on a map or a link chart, setting the id list to empty or null for a given type means "add all records for that type". The intent being to make it easy for addin developers to create an id set to be used to add all records for a given type. Addin developers can simply use an empty or null list for this purpose without having to extract all ids for all records of a particular type from the database. However, an id set containing an empty or null list for a particular type is processed differently depending upon whether the KG layer is to be created on a link chart or on a map. In the case of creating a KG layer on a link chart, internally, the id set is translated to an explicit list of all ids present in the graph for the given type at the time of the operation. If the id set is used to create a KG layer on a map, then a null or empty list for a type is not translated and remains empty and always means all records for that type.
-
When retrieving an id set, the content of the id set can be different depending on whether the KG layer is in a map (or scene) vs in a link chart. For a map, if all records for a particular type were added to the KG layer, then, same as on the create, the retrieved list of ids for "that" type will be empty. However, if an id set is retrieved from a KG layer on a link chart, the lists of ids will always be populated with the explicit records being displayed regardless of whether the id set was created using null or empty id lists or not.
The key reason for this difference comes down to how a KG layer behaves in a map versus in a link chart when the underlying graph is changed. Assume that a KG layer on a map and on a link chart includes all records for the same number of child entity and relate types in the graph. Both layers were created with an id set using null/empty for each of the particular id lists. The lists of ids for the various types in the id set extracted from the KG layer in the map will still be empty (count = 0) whereas the lists of ids for the same types in the id set extracted from the KG layer in the link chart will now be populated with the specific ids of the records for all of the types (count = n).
At some point in time additional records are added to the graph (eg via an edit). On a refresh, the new content will automatically be displayed on the map but will not be displayed on the link chart. As the id set on the KG layer uses empty lists for "all records", it always implicitly includes all records in the graph (for the relevant types), even though some of the records were added after the KG layer was created. However, because the id set on the KG layer in the link chart uses an explicit list of ids per type, no new records will be shown. Only the records explicitly listed in the id set are displayed. To display additional content on a link chart, additional ids have to be added or "appended" to the id set of the layer in order to be shown. Appending records to a link chart is covered in Appending data to a Link Chart.
In this example, an id set is retrieved from a KG layer which includes all "default" content (ie all records for all types in the graph) - first from a map first and then from a link chart
//at an earlier point in time, one id set was defined for use in creating
//both a KG layer on a map and a KG layer on a link chart...
//We create the two KG layers with _all_ records for
//_all_ entity and relate types in the graph with our id set...
var idSet = KnowledgeGraphLayerIDSet.FromKnowledgeGraph(
kg, KnowledgeGraphFilterType.AllNamedObjects);
//We retrieve the newly created KG layers from both the map and link
//chart and examine the count of ids in their respective id set
//id lists per type...
//Get the KG layer from the map first...
var kg_layer_in_a_map = map.GetLayersAsFlattenedList()
.OfType<KnowledgeGraphLayer>().First();
//assume the KG layer includes all content for the graph
var idset = kglayer.GetIDSet();
var dict = idset.ToOIDDictionary();//convert to a dictionary
//print out the names of all types + count of records per type in the id set
var entries = new List<string>();
foreach(var kvp in dict)
entries.Add( $"{kvp.Key} {kvp.Value.Count} recs");
//In the case of the map, the id set contains an empty list for each type, same
//as was on the input - i.e. "kvp.Value.Count" above evaluates to "0" for all types.
System.Diagnostics.Debug.WriteLine(string.Join(",", entries.ToArray()));
//repeat for the link chart
//note: link chart uses the "map" class, same as a 2d map or 3d scene
var kg_layer_in_a_link_chart = mapLinkChart.GetLayersAsFlattenedList()
.OfType<KnowledgeGraphLayer>().First();
//as before, the KG layer on the link chart includes all content for the graph
var idset = kg_layer_in_a_link_chart.GetIDSet();
var dict = idset.ToOIDDictionary();//convert to a dictionary
//print out the names of all types + count of records per type in the id set
var entries = new List<string>();
foreach(var kvp in dict)
//entries from the link chart include all of the ids listed explicitly
entries.Add( $"{kvp.Key} {kvp.Value.Count} recs");
//In the case of the link chart, the id set contains an populated lists for each type -
//even though empty lists were used to create it. "kvp.Value.Count" = "n" for each type
System.Diagnostics.Debug.WriteLine(string.Join(",", entries.ToArray()));Note: Because an empty or null list means "add all records" for "that" type when creating a KG layer, you cannot explicitly define an empty "type" in an id set. To define an empty type in a new id list, simply omit an entry for that type in the id set and no records for it will be added to the KG layer.
A link chart can be created to visualize, analyze, and explore a knowledge graph's content. Link charts are explained in great detail in the ArcGIS Pro online help: ArcGIS Pro Help, Link Charts.
Within the ArcGIS Pro SDK, the link chart is modelled as a special type of Map and has a MapType equal to MapType.LinkChart. Just like a standard map and map view, the link chart view appears in the Maps folder collection in the Catalog pane and Catalog view. Use the standard Map, MapView and MapProjectItem model classes to access link charts, making use of the MapType property as appropriate.
Here is an example showing how to retrieve link charts items from the project.
// find all the project items that are link charts
var linkChartItems = Project.Current.GetItems<MapProjectItem>().Where(pi => pi.MapType == MapType.LinkChart);
// find a link chart project item by name
var linkChartItem = Project.Current.GetItems<MapProjectItem>()
.FirstOrDefault(pi => pi.Name == "Acme Link Chart");Once you have the project item, use the GetMap method to obtain the link chart map (same as with a "regular" map or scene):
// must be on the MCT - used QueuedTask.Run
// find a link chart project item by name
var linkChartMapItem = Project.Current.GetItems<MapProjectItem>()
.FirstOrDefault(pi => pi.Name == "Acme Link Chart");
var linkChartMap = linkChartMapItem.GetMap();
if (linkChartMap.MapType != MapType.LinkChart)
return;Here's an example of determining if the active map view contains a link chart:
// get the active map
var map = MapView.Active.Map;
// check the MapType to determine if it's a link chart map
var isLinkChart = map.MapType == MapType.LinkChart;
// or you could use the following
// var isLinkChart = map.IsLinkChart;Here's another example of finding a link chart map from the set of open panes in the application; this time using the MapView.IsLinkChartView function:
var mapPanes = FrameworkApplication.Panes.OfType<IMapPane>().ToList();
var mapPane = mapPanes.FirstOrDefault(
mp => mp.MapView.IsLinkChartView && mp.MapView.Map.Name == "Acme Link Chart");
var linkChartMap = mapPane.MapView.Map;There are a few key differences between a KG layer in a link chart versus in a map:
- A link chart MUST always contain a KG layer. It is not possible to remove this KG layer from the link chart (either via the application or the API).
- A link chart can contain one, and only one KG layer. Not only can you not delete the KG layer from a link chart, you cannot add another KG layer to the link chart as well. There is, however, no limit to the number of KG layers you can add to a map beyond whatever are relevant practical considerations.
- A KG layer in a link chart always shows child (aggregation*) layer entries in the TOC for all its child content to include empty content not specified in the id set (id sets do not include entries for "empty" types). In a map, a KG layer omits entries from its TOC for child content that is empty same as is done in the id set. (*Aggregation layers in link charts are used for controlling the grouping and symbology of the relationships between entities - refer to KnowledgeGraphLayer in Link Chart).
The following sections discuss link charts in more detail including how to create a link chart.
Use one of the MapFactory.Instance.CreateLinkChart methods to create a new link chart. These methods will create a new link chart map and add it to the Maps folder collection in the Catalog window in the project. It will also automatically add a KG layer into the link chart map. Note that an empty link chart map can NOT exist; it must contain a KG layer. Create and populate a KnowledgeGraphLayerIDSet to control the content of the Knowledge Graph layer and pass this as a parameter to the CreateLinkChart method. As with map creation, open a newly created link chart map via the FrameworkApplication.Panes.CreateMapPaneAsync(map) method.
Note: You cannot use MapFactory.Instance.CreateMap (with MapType.LinkChart) to create a link chart. MapFactory.Instance.CreateMap will throw an ArgumentException when used with MapType.LinkChart - you must use MapFactory.Instance.CreateLinkChart(...) instead.
Here are some examples of how to create a link chart.
To create a link chart with all content from the KnowledgeGraph, create a KnowledgeGraphLayerIDSet using the KnowledgeGraphFilterType.AllNamedObjects value.
string url =
@"https://acme.server.com/server/rest/services/Hosted/AcmeKnowledgeGraph/KnowledgeGraphServer";
var uri = new Uri(url);
// create and open a new link chart
Map linkChart = await QueuedTask.Run(() => {
//build the idSet using KnowledgeGraphFilterType.AllNamedObjects
var idSet = KnowledgeGraphLayerIDSet.FromKnowledgeGraph(
uri, KnowledgeGraphFilterType.AllNamedObjects);
// create a new link chart
Map lc = MapFactory.Instance.CreateLinkChart(
"Acme Link Chart", uri, idSet);
return lc;
});
//should be called on the UI
await ProApp.Panes.CreateMapPaneAsync(linkChart);To create an empty KG layer on the link chart pass in null for the id set parameter. An empty KG layer will contain all relevant sub-layers/aggregation layers for each entity and relationship in the KnowledgeGraph but the sublayers will have no content. Recall: An empty KG layer is valid for a link chart (but is not valid for a map):
// create and open a new link chart
Map emptyLinkChart = await QueuedTask.Run(() => {
return MapFactory.Instance.CreateLinkChart("Empty Link Chart", uri, null);
});
await ProApp.Panes.CreateMapPaneAsync(emptyLinkChart);To create a link chart with just the entities from the KnowledgeGraph create a KnowledgeGraphLayerIDSet using the KnowledgeGraphFilterType.AllEntities value:
string url =
@"https://acme.server.com/server/rest/services/Hosted/AcmeKnowledgeGraph/KnowledgeGraphServer";
var uri = new Uri(url);
// create and open a new link chart
Map linkChart = await QueuedTask.Run(() => {
// build the idSet of all entities
var idSet = KnowledgeGraphLayerIDSet.FromKnowledgeGraph(uri, KnowledgeGraphFilterType.AllEntities);
// create a new link chart using the idSet
var lc = MapFactory.Instance.CreateLinkChart(
"Just Entities Link Chart", uri, idSet);
return lc;
});
//should be called on the UI
await ProApp.Panes.CreateMapPaneAsync(linkChart);To create a link chart with a mixture of entity and relate records, build a more complex KnowledgeGraphLayerIDSet from a dictionary of named object types and their corresponding list of record ids (similar to how a selection set is constructed).
string url =
@"https://acme.server.com/server/rest/services/Hosted/AcmeKnowledgeGraph/KnowledgeGraphServer";
var uri = new Uri(url);
// create and open a new link chart
Map linkChart = await QueuedTask.Run(() =>
{
// build the dictionary of named object types and records
var IDDict = new Dictionary<string, List<long>>();
IDDict.Add("entity1", null); // adds all records of entity1
IDDict.Add("entity2", new List<long>() {1, 2, 34, 5});//adds records explicitly
IDDict.Add("relationship1", new List<long>()); // adds all records of relationship1
// build the idSet from the dictionary
var idSet = KnowledgeGraphLayerIDSet.FromDictionary(uri, IDDict);
// create a new link chart using the idSet
var lc = MapFactory.Instance.CreateLinkChart(
"Acme Link Chart", uri, idSet);
return lc;
});
//should be called on the UI
await ProApp.Panes.CreateMapPaneAsync(linkChart);You can also create an id set from a SelectionSet and add those entities and relationship to a new link chart.
string url =
@"https://acme.server.com/server/rest/services/Hosted/AcmeKnowledgeGraph/KnowledgeGraphServer";
var uri = new Uri(url);
// create and open a new link chart
Map linkChart = await QueuedTask.Run(() => {
// get the selection set
var sSet = map.GetSelection();
// translate to an id set. note that if the selectionset does not contain any
//KG entity or relationship records then the resulting idSet will be null
var idSet = KnowledgeGraphLayerIDSet.FromSelectionSet(sSet);
if (idSet == null)
return;
// create a new link chart using the idSet
var lc = MapFactory.Instance.CreateLinkChart(
"Acme Link Chart", uri, idSet);
return lc;
});
//should be called on the UI
await ProApp.Panes.CreateMapPaneAsync(linkChart);By default, the entities and relationships added to the new link chart are arranged using the layout algorithm "standard organic", KnowledgeLinkChartLayoutAlgorithm.Organic_Standard. However, the CreateLinkChart method also gives you the option to use an existing link chart as a template (for formatting, labeling, popups, etc.) in order to create a link chart using the layout in use by the template link chart. The layout currently in use on the template will be applied to the new link chart, to include its symbology. The KG layer referenced in the link chart template map item must point to the same KnowledgeGraph service as the new KG layer being created otherwise an ArgumentException will be thrown.
Here is a code example showing how to create a link chart using an existing link chart as a template:
string url =
@"https://acme.server.com/server/rest/services/Hosted/AcmeKnowledgeGraph/KnowledgeGraphServer";
QueuedTask.Run(() => {
// find the existing link chart by name
var projectItem = Project.Current.GetItems<MapProjectItem>()
.FirstOrDefault(pi => pi.Name == "Acme Link Chart");
var linkChartMap = projectItem?.GetMap();
if (linkChartMap == null)
return;
// Create a connection properties
var kg_props =
new KnowledgeGraphConnectionProperties(new Uri(url));
try
{
//Open a connection
using (var kg = new KnowledgeGraph(kg_props))
{
//Create the new link chart and show it
var newLinkChart = MapFactory.Instance.CreateLinkChart(
"KG from Template", kg, idSet, linkChartMap.URI);
FrameworkApplication.Panes.CreateMapPaneAsync(newLinkChart);
}
}
catch (Exception ex)
{
System.Diagnostics.Debug.WriteLine(ex.ToString());
}
});Note: Names of named object types within an id set entry are case-sensitive. Passing an id set containing invalid named object type names to MapFactory.Instance.CreateLinkChart will throw a KnowledgeGraphLayerException with an "Invalid named types" message. The KnowledgeGraphLayerException will contain a list of any invalid named object type names.
Many of the API functions that are available on the Map and MapView objects (for example zoom, pan, accessing layers etc.) apply to link charts same as for 2D and 3D maps. There are however a couple of key differences; notably when it comes to layer manipulation (some of these were previously mentioned in the KnowledgeGraph Link Charts overview):
- A link chart map can contain only 1 KG layer whereas a map may contain any number of KG layers (from the same or different knowledge graph datastores).
- You cannot remove the KG layer from the link chart map.
Map.RemoveLayerwill throw an exception andMap.CanRemoveLayerwill return false. - You cannot add any other layers to a link chart map unless the map view it is being displayed on has its layout algorithm set to
KnowledgeLinkChartLayoutAlgorithm.Geographic_Organic_Standard- otherwiseLayerFactory.Instance.CreateLayerwill throw an exception when the input container is a map of map type LinkChart. - You cannot alter the TOC order of any layers in a link chart map unless the map view it is being displayed on has its layout algorithm set to
KnowledgeLinkChartLayoutAlgorithm.Geographic_Organic_Standard.
QueuedTask.Run(()=> {
//check the layout algorithm for a particular link chart...
var linkChartMap = ... ;
var mapPanes = FrameworkApplication.Panes.OfType<IMapPane>().ToList();
var mapPane = mapPanes.FirstOrDefault(
mp => mp.MapView.IsLinkChartView &&
mp.MapView.Map.URI == linkChartMap.URI);
if (mapPane == null)
//no pane is currently open for the link chart
return;
var layout_algorithm = mapPane.MapView.GetLinkChartLayout();
if (layout_algorithm ==
KnowledgeLinkChartLayoutAlgorithm.Geographic_Organic_Standard) {
//TODO - manipulate the link chart
}
...
}The sublayers of a KG layer are defined differently between a map and link chart. This is discussed below.
Within a link chart, the KG layer is comprised of one sub-layer/aggregation layer for each entity type and relationship type in the KG regardless of whether the sub layer contains content or not. Entities are represented as point features and the relationships as lines. KG layers and child sub-layer or aggregation layers are detailed in the ArcGIS Pro help documentation: Knowledge graph layer and link chart layer.
On the link chart, each sub-layer is a composite layer of type LinkChartFeatureLayer. LinkChartFeatureLayers are used to control the grouping and symbolizing of the entities and relates between them. This is different from a KG layer that is in a map, where each of the sublayers is a FeatureLayer (or StandaloneTable). LinkChartFeatureLayers are not specified as part of the KG layer's id set. A LinkChartFeatureLayer/aggregation layer is always added to the KG layer in the link chart for each entity and relate type. Sub-layers for entity and relate types not specified in the id set will be empty.
Here is a snippet showing how to access the sub-layers of a KG layer:
var map = MapView.Active.Map;
var kgLayer = map.GetLayersAsFlattenedList().OfType<KnowledgeGraphLayer>().FirstOrDefault();
if (kgLayer == null)
return;
if (map.MapType == MapType.LinkChart)
{
// if map is of MapType.LinkChart then the first level children of the kgLayer are of
//type LinkChartFeatureLayer
var childLayers = kgLayer.Layers;
foreach (var childLayer in childLayers)
{
if (childLayer is LinkChartFeatureLayer lcFeatureLayer)
{
var isEntity = lcFeatureLayer.IsEntity;
var isRel = lcFeatureLayer.IsRelationship;
// TODO - continue processing
}
}
}
else if (map.MapType == MapType.Map)
{
// if map is of MapType.Map then the children of the kgLayer are the standard Featurelayer
//and StandAloneTable
var chidlren = kgLayer.GetMapMembersAsFlattenedList();
foreach (var child in chidlren)
{
if (child is FeatureLayer fl)
{
// TODO - process the feature layer
}
else if (child is StandaloneTable st)
{
// TODO - process the standalone table
}
}
}Once a link chart has been created and populated, to visualize and investigate different information you must either create a new link chart or append data to the existing KG layer. You cannot create another KG layer in the same link chart (recall: a link chart can only contain one KG layer). Calling LayerFactory.Instance.CreateLayer in an attempt to create a KG layer on a map of map type "link chart" will throw an exception.
Append data to a link chart using an id set, same as with create. To append data, create an id set (KnowledgeGraphLayerIDSet) that contains the additional information to be appended. Add an entry to the id set for each Named Object type to be appended along with a list of its/their relevant records. Specify a null or empty list for a given entry to include all the records for that particular type same as with create. With the id set populated, call CanAppendToLinkChart followed by the AppendToLinkChart methods on the link chart map.
CanAppendToLinkChart will check that the map is of type MapType.LinkChart; that the id set is neither null or empty; and that the id set does not contain any invalid named object type names that. Named object type names in an id set entry are case-sensitive. AppendToLinkChart method will throw a KnowledgeGraphLayerException if there are named object type names specified in the id set that do not match the named object type names in the knowledge graph. A KnowledgeGraphLayerException exception, if thrown, will contain a list of any named object type names that were invalid.
// We create an id set to contain the records to be appended
var dict = new Dictionary<string, List<long>>();
dict["Suspects"] = new List<long>();
// In this case, via results from a query...
var qry2 = "MATCH (s:Suspects) RETURN s";
QueuedTask.Run(async () => {
using (var kg = kg_layer.GetDatastore())
{
var graphQuery = new KnowledgeGraphQueryFilter()
{
QueryText = qry2
};
using (var kgRowCursor = kg.SubmitQuery(graphQuery))
{
while (await kgRowCursor.WaitForRowsAsync())
{
while (kgRowCursor.MoveNext())
{
using (var graphRow = kgRowCursor.Current)
{
var obj_val = graphRow[0] as KnowledgeGraphNamedObjectValue;
var oid = (long)obj_val.GetObjectID();
dict["Suspects"].Add(oid);
}
}
}
}
// make an id Set to append to the LinkChart
var idSet = KnowledgeGraphLayerIDSet.FromDictionary(kg, dict);
// Get the relevant link chart to which records will be
// appended....
var mapPanes = FrameworkApplication.Panes.OfType<IMapPane>().ToList();
var mapPane = mapPanes.FirstOrDefault(
mp => mp.MapView.IsLinkChartView && mp.MapView.Map.Name == "Acme Link Chart");
var linkChartMap = mapPane.MapView.Map;
// Call AppendToLinkChart with the id set
if (linkChartMap.CanAppendToLinkChart(idSet))
linkChartMap.AppendToLinkChart(idSet);
}
});When a new link chart is created the content is arranged using the layout algorithm KnowledgeLinkChartLayoutAlgorithm.Organic_Standard by default: KnowledgeLinkChartLayoutAlgorithm.Organic_Standard. You can obtain the link chart's current layout algorithm from the MapView displaying it via the map view GetLinkChartLayout extension method. Alter the layout pattern on the MapView via SetLinkChartLayoutAsync along with the desired layout algorithm enum as the input param. Use the SetLinkChartLayoutAsync overload with a value of forceLayoutUpdate=true to force an update.
var mv = MapView.Active;
var map = mv.Map;
var isLinkChart = map.MapType == MapType.LinkChart;
if (!isLinkChart)
return;
QueuedTask.Run(() => {
// get the layout algorithm
var layoutAlgorithm = mv.GetLinkChartLayout();
// toggle the value
if (layoutAlgorithm == KnowledgeLinkChartLayoutAlgorithm.Geographic_Organic_Standard)
layoutAlgorithm = KnowledgeLinkChartLayoutAlgorithm.Organic_Standard;
else
layoutAlgorithm = KnowledgeLinkChartLayoutAlgorithm.Geographic_Organic_Standard;
// set it
mv.SetLinkChartLayoutAsync(layoutAlgorithm);
// OR set it and force a redraw / update
// await mv.SetLinkChartLayoutAsync(layoutAlgorithm, true);
});A KnowledgeGraph datastore also provides access to the components of the graph data model (in addition to the afore mentioned gdb relational model components). Graph data model components are retrieved via openCypher graph query and (text) search results. The KnowledgeGraph datastore provides a KnowledgeGraphDataModel that can be traversed for the metadata that describes the individual knowledge graph "graph" entity and relationship types within the graph data model as well as their provenance (if any). The graph data model is detailed in the following section.
The knowledge graph data model defines the types of entities and relationships that exist in the knowledge graph and their properties. Entities typically represent people, places, and things, and relationships define how the entities are associated or "connected". Entities and relationships can be spatial or non-spatial. Their properties define/describe their characteristics. A special type of entity, called a Document can be added to any ArcGIS knowledge graph to provide additional context for an entity or a relationship (in which it participates). Documents can be pictures, presentations, text or Adobe Acrobat PDF files, website links, and so on. Knowledge graph can also contain provenance. Provenance can be added to a knowledge graph to describe entity and relationship "origins" or "lineage". Provenance can describe which organizations, people, sensors, etc. may have been involved with an associated entity or relationship between entities (i.e. provenance is a "chain of custody" of sorts). Provenance is only ever stored as entities "itself".
The different types of entities present within a knowledge graph are represented by entity types. An entity type defines a homogeneous collection of entities with a common set of properties and a spatial feature type (if the given entity type is spatially enabled). A relationship type performs a similar role for relationships. Each relationship type defines a homogenous collection of relationships that can exist between two entity types, with a common set of properties and a spatial feature type (if the given relationship type is spatially enabled). A property, similar to a GDB field, has a name and value type. Properties each have a role, identified by the public KnowledgeGraphPropertyRole GetRole() method. KnowledgeGraphPropertyRole can be: a "regular" role meaning the property is an attribute or "characteristic" of an entity or relationship; the role can be a document property role (self-explanatory); or a role indicating the property is associated with provenance.
The KnowledgeGraphDataModel and primary associated classes (and enums) are diagrammed and described below:
-
KnowledgeGraphDataModel represents the data model for the knowledge graph, including but not limited to entity and relationship types, spatial reference, and information about generation of unique identifiers. Access via a
knowledgeGraph.GetDataModel()method call. - KnowledgeGraphNamedObjectType is the abstract base class for all entity and relationship types. Named object types contain metadata about the entity and relationship types in the knowledge graph including their role and properties.
-
KnowledgeGraphEntityType derives from
KnowledgeGraphNamedObjectType. There is one KnowledgeGraphEntityType per individual entity type (or "category") in the graph. All entities stored in the knowledge graph must belong to aKnowledgeGraphEntityType. Entity instances will each have a type name value that matches the entity type name of which they are a part* -
KnowledgeGraphRelationshipType derives from
KnowledgeGraphNamedObjectType. There is one KnowledgeGraphRelationshipType per individual relationship "type" in the graph. All relationships stored in the knowledge graph must belong to aKnowledgeGraphRelationshipType. Relationship instances will each have a type name value that matches the relate type name of which they are a part. Relationship types also store a collection of end points represented as aKnowledgeGraphEndPoint. An end point stores the type names of the origin and destination entity types that participate in the relationship. -
KnowledgeGraphProperty describes a property or "attribute" of an entity or a relationship. Properties are stored in the knowledge graph as key/value pairs (similar to a .NET dictionary) where the key is a unique (string) name within the set of properties for the given type. Property descriptions are retrieved off either an entity or relationship type (via the abstract base class
KnowledgeGraphNamedObjectType"GetProperties()" method). The key of a given property value within an entity or relationship will match the name of its correspondingKnowledgeGraphProperty"type" (retrieved from the relevant KnowledgeGraphEntityType or KnowledgeGraphRelationshipType). Properties also have a role, identified by theKnowledgeGraphPropertyRoleenum accessed viakg_prop.GetRole(). The role can be Regular, Document, or Provenance. -
KnowledgeGraphEndPoint is associated with
KnowledgeGraphRelationshipType. KnowledgeGraphRelationshipTypes have a collection of end points from which the entity type names for the origin and destination entity types participating in a given relationship can be retrieved.
*Knowledge graphs can contain two special "entity" types:
- An entity type with the role
KnowledgeGraphNamedObjectTypeRole.Document. There can only be one document entity type per knowledge graph - meaning all documents stored in a knowledge graph will be entities of that same entity type. The name of the document entity type is the name of "the"KnowledgeGraphEntityTypewith the document role. There is always a Document entity type defined in a knowledge graph unless it uses a NoSQL data store with user-managed data in Neo4j. - An entity type with the role
KnowledgeGraphNamedObjectTypeRole.Provenance. The provenance type is stored in the knowledge graph meta entity type collection which is accessed viakg_dm.GetMetaEntityTypes()(note the "meta" in the method name) and not viakg_dm.GetEntityTypes(). Provenance is the only entity type that is currently stored in the knowledge graph meta entity type collection. Provenance represents origin and/or chain of custody information related to graph entities and relationships. Unlike Documents, the ability to capture provenance within a knowledge graph is not enabled by default.
Additional knowledge graph data model descriptions can be found in the Essential ArcGIS Knowledge vocabulary. The following examples show how to get the names of the Document and Provenance entity types, if present:
protected string GetDocumentEntityTypeName(KnowledgeGraphDataModel kg_dm) {
var entity_types = kg_dm.GetEntityTypes();
foreach (var entity_type in entity_types){
if entity_type.Value.GetRole() == KnowledgeGraphNamedObjectTypeRole.Document)
return entity_type.Value.GetName();
}
return "";//prob a Neo4j user managed KG
}
protected string GetProvenanceEntityTypeName(KnowledgeGraphDataModel kg_dm) {
var entity_types = kg_dm.GetMetaEntityTypes();
foreach (var entity_type in entity_types) {
if (entity_type.Value.GetRole() == KnowledgeGraphNamedObjectTypeRole.Provenance)
return entity_type.Value.GetName();
}
return "";//Not all knowledge graphs have Provenance
}The KnowledgeGraphDataModel also contains meta data describing how unique identifiers in the knowledge graph are generated (not shown in the above class diagram). Identifier metadata can be accessed off the KnowledgeGraphDataModel via the public KnowledgeGraphIdentifierInfo GetIdentifierInfo() method. KnowledgeGraphIdentifierInfo is the abstract base class for knowledge graph identifier information. There are two concrete classes that derive from KnowledgeGraphIdentifierInfo: KnowledgeGraphNativeIdentifier meaning that the knowledge graph is using a database native identifier as the unique identifier for entities and relationships, or a KnowledgeGraphUniformIdentifier meaning that the knowledge graph is using a specific property as the unique identifier for entities and relationships.
The following example illustrates how to retrieve data model information from the knowledge graph:
//Utility method showing how to access named object type metadata
private void ProcessKGNamedObjectType(
int level, KnowledgeGraphNamedObjectType kg_no_type,
string prefix) {
var spaces = new string(' ', level);
var indent = $"{spaces}{prefix}";
var end_points = new List<KnowledgeGraphEndPoint>()
as IReadOnlyList<KnowledgeGraphEndPoint>;
switch (kg_no_type)
{
case KnowledgeGraphEntityType kg_e_type:
break;
case KnowledgeGraphRelationshipType kg_r_type:
end_points = kg_r_type.GetEndPoints();
break;
}
System.Diagnostics.Debug.WriteLine($"{indent} Name: '{kg_no_type.GetName()}'");
System.Diagnostics.Debug.WriteLine($"{indent} Role: {kg_no_type.GetRole()}");
System.Diagnostics.Debug.WriteLine($"{indent} AliasName: '{kg_no_type.GetAliasName()}'");
System.Diagnostics.Debug.WriteLine($"{indent} HasObjectID: {kg_no_type.GetHasObjectID()}");
System.Diagnostics.Debug.WriteLine(
$"{indent} ObjectIDPropertyName: {kg_no_type.GetObjectIDPropertyName()}");
System.Diagnostics.Debug.WriteLine($"{indent} IsStrict: {kg_no_type.GetIsStrict()}");
System.Diagnostics.Debug.WriteLine($"{indent} Properties:\r\n{indent} ---------------");
var kg_props = kg_no_type.GetProperties();
var prop = 0;
foreach (var kg_prop in kg_props) {
System.Diagnostics.Debug.WriteLine($"{indent} Property[{prop}]:");
System.Diagnostics.Debug.WriteLine(
$"{indent} DefaultVisibility: {kg_prop.GetHasDefaultVisibility()}");
System.Diagnostics.Debug.WriteLine(
$"{indent} IsSystemMaintained: {kg_prop.GetIsSystemMaintained()}");
System.Diagnostics.Debug.WriteLine(
$"{indent} Role: {kg_prop.GetRole()}");
prop++;
}
if (end_points.Count == 0)
return;
System.Diagnostics.Debug.WriteLine($"{indent} EndPoints:");
foreach (var end_point in end_points) {
System.Diagnostics.Debug.WriteLine(
$"{indent} " +
$"OriginEntityTypeName: '{end_point.GetOriginEntityTypeName()}', " +
$"DestinationEntityTypeName: '{end_point.GetDestinationEntityTypeName()}'");
}
}
//Utility method showing how to access knowledge graph identifier info
private void ProcessKGIdentifierInfo(KnowledgeGraphIdentifierInfo kg_id_info) {
var kg_id_gen = kg_id_info.GetIdentifierGeneration();
if (kg_id_info is KnowledgeGraphNativeIdentifier kg_ni) {
System.Diagnostics.Debug.WriteLine($"IdentifierInfo is KnowledgeGraphNativeIdentifier");
}
else if (kg_id_info is KnowledgeGraphUniformIdentifier kg_ui) {
System.Diagnostics.Debug.WriteLine($"IdentifierInfo is KnowledgeGraphUniformIdentifier");
System.Diagnostics.Debug.WriteLine($"IdentifierName {kg_ui.GetIdentifierName()}");
}
System.Diagnostics.Debug.WriteLine($"Identifier MethodHint {kg_id_gen.GetMethodHint()}");
}
...
//elsewhere - retrieve the KG graph data model from the KnowledgeGraph via "GetDataModel()"
using(var kg = new KnowledgeGraph(
new KnowledgeGraphConnectionProperties(new Uri(kg_service_uri)))) {
var kg_name = System.IO.Path.GetFileName(
System.IO.Path.GetDirectoryName(kg_service_uri));
//Get the graph data model
var kg_dm = kg.GetDataModel();
System.Diagnostics.Debug.WriteLine($"\r\n'{kg_name}' Datamodel:\r\n-----------------");
var time_stamp = kg_dm.GetTimestamp();
var sr = kg_dm.GetSpatialReference();
System.Diagnostics.Debug.WriteLine($"Timestamp: {time_stamp}");
System.Diagnostics.Debug.WriteLine($"Sref {sr.Wkid}");
System.Diagnostics.Debug.WriteLine($"IsStrict: {kg_dm.GetIsStrict()}");
System.Diagnostics.Debug.WriteLine($"OIDPropertyName: {kg_dm.GetOIDPropertyName()}");
System.Diagnostics.Debug.WriteLine($"IsArcGISManaged: {kg_dm.GetIsArcGISManaged()}");
//Write out KG identifier info
var kg_id_info = kg_dm.GetIdentifierInfo();
System.Diagnostics.Debug.WriteLine("");
ProcessKGIdentifierInfo(kg_id_info);
//Write out KG Meta Entity Type info - i.e. Provenance, if there is any
System.Diagnostics.Debug.WriteLine("\r\n MetaEntityTypes:\r\n ------------");
var dict_types = kg_dm.GetMetaEntityTypes();
var key_count = 0;
foreach(var kvp in dict_types)
{
System.Diagnostics.Debug.WriteLine($"\r\n MetaEntity ({key_count++}): '{kvp.Key}'");
ProcessKGNamedObjectType(1, kvp.Value, " ");
}
//Write out KG Entity Type info (includes Document entity type)
System.Diagnostics.Debug.WriteLine("\r\n EntityTypes:\r\n ------------");
dict_types = kg_dm.GetEntityTypes();
key_count = 0;
foreach (var kvp in dict_types)
{
System.Diagnostics.Debug.WriteLine($"\r\n Entity ({key_count++}): '{kvp.Key}'");
ProcessKGNamedObjectType(1, kvp.Value, " ");
}
//Write out KG Relationship Type info
System.Diagnostics.Debug.WriteLine("\r\n RelationshipTypes:\r\n ------------");
var dict_rel_types = kg_dm.GetRelationshipTypes();
key_count = 0;
foreach (var kvp in dict_rel_types)
{
System.Diagnostics.Debug.WriteLine($"\r\n Relationship ({key_count++}): '{kvp.Key}'");
ProcessKGNamedObjectType(1, kvp.Value, " ");
}
}The KnowledgeGraph api supports both graph query and text searches. Graph queries via the api are based on the openCypher declarative query language. Text searches via the api use the Apache Lucene - Query Parser syntax. Graph queries and text searches are asynchronous and both use a similar pattern derived from the RealtimeCursor, via KnowledgeGraphCursor to retrieve query or text search results. Similar to real-time/stream layer queries, when a graph query or text search has been specified, callers wait or "a-wait" for rows to be returned (asynchronously) from the server. Returned rows can contain either primitives (such as ints, strings, doubles, etc.) or knowledge graph values of type KnowledgeGraphValue.
The general pattern for processing queries and searches is as follows:
//submitting either of a query or search returns a KG row cursor...
using(var kgRowCursor = kg.SubmitQuery(qryFilter) | kg.SubmitSearch(srchFilter) {
//Wait for the server to process rows and call back on WaitForRowsAsync() with _true_
//note also, WaitForRowsAsync is cancellable
while(await kgRowCursor.WaitForRowsAsync()) { //non-blocking await
//Rows are available - process the returned rows
while (kgRowCursor.MoveNext()) {
//process each row
using (var graph_row = kgRowCursor.Current) {
//get the row values from the KG row array
var val1 = graph_row[0];
var val2 = graph_row[1]; //etc - cast as necessary
...
}
}
}//call WaitForRowsAsync
//If we are here there are no (more) rows...
...Generally speaking, the pattern is very similar to processing results from a traditional GDB - namely, submit the query and iterate through the returned rows until MoveNext returns false. The current row will be available in the KG row ".Current" property (also similar to a GDB row cursor). The two primary differences are the use of "WaitForRowsAsync" and the array index notation used to extract values from the KG row.
As queries are processed asynchronously on the KG service server, clients need to wait for rows to be returned. This is accomplished with "WaitForRowsAsync". Note also the use of "await" allowing clients to "a-wait" returned rows, or, perform a non-blocking wait meaning the Pro UI remains responsive while the addin is waiting for rows. When a batch of rows is ready, WaitForRowsAsync completes and returns true. Addins can then process the returned rows in much the same way as with a GDB. When the batch of rows has been processed (and MoveNext returns false), the addin should call "WaitForRowsAsync" again - a-waiting the next batch of rows. When the next batch is returned, they are processed same as before. The processing of rows and repetitive calls to "WaitForRowsAsync" continues until "WaitForRowsAsync" returns false, in which case, the addin exits the while loop. "WaitForRowsAsync" can also return false up-front (without ever returning true) if a given query or search results in no rows. WaitForRowsAsync is described in more detail within the KnowledgeGraphCursor WaitForRowsAsync section.
When a row is ready, MoveNext on the KG row cursor provisions the KG row cursor ".Current" property with that row (same as with a GDB row cursor and its .Current property). The biggest difference with the KG row cursor (as compared to the GDB row cursor) is that its .Current property is actually a value array. OpenCypher queries can contain multiple return values - depending on the nature of the query "RETURN" statement. For example, this query "MATCH .... WHERE .... RETURN e1" has one return value whereas this query "MATCH .... WHERE .... RETURN e1, e2, e3" has three ("e1", "e2", and "e3" whatever the given aliases represent). The number of values returned in each "row" of the row array will always match the number of return values specified in the query. Text searches will only return single values per row in the row array even though each individual value, per row, can be either an entity or a relate. For a query, the individual return values can be almost anything. Here are a few examples:
//assume e1 is an alias for an entity type
var qry = @"MATCH ... RETURN e1";
...
var graph_row = kgRowCursor.Current//Current row
//Only one value in the array - a because the RETURN only specified a single
//return value "e1"
var entity_val = graph_row[0] as KnowledgeGraphEntityValue;
//Multiple return values...
//assume 'e's for entity types and 'e's for relate types
var qry = @"MATCH ... RETURN e1, r1, e2, r2";
...
var graph_row = kgRowCursor.Current//Current row
var entity_val = graph_row[0] as KnowledgeGraphEntityValue; //e1
var relate_val = graph_row[1] as KnowledgeGraphRelateValue; //r1
var entity_val2 = graph_row[2] as KnowledgeGraphEntityValue; //e2
var relate_val2 = graph_row[3] as KnowledgeGraphRelateValue; //r2
//Multiple values consisting of primitives _and_ "types"
var qry = @"MATCH ... RETURN e1.FULL_NAME, e1.START_DATE, e1.DURATION, e2, r1";
...
var graph_row = kgRowCursor.Current//Current row
var full_name = (string)graphRow[0]; //e1.FULL_NAME
var call_date = (DateTimeOffset)graphRow[1]; //e1.START_DATE
var call_mins = (long)graphRow[2]; //e1.DURATION
var entity_val2 = graph_row[3] as KnowledgeGraphEntityValue; //e2
var relate_val1 = graph_row[4] as KnowledgeGraphRelateValue; //r1
//etc.Entity and relate types, as was briefly discussed in the KnowledgeGraph Graph Data Model Types section, store their values as properties - the corollary in the GDB being rows and features storing their values as fields. Properties have a name and a value, the value can be any of the supported esri value types (strings, date and time types, numeric values, shapes, identifiers, and blobs). Accessing the value for a given entity or relate uses the same indexer notation as with a GDB feature or row, namely: entity["PROPERTY_NAME_HERE"] followed by the appropriate cast of the value (from "object" which is the default return value type). Assuming an entity type has three string properties "NAME", "PHONE_NUMBER", and "CITY", retrieving the values would be:
var person = graphRow[0] as KnowledgeGraphEntityValue;
var person_name = (string)person_called["NAME"];
var cell_num = (string)person_called["PHONE_NUMBER"];
var city = (string)person_called["CITY"];which is identical (with the exception of the row array accessor) to the syntax that would be used to retrieve corresponding field values from a row or feature. More details on return values and how to process values returned in the KnowledgeGraphRow are provided in the KnowledgeGraph Graph Query and Text Search Results section.
To submit a graph query, addins instantiate a knowledge graph query filter of type KnowledgeGraphQueryFilter with a kg_query_filter.QueryText using an openCypher formatted query string. openCypher borrows quite a lot of syntax from SQL and, similar to SQL, is built using clauses. Clauses can use keywords like WHERE and ORDER BY, familiar to users of SQL, and a construct, not found in SQL, called MATCH. MATCH clauses specify which entities, relationships, and properties are to be searched for in the query - (similar to a SQL SELECT). The Neo4j primer on their Cypher query language is also an excellent resource for the syntax and usage of graph queries (note: ArcGIS Knowledge graph queries can only retrieve values. Graph query clauses that can update values are not supported.)
ArcGIS has extended the cypher language with its own spatial operators that can be added to graph query expressions. ArcGIS Knowledge supports the following custom spatial operators for use in graph queries:
- ST_Equals—Returns entities with equal geometries. The syntax is
esri.graph.ST_Equals(geometry1, geometry2). - ST_Intersects—Returns entities with intersecting geometries. The syntax is
esri.graph.ST_Intersects(geometry1, geometry2). - ST_Contains—Returns entities whose geometries are contained by the specified geometry. The syntax is
esri.graph.ST_Contains(geometry1, geometry2).
ArcGIS Knowledge also provides a datetime(DATE-TIME-VALUE-HERE) method that can be combined into a knowledge graph graph query to convert dates and times to coordinated universal time (UTC). Within an ArcGIS knowledge graph date-time values must always be expressed in coordinated universal time (UTC). As the knowledge graph does not convert dates and times within queries to UTC automatically, the datetime() utility should be used to do the conversion. More information on the datetime() utility and spatial operators can be found in ArcGIS Enterprise Query a knowledge graph. More general information on cypher support for date and time types can be found on the Cypher Property Graph Query Language github here
Addins can also specify a kg_query_filter.ProvenanceBehavior to determine whether or not provenance entities should be included in the query results if the knowledge graph contains provenance information. To include provenance in the results, specify kg_query_filter.ProvenanceBehavior = KnowledgeGraphProvenanceBehavior.Include. The default is "Exclude". Specifying KnowledgeGraphProvenanceBehavior.Include for a query against a knowledge graph that has no provenance will result in an empty result set (i.e. graph_row_cursor.MoveNext() will return false). To check for provenance, use the following routine:
protected string GetProvenanceEntityTypeName(KnowledgeGraphDataModel kg_dm) {
//same example as provided above in the data model section...
var entity_types = kg_dm.GetMetaEntityTypes();
foreach (var entity_type in entity_types) {
if (entity_type.Value.GetRole() == KnowledgeGraphNamedObjectTypeRole.Provenance)
return entity_type.Value.GetName();
}
return "";
}
protected bool SupportsProvenance(KnowledgeGraph kg) {
//if there is a provenance entity type then the KnowledgeGraph
//supports provenance
return !string.IsNullOrEmpty(GetProvenanceEntityTypeName(kg.GetDataModel()));
}
//elsewhere...usage...
private bool _includeProvenance = ...;
var kg_qf = new KnowledgeGraphQueryFilter() {
QueryText = query,
};
//do we have provenance that can be included?
if (_includeProvenance && SupportsProvenance(kg)) {
//Only use "Include" if the Knowledge graph _has_ provenance
kg_qf.ProvenanceBehavior = KnowledgeGraphProvenanceBehavior.Include;
}An output spatial reference for all returned geometries can be specified via KnowledgeGraphQueryFilter.OutputSpatialReference. However, this parameter is currently ignored. It is added for use in a future release. Currently, geometry values are returned in the spatial reference of the underlying knowledge graph and must be projected to a different spatial reference by the addin code (regardless of the OutputSpatialReference value).
In the following example provenance is specified as being included in the returned values if the knowledge graph (being queried) has provenance. Processing the query follows the same basic pattern outlined in the KnowledgeGraph Graph Queries and Text Search overview:
//Define a query - select the first 10 entities
var kg_qf = new KnowledgeGraphQueryFilter() {
QueryText = "MATCH (n) RETURN n LIMIT 10",
ProvenanceBehavior = SupportsProvenance(kg) ?
KnowledgeGraphProvenanceBehavior.Include :
KnowledgeGraphProvenanceBehavior.Exclude
};
//Submit the graph query
using (var kg_rc = kg.SubmitQuery(kg_qf)) {
//Do a non-blocking await waiting for rows to be retrieved
while (await kg_rc.WaitForRowsAsync()) {
//Rows are available
while (kg_rc.MoveNext()) {
//process each row
using (var graph_row = kg_rc.Current) {
int val_count = (int)graph_row.GetCount();
for (int i = 0; i < val_count; i++) {
var row_val = graph_row[i];
//TODO - process value
//...
}
}
}
}//keep looping until there are no more rows
}Note again a couple of key differences in the logic for processing Knowledge Graph query and search results as compared to relational GDB queries:
- The code is loopoing on successive calls to kg_rc.WaitForRowsAsync() (with "await")
- The returned row is an array - note
int val_count = (int)graph_row.GetCount()andvar row_val = graph_row[i].
Bind parameters allow the use of substitution variables within the open cypher query text. Most commonly, a bind parameter is used when the value to be used in the query will be determined "dynamically" when the application is already running, for example, a list of ids or a geometry. Bind parameters are referenced in the query using an arbitrary variable name identified by a "leading" dollar-sign "$", such as $ids, $extent, $list_of_movie_titles, $city_name, and so on. The value, or values, must be assigned to variable before the query is executed and are assigned via the KnowledgeGraphQueryFilter.BindParameters dictionary property. Bind parameters are assigned as key/value pairs. The key must match the name of the relevant variable, without the "$", and the value will be the value to be assigned to the bind parameter when the query is executed. If a value represents a collection (eg a list or array of ids, names, dates, etc), then the collection must be converted to a KnowledgeGraphArrayValue first or the query will fail. Bind parameters can not be used to substitute any values that will be included in the query plan (eg table or property names). Some examples:
//a query is specified with a bind parameter for a "TO BE" list of ids...
var qry = @"MATCH (p:PhoneNumber) " +
@" WHERE p.objectid IN $object_ids " +
@"RETURN p";
//Create a KG query filter
var kg_qry_filter = new KnowledgeGraphQueryFilter() {
QueryText = qry //includes the bind param "$object_ids"
};
//The list of ids must be provisioned before the query is executed...
//perhaps from a selection...
var sel_set = MapView.Active.SelectFeatures(...);
var dict = ss.ToDictionary();
var ids = dict.First(mm => mm.Key.Name == "PhoneNumber").Value;
//Because the ids use a collection, it must be converted to a KnowledgeGraphArrayValue first
var kg_oid_array = new KnowledgeGraphArrayValue();
kg_oid_array.AddRange(oids);
//Assign the ids to the bind parameter of the KG query filter
//use the name of the variable for the assignment without the "$"
kg_qry_filter.BindParameters["object_ids"] = kg_oid_array;
//submit the query in the usual way
using (var kgRowCursor = kg.SubmitQuery(kg_qry_filter)) {
...Geometry bind parameters can also be used with the Esri custom spatial operators. For example:
//a query is specified with a bind parameter for a "TO BE" list of ids
//and a "TO BE" geometry to be used w/ the intersects operator
var qry = @"MATCH (p:PhoneNumber) " +
@"WHERE p.objectid IN $object_ids AND " +
@"esri.graph.ST_Intersects($sel_geom, p.shape) " +
@"RETURN p";
...
//The ids are provisioned same as before..
kg_qry_filter.BindParameters["object_ids"] = kg_oid_array;
//The geometry is provisioned and must be in the same projection as the kg
var poly = ... ;
var sr = kg.GetSpatialReference();
var proj_poly = GeometryEngine.Instance.Project(poly, sr);
//Create a bind param for the geometry
kg_qry_filter.BindParameters["sel_geom"] = proj_poly;
//submit the query in the usual way
using (var kgRowCursor = kg.SubmitQuery(kg_qry_filter)) {
...
Text searches can be submitted against a knowledge graph using a KnowledgeGraphSearchFilter. Addins specify the kg_search_filter.SearchText to be used for the search. All text property values of all entities and/or relates in the graph will be searched. The simplest type of query can consist of a single term or phrase like "book", "Car", or "cats and dogs". More complex search strings can be constructed using boolean operators, fields, wildcards, and so forth incorporating the Apache Lucene - Query Parser syntax. Consult the Apache Lucene - Query Parser syntax reference for more information. A kg_search_filter.SearchTarget of type KnowledgeGraphNamedTypeCategory should be specified to control what will be the target of the search. Entities, relationships, entities and relationships, and provenance can all be specified. The default search target will be KnowledgeGraphNamedTypeCategory.Entity if a SearchTarget is not specified. A limit on the number of returned rows can be specified via kg_search_filter.MaxRowCount. The default is 100. Use kg_search_filter.Offset to skip rows (in the results). The Offset sets the index of the first result to be returned. The Offset can be used in conjunction with the MaxRowCount to return results in "batches".
In this example, a text search is implemented that looks for all entities with a name or property value of "Redlands". Notice that, with the exception of calling "SubmitSearch" (rather than "SubmitQuery"), the workflow for processing a text search is the same as the workflow for processing a graph query:
var kg_sf = new KnowledgeGraphSearchFilter() {
SearchTarget = KnowledgeGraphNamedTypeCategory.Entity,
SearchText = "Redlands",
ReturnSearchContext = true,
MaxRowCount = 10
};
//Submit the text search
using (var kg_rc = kg.SubmitSearch(kg_sf)) {
//Same workflow for a search as w/ processing a query...
while (await kg_rc.WaitForRowsAsync()) {
//Rows are available
while (kg_rc.MoveNext()) {
//process each row
using (var graph_row = kg_rc.Current) {
int val_count = (int)graph_row.GetCount();
for (int i = 0; i < val_count; i++) {
var row_val = graph_row[i];
//TODO - process value
//...
}
}
}
}//keep looping until there are no more rows
}Once a query or text search has been submitted, callers perform a non-blocking wait, or "a-wait" via the returned KG row cursor and a kg_row_cursor.WaitForRowsAsync() method call. WaitForRowsAsync has a built-in timeout of (approximately) 30 seconds - this is the maximum amount of time it will wait on an open connection to receive a batch of rows from the server and is not configurable. Once a batch of rows is retrieved (within the built-in timeout duration), the WaitForRowsAsync Task completes and returns true. Clients can then call kg_row_cursor.MoveNext() to retrieve and process the returned rows (same as with a relational gdb query). Once the rows have been processed, the caller makes another call to WaitForRowsAsync to retrieve the next batch, and so-on until WaitForRowsAsync returns false meaning there are no more rows. WaitForRowsAsync will also return false when a search or query has no results.
Each call to WaitForRowsAsync resets the 30 second timeout and callers "a-wait" (i.e. do a non-blocking wait for) the next batch of rows. If the full 30 second elapses before any rows are retrieved on the open connection, the connection is closed and no further rows can be retrieved for the given search or query.
Looking back at the general pattern for processing row results for both searches and queries given in preceding examples, we see the outer loop on WaitForRowsAsync until it returns false:
//note the "await"
while (await kg_rc.WaitForRowsAsync()) {//true means we have rows
//Rows are available
while (kg_rc.MoveNext()) {
//process each row
using (var graph_row = kg_rc.Current) {
...
}
}
}//loop again
//if we are here, WaitForRowsAsync has returned false meaning we are doneEven though the default timeout duration is not configurable, clients can build-in their own timeout using the overload of WaitForRowsAsync that takes a CancellationToken constructed with a user defined timeout. When the timeout specified for the CancellationToken has been reached, the current WaitForRowsAsync Task is cancelled and completes immediately and a TaskCanceledException is thrown from WaitForRowsAsync to indicate that the awaited WaitForRowsAsync Task was cancelled (due to the timeout). Note: The cancelled Task will complete regardless of whether the underlying streaming connection is still retrieving rows. A user defined timeout defined within a CancellationToken can not be reset. The only way to reset the timeout is to construct a new CancellationToken.
In the following example, an addin is specifying a user defined timeout of 20 seconds. If the total row retrieval and processing time exceeds 20 seconds, the CancellationToken will timeout and throw a TaskCanceledException halting the row retrieval regardless of whether the client is still processing rows or not.
//assume a knowledge graph row cursor has been returned from a SubmitQuery
//or SubmitSearch...
//auto-cancel after 20 seconds
var cancel = new CancellationTokenSource(new TimeSpan(0, 0, 20));
//wrap the WaitForRowsAsync call with a try/catch for a
//TaskCanceledException
try {
//wait for rows. Monitor for TaskCanceledException
//successive calls to WaitForRowsAsync will _not_ reset a cancellation
//token timeout if one was specified...
while (await kg_rc.WaitForRowsAsync(cancel.Token))
{
//process retrieved rows...
while (kg_rc.MoveNext())
{
}
}
}
catch(TaskCanceledException tce)
{
//TODO - we were cancelled! A TaskCanceledException
//will be thrown if row retrieval and processing exceeds
//the specified user-defined timeout (if there was one)
}
finally
{
//Clean up, etc.
}Additional information on WaitForRowsAsync and cancellation can be found in the ProConcepts StreamLayers document.
Results returned from graph queries and text searches are in the form of "graph values" and/or primitives. The biggest difference when dealing with KG query and search results, as compared to the GDB, is that the returned KnowledgeGraphRow is an array value. As open cypher queries can contain multiple (arbitrary) return values per row, the returned row must be able to accommodate that. For queries, returned values (per row) can be a mix of primitives (ints, doubles, dates, ids, strings, geometries, etc.) as well as object values (entities, relationships, and paths). Rows returned from searches will only ever contain single values (either entities or relates depending on the specified search target). For example ( from the earlier KnowledgeGraph Graph Queries and Text Search overview section):
//Multiple values consisting of primitives _and_ "types"
var qry = @"MATCH ... RETURN e1.FULL_NAME, e1.START_DATE, e1.DURATION, e2, r1";
...
var graph_row = kgRowCursor.Current//Current row
var full_name = (string)graphRow[0]; //e1.FULL_NAME
var call_date = (DateTimeOffset)graphRow[1]; //e1.START_DATE
var call_mins = (long)graphRow[2]; //e1.DURATION
var entity_val2 = graph_row[3] as KnowledgeGraphEntityValue; //e2
var relate_val1 = graph_row[4] as KnowledgeGraphRelateValue; //r1Note how each value specified in the RETURN statement of the open cypher query above matches a corresponding "slot" in the returned row array. Open Cypher queries have a huge amount of flexibility that they can use to specify what values are to be returned. For example, the return statement ... RETURN [1,2,3,4,5] returns a list/array of literals and ... RETURN [p, p.name] returns an array of (presumably) and entity or relate aliased as "p" and "p.name", its name property. Note the use of "[]" -square brackets- in the RETURN statement in both cases indicating an array is to be returned. This leads to another aspect of return values - they can, themselves, also be arrays (same as the KG row). For example, given:
var graphQuery = new KnowledgeGraphQueryFilter() {
QueryText = @"MATCH (p:Person) RETURN [p, p.name, p.age]"
...The number of returned values in the row array will be 1 (not 3) - (int)graphRow.GetCount() equals "1". Even though the RETURN statement does indeed specify that three values are to be returned, it is using "[]" in the return statement to enclose the 3 values meaning that the 3 values will be, themselves, returned within an array (within the row array). The returned array, within the row array, would itself need to be iterated to process the contained values. Thus, returned KG row array values may need to be processed recursively depending on what they are. At the end of this main section, a complete example is given showing one way of recursively processing returned values - specifically the custom utility method called "ProcessKnowledgeGraphRowValue(...)" in the example.
In the below example, the returned array (within the row array) is being processed "directly" or in a loop (not recursively):
var graphQuery = new KnowledgeGraphQueryFilter() {
QueryText = @"MATCH (p:Person) RETURN [p, p.name, p.age]" //values returned within an array
...
//KnowledgeGraphRow contains a single value, an array...
var kg_array = graphRow[0] as KnowledgeGraphArrayValue;//retrieve the returned array
//Pull the values out of the returned array directly
var person = kg_array[(ulong)0] as KnowledgeGraphEntityValue;//p
var name = (string)kg_array[(ulong)1]; //p.name
var age = (int)kg_array[(ulong)2]; //p.age
//or use a loop...
var count = (int)kg_array.GetSize();
for (int i = 0; i < count; i++) {
var array_val = kg_array[(ulong)i];
...
}Note: The current row being processed is always available in the KnowledgeGraphCursor's current graph row kg_cursor.Current property.
Graph queries can also return "paths". Graph queries that take the general form of MATCH p=(e1)-[]->(e2) describe a relationship, or "path" "p", between two or more entities (in this case, a "directed" relationship or path between "e1" and "e2" - note the "->" pointing to e2 and the "p=" at the front of the MATCH statement) - the use of "p" as the path variable is completely arbitrary, "foo=" or "bar=" could equally have been used. The series of connected nodes and relationships that result from this type of query form the path. Paths are returned as a KnowledgeGraphPathValue. KnowledgeGraphPathValues can contain a collection of entities and relationships (depending on how the path was specified in the graph query). Both collections in the path value must be enumerated to evaluate all returned knowledge graph values. Typically, when executing queries via code, rather than interactively via the UI, it is more straightforward to specify the individual values to be returned in the RETURN statement as a comma-separated list (same as previous examples) rather than via a path variable (specified in the MATCH clause)
The complete KnowledgeGraphValue data model in the ArcGIS.Core.Data.Knowledge namespace is shown below:
-
KnowledgeGraphValue is the abstract base class for all derived knowledge graph values. Examine
public KnowledgeGraphValueType KnowledgeGraphValueTypeto determine the value type and/or check using a cast (to the derived type). -
KnowledgeGraphPrimitiveValue can wrap any (supported) primitive value - text, numeric, data/time, guid, geometry, blob. The primitive value itself can be retrieved via the
public object GetValue()method. Note: queries and searches return primitive values directly (as ints, longs, strings, doubles, etc.). They do not useKnowledgeGraphPrimitiveValue.KnowledgeGraphPrimitiveValueis for future use in the api. -
KnowledgeGraphPathValue represents a series of connected entities and relationships usually described by an openCypher query of the form
p=(e1)-[]->(e2), p=(e1)-[]->(e2)<-[]->(e3)and so on. Entities and relationships connected by a path can be retrieved via thepublic KnowledgeGraphEntityValue GetEntity(ulong index)andpublic KnowledgeGraphRelationshipValue GetRelationship(ulong index)methods respectively. -
KnowledgeGraphArrayValue can contain an array of 0 or more
KnowledgeGraphValuevalues and/or primitive values depending on the nature of the query. -
KnowledgeGraphObjectValue is the base class for any object value in the knowledge graph. An object value stores values as properties (not unlike the fields of a row or feature). Each property consists of a key|value pair. The value can be any
KnowledgeGraphValueor primitive. The list of property keys can be retrieved via thepublic IReadOnlyList<string> GetKeys()method. Use the keys with the object value indexerpublic object this[string key]to retrieve the corresponding property value. Queries typically return named objects like entities and relationships (derived fromKnowledgeGraphObjectValue), however,KnowledgeGraphObjectValueinstances themselves can be returned in queries when the query defines an anonymous type "on-the-fly". For example, the query stringMATCH (b:Beer) RETURN { Xbeer: { Xname: b.name, Xid: b.id } }defines an anonymous type "XBeer" (based on the entity "Beer") which would be returned as a KnowledgeGraphObjectValue (and not as an entity). -
KnowledgeGraphNamedObjectValue is the base class for named object values and derives from
KnowledgeGraphObjectValue. Named objects include entities and relationships. In addition to being able to store property values, named objects (can) also have an id (usually a guid) that uniquely identifies them, an object id, and a "type". The "type" describes the type or category for the particular named object value and corresponds to their associated underlying geodatabase table name and correspondingKnowledgeGraphNamedObjectTypecontained in theKnowledgeGraphDataModel(refer to the data model) section) -
KnowledgeGraphEntityValue derives from
KnowledgeGraphNamedObjectValue. All entities, therefore, have values stored as properties, a unique id, can have an object id, and have a (string) type. Entities also have an arbitrary string label retrieved viapublic string GetLabel(). Entities that have a geometry can be accessed as features from a feature class whose name is the same as the*KnowledgeGraphNamedObjectValuetype string. Entities that do not have a geometry can be accessed as rows from a table (whose name is the same as the*KnowledgeGraphNamedObjectValuetype string). -
KnowledgeGraphRelationshipValue derives from
KnowledgeGraphNamedObjectValue, same as entities, meaning that relationships in a graph model can also have properties, a unique id, an object id, and have a (string) type. Relationships too are accessible as features or rows from the knowledge graph datastore depending on whether they have a geometry or not. Relationships contain an origin and destination id identifying the origin and destination entities associated with the relationship. The ids can be retrieved via thepublic object GetOriginID()andpublic virtual object GetDestinationID()methods respectively.
An example of a general pattern for processing any supported value retrieved from the KnowledgeGraphRow array is shown below. Note the use of GetCount() and the KnowledgeGraphRow indexer to retrieve all values from the returned row array as well as the use of recursion to handle any "nested" values such as within KnowledgeGraphArrayValues and KnowledgeGraphPathValues (the custom "ProcessKnowledgeGraphRowValue" method is provided further below in this section):
KnowledgeGraph kg = .... ;
var kg_qf = new KnowledgeGraphQueryFilter() {
QueryText = "....",
ProvenanceBehavior = SupportsProvenance(kg) ?
KnowledgeGraphProvenanceBehavior.Include :
KnowledgeGraphProvenanceBehavior.Exclude
};
//Submit the graph query
using (var kg_rc = kg.SubmitQuery(kg_qf)) {//Same workflow for SubmitSearch
//Do a non-blocking await waiting for rows to be retrieved
while (await kg_rc.WaitForRowsAsync()) {
//Rows are available
while (kg_rc.MoveNext()) {
//process each row
using (var graph_row = kg_rc.Current) {
int val_count = (int)graph_row.GetCount();
for (int i = 0; i < val_count; i++) {
var row_val = graph_row[i];
//recursively process...
ProcessKnowledgeGraphRowValue(0, row_val);//can be a KG Value or primitive
}
}
}
}//loop for the next batch
}
...
...
#region KG Utilities
protected string GetDocumentEntityTypeName(KnowledgeGraphDataModel kg_dm) {
var entity_types = kg_dm.GetEntityTypes();
foreach (var entity_type in entity_types){
if entity_type.Value.GetRole() == KnowledgeGraphNamedObjectTypeRole.Document)
return entity_type.Value.GetName();
}
return "";//prob a Neo4j user managed KG
}
protected string GetProvenanceEntityTypeName(KnowledgeGraphDataModel kg_dm) {
var entity_types = kg_dm.GetMetaEntityTypes();
foreach (var entity_type in entity_types) {
if (entity_type.Value.GetRole() == KnowledgeGraphNamedObjectTypeRole.Provenance)
return entity_type.Value.GetName();
}
return "";//Not all knowledge graphs have Provenance
}
protected bool SupportsProvenance(KnowledgeGraph kg) {
//if there is a provenance entity type then the KnowledgeGraph
//supports provenance
return !string.IsNullOrEmpty(GetProvenanceEntityTypeName(kg.GetDataModel()));
}
protected bool GetEntityIsProvenance(KnowledgeGraphEntityValue entity,
string provenanceName = "") {
if (string.IsNullOrEmpty(provenanceName))
return false;
return entity.GetTypeName() == provenanceName;
}
#endregion KG Utilities
#region Read KG Values
//All entities and relationships, including Documents and Provenance
private void PrintGraphNamedObjectValue(int level,
KnowledgeGraphNamedObjectValue kg_named_obj_val) {
var spaces = new string(' ', level);
var indent = $"{spaces}";
if (kg_named_obj_val is KnowledgeGraphEntityValue kg_entity) {
var is_doc = false;
var is_provenance = false;
if (!string.IsNullOrEmpty(_kg_DocName)) {
is_doc = GetEntityIsDocument(kg_entity, _kg_DocName);
}
if (!string.IsNullOrEmpty(_kg_ProvenanceName)){
is_provenance = GetEntityIsProvenance(kg_entity, _kg_ProvenanceName);
}
System.Diagnostics.Debug.WriteLine($"{indent} IsDocument: {is_doc}");
System.Diagnostics.Debug.WriteLine($"{indent} IsProvenance: {is_provenance}");
var label = kg_entity.GetLabel();
System.Diagnostics.Debug.WriteLine($"{indent} Label: '{label}'");
}
else if (kg_named_obj_val is KnowledgeGraphRelationshipValue kg_rel) {
var has_entity_ids = kg_rel.GetHasRelatedEntityIDs();
System.Diagnostics.Debug.WriteLine($"{indent} HasRelatedEntityIDs: {has_entity_ids}");
if (has_entity_ids) {
var origin_id = kg_rel.GetOriginID();
var dest_id = kg_rel.GetDestinationID();
System.Diagnostics.Debug.WriteLine($"{indent} OriginID: {origin_id}");
System.Diagnostics.Debug.WriteLine($"{indent} DestinationID: {dest_id}");
}
}
var id = kg_named_obj_val.GetID();
var oid = kg_named_obj_val.GetObjectID();
var type_name = kg_named_obj_val.GetTypeName();
System.Diagnostics.Debug.WriteLine($"{indent} ID: {id}");
System.Diagnostics.Debug.WriteLine($"{indent} ObjectID: {oid}");
System.Diagnostics.Debug.WriteLine($"{indent} TypeName: '{type_name}'");
}
//Base class for named objects (e.g. entities and relationships) _and_
//anonymous objects
private void ProcessGraphObjectValue(
int level, KnowledgeGraphObjectValue kg_obj_val, string prefix = "") {
var spaces = new string(' ', level);
var indent = $"{spaces}{prefix}";
switch (kg_obj_val) {
case KnowledgeGraphEntityValue kg_entity:
PrintGraphNamedObjectValue(level + 1, kg_entity);
break;
case KnowledgeGraphRelationshipValue kg_rel:
PrintGraphNamedObjectValue(level + 1, kg_rel);
break;
default:
break;
}
var keys = kg_obj_val.GetKeys();
var key_names = new List<string>();
foreach (var key in keys)
key_names.Add($"'{key}'");
var key_string = string.Join(',', key_names.ToArray());
System.Diagnostics.Debug.WriteLine($"{indent} Keys: {key_string}");
var count = 0;
foreach (var key2 in keys) {
var key_val = kg_obj_val[key2];
//Recurse to process property key values
ProcessKnowledgeGraphRowValue(level + 1, key_val, $"({count++})['{key2}'] = ");
}
}
//All graph value types
private void ProcessGraphValue(
int level, KnowledgeGraphValue kg_val, string prefix = "") {
var spaces = new string(' ', level);
var indent = $"{spaces}{prefix}";
switch (kg_val) {
case KnowledgeGraphPrimitiveValue kg_prim:
//We should not get a KG Primitive Value from a Query or Search
System.Diagnostics.Debug.WriteLine($"{indent} Primitive:");
var val = kg_prim.GetValue();
ProcessKnowledgeGraphRowValue(level + 1, val, " ");//Recurse
return;
case KnowledgeGraphArrayValue kg_array:
System.Diagnostics.Debug.WriteLine($"{indent} Array:");
var count = (int)kg_array.GetSize();
for (int i = 0; i < count; i++) {
var array_val = kg_array[(ulong)i];
ProcessKnowledgeGraphRowValue(level + 1, array_val, $"[{i}] = ");//Recurse
}
System.Diagnostics.Debug.WriteLine("");
return;
case KnowledgeGraphPathValue kg_path:
System.Diagnostics.Debug.WriteLine($"{indent} Path:");
//Entities:
var entity_count = (long)kg_path.GetEntityCount();
System.Diagnostics.Debug.WriteLine($"{indent} Entities - count: {entity_count}:");
for (long i = 0; i < entity_count; i++) {
ProcessGraphObjectValue(
level + 2, kg_path.GetEntity((ulong)i), $" e[{i}]:");//Recurse
}
//Relationships
var relate_count = (long)kg_path.GetRelationshipCount();
System.Diagnostics.Debug.WriteLine($"\r\n{indent} Relationships - count: {relate_count}:");
for (long i = 0; i < relate_count; i++)
{
ProcessGraphObjectValue(
level + 2, kg_path.GetRelationship((ulong)i), $" r[{i}]:");//Recurse
}
return;
case KnowledgeGraphObjectValue kg_object:
//Anonymous
ProcessGraphObjectValue(level, kg_object, " object:");//Recurse
return;
default:
//Should never get here
var type_string = kg_val.GetType().ToString();
System.Diagnostics.Debug.WriteLine($"{indent} Unknown: '{type_string}'");
return;
}
}
//Process all primitives and graph values
private void ProcessKnowledgeGraphRowValue(
int level, object value, string prefix = ""){
var spaces = new string(' ', level + 1);
var indent = $"{spaces}{prefix}";
if (null == value) {
System.Diagnostics.Debug.WriteLine($"{indent} null");
return;
}
switch (value) {
//Graph values
case KnowledgeGraphValue kg_val:
var kg_type = kg_val.KnowledgeGraphValueType.ToString();
System.Diagnostics.Debug.WriteLine($"{indent} KnowledgeGraphValue: '{kg_type}'");
ProcessGraphValue(level + 1, kg_val, " ");//Recurse
return;
//Primitives
case System.DBNull dbn:
System.Diagnostics.Debug.WriteLine($"{indent} DBNull.Value");
return;
case string str:
System.Diagnostics.Debug.WriteLine($"{indent} '{str}' (string)");
return;
case long l_val:
System.Diagnostics.Debug.WriteLine($"{indent} {l_val} (long)");
return;
case int i_val:
System.Diagnostics.Debug.WriteLine($"{indent} {i_val} (int)");
return;
case short s_val:
System.Diagnostics.Debug.WriteLine($"{indent} {s_val} (short)");
return;
case double d_val:
System.Diagnostics.Debug.WriteLine($"{indent} {d_val} (double)");
return;
case float f_val:
System.Diagnostics.Debug.WriteLine($"{indent} {f_val} (float)");
return;
case DateTime dt_val:
System.Diagnostics.Debug.WriteLine($"{indent} {dt_val} (DateTime)");
return;
case DateOnly dt_only_val:
System.Diagnostics.Debug.WriteLine($"{indent} {dt_only_val} (DateOnly)");
return;
case TimeOnly tm_only_val:
System.Diagnostics.Debug.WriteLine($"{indent} {tm_only_val} (TimeOnly)");
return;
case DateTimeOffset dt_tm_offset_val:
System.Diagnostics.Debug.WriteLine($"{indent} {dt_tm_offset_val} (DateTimeOffset)");
return;
case System.Guid guid_val:
var guid_string = guid_val.ToString("B");
System.Diagnostics.Debug.WriteLine($"{indent} '{guid_string}' (Guid)");
return;
case Geometry geom_val:
var geom_type = geom_val.GeometryType.ToString();
var is_empty = geom_val.IsEmpty;
var wkid = geom_val.SpatialReference?.Wkid ?? 0;
System.Diagnostics.Debug.WriteLine(
$"{indent} geometry: {geom_type}, empty: {is_empty}, sr_wkid {wkid} (shape)");
return;
default:
//Blob?, others?
var type_str = value.GetType().ToString();
System.Diagnostics.Debug.WriteLine($"{indent} Primitive: {type_str}");
return;
}
}
#endregion Read KG Values- ArcGIS Knowledge home page.
- ArcGIS Pro Knowledge - getting started
- Query a knowledge graph - includes use of date-time values and spatial operators. Pro specific query help
- Search the knowledge graph
- NoSQL user-managed data stores
- Neo4j home page.
- Cypher Property Graph Query Language github
- openCypher query language - downloadable pdf.
- Apache Lucene - Query Parser syntax.
- Neo4j Cypher query language.
- ArcGIS Enterprise Query a knowledge graph
