-
Notifications
You must be signed in to change notification settings - Fork 49
/
globals.go
311 lines (228 loc) · 7.57 KB
/
globals.go
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
/*
* EliasDB
*
* Copyright 2016 Matthias Ladkau. All rights reserved.
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
/*
Package graph contains the main API to the graph datastore.
Manager API
The main API is provided by a Manager object which can be created with the
NewGraphManager() constructor function. The manager CRUD functionality for
nodes and edges through store, fetch and remove functions. It also provides
the basic traversal functionality which allos the traversal from one node to
other nodes.
Node iterator
All available node keys in a partition of a given kind can be iterated by using
a NodeKeyIterator. The manager can produce these with the NodeKeyIterator()
function.
Fulltext search
All nodes and edges in the datastore are indexed. The index can be queried
using a IndexQuery object. The manager can produce these with the NodeIndexQuery()
or EdgeIndexQuery function.
Transactions
A transaction is used to build up multiple store and delete tasks for the
graph database. Nothing is written to the database before calling commit().
A transaction commit does an automatic rollback if an error occurs
(except fatal disk write errors which might cause a panic).
A trans object can be created with the NewGraphTrans() function.
Rules
(Use with caution)
Graph rules provide automatic operations which help to keep the graph consistent.
Rules trigger on global graph events. The rules SystemRuleDeleteNodeEdges and
SystemRuleUpdateNodeStats are automatically loaded when a new Manager is created.
See the code for further details.
Graph databases
A graph manager handles the graph storage and provides the API for
the graph database. The storage is divided into several databases:
Main database
MainDB stores various meta information such as known node/edge kinds, attributes
or version information.
Names database
Names can be encoded (into a number) or decoded (into a string)
32 bit values for any given node attribute names
16 bit values for any given edge role names
16 bit values for any given edge kind names
Nodes database
Each node kind database stores:
PrefixNSAttrs + node key -> [ ATTRS ]
(a list of attributes of a certain node)
PrefixNSAttr + node key + attr num -> value
(attribute value of a certain node)
PrefixNSSpecs + node key -> map[spec]<empty string>
(a lookup for available specs for a certain node)
PrefixNSEdge + node key + spec -> map[edge key]edgeinfo{other node key, other node kind}]
(connection from one node to another via a spec)
Edges database
Each edge kind database stores:
PrefixNSAttrs + edge key -> [ ATTRS ]
(a list of attributes of a certain edge)
PrefixNSAttr + edge key + attr num -> value
(attribute value of a certain edge)
Index database
The text index managed by util/indexmanager.go. IndexQuery provides access to
the full text search index.
*/
package graph
import "errors"
/*
VERSION of the GraphManager
*/
const VERSION = 1
/*
MainDBEntryPrefix is the prefix for entries stored in the main database
*/
const MainDBEntryPrefix = "\x02"
// MainDB entries
// ==============
/*
MainDBVersion is the MainDB entry key for version information
*/
const MainDBVersion = MainDBEntryPrefix + "ver"
/*
MainDBNodeKinds is the MainDB entry key for node kind information
*/
const MainDBNodeKinds = MainDBEntryPrefix + "nodekind"
/*
MainDBEdgeKinds is the MainDB entry key for edge kind information
*/
const MainDBEdgeKinds = MainDBEntryPrefix + "edgekind"
/*
MainDBParts is the MainDB entry key for partition information
*/
const MainDBParts = MainDBEntryPrefix + "part"
/*
MainDBNodeAttrs is the MainDB entry key for a list of node attributes
*/
const MainDBNodeAttrs = MainDBEntryPrefix + "natt"
/*
MainDBNodeEdges is the MainDB entry key for a list of node relationships
*/
const MainDBNodeEdges = MainDBEntryPrefix + "nrel"
/*
MainDBNodeCount is the MainDB entry key for a node count
*/
const MainDBNodeCount = MainDBEntryPrefix + "ncnt"
/*
MainDBEdgeAttrs is the MainDB entry key for a list of edge attributes
*/
const MainDBEdgeAttrs = MainDBEntryPrefix + "eatt"
/*
MainDBEdgeCount is the MainDB entry key for an edge count
*/
const MainDBEdgeCount = MainDBEntryPrefix + "ecnt"
// Root IDs for StorageManagers
// ============================
/*
RootIDNodeHTree is the root ID for the HTree holding primary information
*/
const RootIDNodeHTree = 2
/*
RootIDNodeHTreeSecond is the root ID for the HTree holding secondary information
*/
const RootIDNodeHTreeSecond = 3
// Suffixes for StorageManagers
// ============================
/*
StorageSuffixNodes is the suffix for a node storage
*/
const StorageSuffixNodes = ".nodes"
/*
StorageSuffixNodesIndex is the suffix for a node index
*/
const StorageSuffixNodesIndex = ".nodeidx"
/*
StorageSuffixEdges is the suffix for an edge storage
*/
const StorageSuffixEdges = ".edges"
/*
StorageSuffixEdgesIndex is the suffix for an edge index
*/
const StorageSuffixEdgesIndex = ".edgeidx"
// PREFIXES for Node storage
// =========================
// Prefixes are only one byte. They should be followed by the node key so
// similar entries are stored near each other physically.
//
/*
PrefixNSAttrs is the prefix for storing attributes of a node
*/
const PrefixNSAttrs = "\x01"
/*
PrefixNSAttr is the prefix for storing the value of a node attribute
*/
const PrefixNSAttr = "\x02"
/*
PrefixNSSpecs is the prefix for storing specs of edges related to a node
*/
const PrefixNSSpecs = "\x03"
/*
PrefixNSEdge is the prefix for storing a link from a node (and a spec) to an edge
*/
const PrefixNSEdge = "\x04"
// Graph events
//=============
/*
EventNodeCreated is thrown when a node was created.
Parameters: partition of created node, created node
*/
const EventNodeCreated = 0x01
/*
EventNodeUpdated is thrown when a node was updated.
Parameters: partition of updated node, updated node, old node
*/
const EventNodeUpdated = 0x02
/*
EventNodeDeleted is thrown when a node was deleted.
Parameters: partition of deleted node, deleted node
*/
const EventNodeDeleted = 0x03
/*
EventEdgeCreated is thrown when an edge was created.
Parameters: partition of created edge, created edge
*/
const EventEdgeCreated = 0x04
/*
EventEdgeUpdated is thrown when an edge was updated.
Parameters: partition of updated edge, updated edge, old edge
*/
const EventEdgeUpdated = 0x05
/*
EventEdgeDeleted is thrown when an edge was deleted.
Parameters: partition of deleted edge, deleted edge
*/
const EventEdgeDeleted = 0x06
/*
EventNodeStore is thrown before a node is stored (always overwriting existing values).
Parameters: partition of node to store, node to store
*/
const EventNodeStore = 0x07
/*
EventNodeUpdate is thrown before a node is updated.
Parameters: partition of node to update, node to update
*/
const EventNodeUpdate = 0x08
/*
EventNodeDelete is thrown before a node is deleted.
Parameters: partition of node to delete, key of node to delete, kind of node to delete
*/
const EventNodeDelete = 0x09
/*
EventEdgeStore is thrown before an edge is stored (always overwriting existing values).
Parameters: partition of stored edge, stored edge
*/
const EventEdgeStore = 0x0A
/*
EventEdgeDelete is thrown before an edge is deleted.
Parameters: partition of deleted edge, key of edge to delete, kind of edge to delete
*/
const EventEdgeDelete = 0x0B
/*
ErrEventHandled is a special error which an event handler can return to
notify the GraphManager that no further action is necessary. No error will
be returned by the GraphManager operation.
*/
var ErrEventHandled = errors.New("Event handled upstream")