Skip to content

Commit b222f13

Browse files
feat: Add upsert and unique checking in local mode (#104)
**Description** This PR adds upsert and unique checking to modusGraph in "local" mode. **Checklist** - [x] Code compiles correctly and linting passes locally - [ ] For all _code_ changes, an entry added to the `CHANGELOG.md` file describing and linking to this PR - [x] Tests added for new functionality, or regression tests for bug fixes added as applicable
1 parent 12342e3 commit b222f13

File tree

13 files changed

+1375
-129
lines changed

13 files changed

+1375
-129
lines changed

README.md

Lines changed: 85 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -114,7 +114,8 @@ Connects to a database stored locally on the filesystem. This mode doesn't requi
114114
database server and is perfect for development, testing, or embedded applications. The directory
115115
must exist before connecting.
116116

117-
File-based databases do not support concurrent access from separate processes.
117+
File-based databases do not support concurrent access from separate processes. Further, there can
118+
only be one file-based client per process.
118119

119120
```go
120121
// Connect to a local file-based database
@@ -131,6 +132,8 @@ Connects to a Dgraph cluster. For more details on the Dgraph URI format, see the
131132
client, err := mg.NewClient("dgraph://hostname:9080")
132133
```
133134

135+
You can have multiple remote clients per process provided the URIs are distinct.
136+
134137
### Configuration Options
135138

136139
modusGraph provides several configuration options that can be passed to the `NewClient` function:
@@ -155,6 +158,15 @@ connections.
155158
client, err := mg.NewClient(uri, mg.WithPoolSize(20))
156159
```
157160

161+
#### WithMaxEdgeTraversal(int)
162+
163+
Sets the maximum number of edges to traverse when querying. The default is 10 edges.
164+
165+
```go
166+
// Set max edge traversal to 20 edges
167+
client, err := mg.NewClient(uri, mg.WithMaxEdgeTraversal(20))
168+
```
169+
158170
#### WithLogger(logr.Logger)
159171

160172
Configures structured logging with custom verbosity levels. By default, logging is disabled.
@@ -187,6 +199,9 @@ Every struct that represents a node in your graph should include:
187199
```go
188200
type MyNode struct {
189201
// Your fields here with appropriate tags
202+
Name string `json:"name,omitempty" dgraph:"index=exact"`
203+
Description string `json:"description,omitempty" dgraph:"index=term"`
204+
CreatedAt time.Time `json:"createdAt,omitempty" dgraph:"index=day"`
190205

191206
// These fields are required for Dgraph integration
192207
UID string `json:"uid,omitempty"`
@@ -198,38 +213,38 @@ type MyNode struct {
198213

199214
modusGraph uses struct tags to define how each field should be handled in the graph database:
200215

201-
| Directive | Option | Description | Example |
202-
| ----------- | -------- | ------------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
203-
| **index** | exact | Creates an exact-match index for string fields | Name string `json:"name" dgraph:"index=exact"` |
204-
| | hash | Creates a hash index (same as exact) | Code string `json:"code" dgraph:"index=hash"` |
205-
| | term | Creates a term index for text search | Description string `json:"description" dgraph:"index=term"` |
206-
| | fulltext | Creates a full-text search index | Content string `json:"content" dgraph:"index=fulltext"` |
207-
| | int | Creates an index for integer fields | Age int `json:"age" dgraph:"index=int"` |
208-
| | geo | Creates a geolocation index | Location `json:"location" dgraph:"index=geo"` |
209-
| | day | Creates a day-based index for datetime fields | Created time.Time `json:"created" dgraph:"index=day"` |
210-
| | year | Creates a year-based index for datetime fields | Birthday time.Time `json:"birthday" dgraph:"index=year"` |
211-
| | month | Creates a month-based index for datetime fields | Hired time.Time `json:"hired" dgraph:"index=month"` |
212-
| | hour | Creates an hour-based index for datetime fields | Login time.Time `json:"login" dgraph:"index=hour"` |
213-
| | hnsw | Creates a vector similarity index | Vector \*dg.VectorFloat32 `json:"vector" dgraph:"index=hnsw(metric:cosine)"` |
214-
| **type** | geo | Specifies a geolocation field | Location `json:"location" dgraph:"type=geo"` |
215-
| | datetime | Specifies a datetime field | CreatedAt time.Time `json:"createdAt" dgraph:"type=datetime"` |
216-
| | int | Specifies an integer field | Count int `json:"count" dgraph:"type=int"` |
217-
| | float | Specifies a floating-point field | Price float64 `json:"price" dgraph:"type=float"` |
218-
| | bool | Specifies a boolean field | Active bool `json:"active" dgraph:"type=bool"` |
219-
| | password | Specifies a password field (stored securely) | Password string `json:"password" dgraph:"type=password"` |
220-
| **count** | | Creates a count index | Visits int `json:"visits" dgraph:"count"` |
221-
| **unique** | | Enforces uniqueness for the field (remote Dgraph only) | Email string `json:"email" dgraph:"index=hash unique"` |
222-
| **upsert** | | Allows a field to be used in upsert operations (remote Dgraph only) | UserID string `json:"userID" dgraph:"index=hash upsert"` |
223-
| **reverse** | | Creates a bidirectional edge | Friends []\*Person `json:"friends" dgraph:"reverse"` |
224-
| **lang** | | Enables multi-language support for the field | Description string `json:"description" dgraph:"lang"` |
216+
| Directive | Option | Description | Example |
217+
| ----------- | -------- | ----------------------------------------------- | ------------------------------------------------------------------------------------ |
218+
| **index** | exact | Creates an exact-match index for string fields | Name string `json:"name" dgraph:"index=exact"` |
219+
| | hash | Creates a hash index (same as exact) | Code string `json:"code" dgraph:"index=hash"` |
220+
| | term | Creates a term index for text search | Description string `json:"description" dgraph:"index=term"` |
221+
| | fulltext | Creates a full-text search index | Content string `json:"content" dgraph:"index=fulltext"` |
222+
| | int | Creates an index for integer fields | Age int `json:"age" dgraph:"index=int"` |
223+
| | geo | Creates a geolocation index | Location `json:"location" dgraph:"index=geo"` |
224+
| | day | Creates a day-based index for datetime fields | Created time.Time `json:"created" dgraph:"index=day"` |
225+
| | year | Creates a year-based index for datetime fields | Birthday time.Time `json:"birthday" dgraph:"index=year"` |
226+
| | month | Creates a month-based index for datetime fields | Hired time.Time `json:"hired" dgraph:"index=month"` |
227+
| | hour | Creates an hour-based index for datetime fields | Login time.Time `json:"login" dgraph:"index=hour"` |
228+
| | hnsw | Creates a vector similarity index | Vector \*dg.VectorFloat32 `json:"vector" dgraph:"index=hnsw(metric:cosine)"` |
229+
| **type** | geo | Specifies a geolocation field | Location `json:"location" dgraph:"type=geo"` |
230+
| | datetime | Specifies a datetime field | CreatedAt time.Time `json:"createdAt" dgraph:"type=datetime"` |
231+
| | int | Specifies an integer field | Count int `json:"count" dgraph:"type=int"` |
232+
| | float | Specifies a floating-point field | Price float64 `json:"price" dgraph:"type=float"` |
233+
| | bool | Specifies a boolean field | Active bool `json:"active" dgraph:"type=bool"` |
234+
| | password | Specifies a password field (stored securely) | Password string `json:"password" dgraph:"type=password"` |
235+
| **count** | | Creates a count index | Visits int `json:"visits" dgraph:"count"` |
236+
| **unique** | | Enforces uniqueness for the field | Email string `json:"email" dgraph:"index=hash unique"` |
237+
| **upsert** | | Allows a field to be used in upsert operations | UserID string `json:"userID" dgraph:"index=hash upsert"` |
238+
| **reverse** | | Creates a bidirectional edge | Friends []\*Person `json:"friends" dgraph:"reverse"` |
239+
| **lang** | | Enables multi-language support for the field | Description string `json:"description" dgraph:"lang"` |
225240

226241
### Relationships
227242

228243
Relationships between nodes are defined using struct pointers or slices of struct pointers:
229244

230245
```go
231246
type Person struct {
232-
Name string `json:"name,omitempty" dgraph:"index=exact"`
247+
Name string `json:"name,omitempty" dgraph:"index=exact upsert"`
233248
Friends []*Person `json:"friends,omitempty"`
234249
Manager *Person `json:"manager,omitempty"`
235250

@@ -268,6 +283,10 @@ Advanced querying is required to properly bind reverse edges in query results. S
268283

269284
modusGraph provides a simple API for common database operations.
270285

286+
Note that in local-mode, unique fields are limited to the top-level object. Fields marked as unique
287+
in embedded or lists of embedded objects that have `unique` tags will not be checked for uniqueness
288+
when the top-level object is inserted.
289+
271290
### Inserting Data
272291

273292
To insert a new node into the database:
@@ -292,9 +311,40 @@ if err != nil {
292311
fmt.Println("Created user with UID:", user.UID)
293312
```
294313

314+
### Upserting Data
315+
316+
modusGraph provides a simple API for upserting data into the database.
317+
318+
Note that in local-mode, upserts are only supported on the top-level object. Fields in embedded or
319+
lists of embedded objects that have `upsert` tags will be ignored when the top-level object is
320+
upserted.
321+
322+
```go
323+
ctx := context.Background()
324+
325+
user := User{
326+
Name: "John Doe", // this field has the `upsert` tag
327+
Email: "john@example.com",
328+
Role: "Admin",
329+
}
330+
331+
// Upsert the user into the database
332+
// If "John Doe" does not exist, it will be created
333+
// If "John Doe" exists, it will be updated
334+
err := client.Upsert(ctx, &user)
335+
if err != nil {
336+
log.Fatalf("Failed to upsert user: %v", err)
337+
}
338+
339+
```
340+
295341
### Updating Data
296342

297-
To update an existing node, first retrieve it, modify it, then save it back:
343+
To update an existing node, first retrieve it, modify it, then save it back.
344+
345+
Note that in local-mode, unique update checks are only supported on the top-level object. Fields in
346+
embedded or lists of embedded objects that have `unique` tags will not be checked for uniqueness
347+
when the top-level object is updated.
298348

299349
```go
300350
ctx := context.Background()
@@ -534,10 +584,14 @@ These operations are useful for testing or when you need to reset your database
534584
modusGraph has a few limitations to be aware of:
535585

536586
- **Unique constraints in file-based mode**: Due to the intricacies of how Dgraph handles unique
537-
fields and upserts in its core package, unique field checks and upsert operations are not
538-
supported (yet) when using the local (file-based) mode. These operations work properly when using
539-
a full Dgraph cluster, but the simplified file-based mode does not support the constraint
540-
enforcement mechanisms required for uniqueness guarantees.
587+
fields in its core package, when using file-based mode, unique field checks are only supported at
588+
the top level object that is being in/upserted or updated. Embedded or lists of embedded objects
589+
that have unique tags will NOT be checked for uniqueness when the top-level object is in/upserted
590+
or updated.
591+
592+
- **Upsert operations**: Upsert operations are only supported on the top-level object. Fields in
593+
embedded or lists of embedded objects that have upsert tags will be ignored when the top-level
594+
object is upserted.
541595

542596
- **Schema evolution**: While modusGraph supports schema inference through tags, evolving an
543597
existing schema with new fields requires careful consideration to avoid data inconsistencies.

buf_server.go

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,8 @@ func (s *serverWrapper) Query(ctx context.Context, req *api.Request) (*api.Respo
4646
s.engine.logger.V(2).Info("Query using namespace", "namespaceID", ns.ID())
4747

4848
if len(req.Mutations) > 0 {
49+
s.engine.logger.V(3).Info("Mutating", "mutations", req.Mutations)
50+
4951
uids, err := ns.Mutate(ctx, req.Mutations)
5052
if err != nil {
5153
return nil, fmt.Errorf("engine mutation error: %w", err)

0 commit comments

Comments
 (0)