Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Gremlin Base Functionality #47

Merged
merged 7 commits into from

2 participants

@sprjr

[Edited by @aseemk: see previous discussion at issue #44.]

Hey guys,

So I believe I finished a basic, working support of the Gremlin plugin for this module. I did not update the documentation, README, or module version as I know there are other features in the works that pertain to all of those.

Included some starter tests that seem to indicate it is working, I'm sure further testing and usage will prove otherwise but I'll keep working on it. My next steps are to see if transactional & multi-line gremlin work as intended with it as well.

Let me know if you need anything else on this though!

Usage: db.execute(script, optional: params, callback)
Test: npm run gremlin

@aseemk
Owner

Awesome stuff! I'll take a look in depth, and we can hopefully release this, sometime this week!

@aseemk aseemk was assigned
@aseemk aseemk referenced this pull request
Closed

Gremlin support #18

@aseemk
Owner

Hey good news, it turns out my transform code didn't handle paths properly, and in the process of fixing that I went ahead and made it recurse maps too:

0551fb2

So now you shouldn't need any special-case for Gremlin results. But let me know if you still do.

(You can rebase your branch onto the latest develop to test.)

@sprjr

Awesome. Yea, I need to get a bit more thorough with the tests I have after looking at some of yours for the cypher ones with the transform function, I'll see if I can figure out this "rebase" magic you mention, get the update worked in and tested, and get it pushed back up to ya.

@aseemk
Owner

Try this:

# start at your latest commit
$ git checkout develop

# create a pointer to this commit as-is for backup
$ git branch backup

# assuming the remote for this thingdom repo is called upstream, fetch the latest
$ git fetch upstream

# now rebase your develop onto the latest thingdom develop
$ git rebase upstream/develop

# hopefully no conflicts. but if there are, just follow the instructions:
# - fix the conflicts, e.g. using git mergetool
# - then git rebase --continue

# if a seeming-conflict arises but there are no changed files:
# - git rebase --skip

# if at any point things are totally fubar:
# - git rebase --abort

# now if you do a git log, you should see your changes cleanly on top of the ones here
$ git log --graph --decorate --pretty=oneline --abbrev-commit

# then update your github repo (which'll update this pull request)
# assuming you call that remote origin
$ git push origin develop -f

A helpful reference:
http://git-scm.com/book/en/Git-Branching-Rebasing

And for future reference, you might prefer to do this kind of work in a feature (AKA topic) branch instead of in develop. That way you can always easily keep your develop in sync with the upstream develop, and then it's just a simple rebase of your topic branch onto your own develop.

Hope this helps!

Stephen Riva... added some commits
Stephen Rivas JR Added functionality to execture Gremlin Scripts on the graph services…
…, was able to wire up the function and build a working starting point, am not getting callbacks to fire when results are complete
4824654
Stephen Rivas JR Added basic Gremlin test to be expanded on, included it in index of d…
…efault tests
50056af
Stephen Rivas JR Changed the name of the gremlin function to execute after talking it …
…over with Aseem. I also commented out the results mapping

in order to get the functionality tested and working. I plan to write more tests and reapply this section in the near future.
7f2db67
Stephen Rivas JR Included a few initial Gremlin tests and verified data with a local g…
…raph. Results returned as expected. Need to rework a bit of graceful handling, but basic functionality is there
7c2af53
Stephen Rivas JR Quick fix to allow inspecting passed in arguments to .execute() with …
…params functionally being optional now
812e013
Stephen Rivas JR Merged in new util.transform() features from thingdom/node-neo4j-deve…
…lop, worked in

to return functionality with .execute(), tests still passing although they may need flushed
out a bit more thoroughly.
3e374b7
Stephen Rivas JR Removed unecessary type check for gremlin results with new util.trans…
…form() functionality
318d68e
@sprjr

Thanks for the rebase tips and resources, that was a first. I like the idea of using topic branches and utilizing that in future projects though - awesome stuff.

This commit passed the tests I have already written: sprjr@318d68e

@aseemk
Owner

Sorry for the delay here, but finally getting to this, and it looks great! I'm just doing a bit of cleanup -- I'll leave comments here for future reference, but don't worry, I'm already taking care of them.

Thanks again @sprjr!

@aseemk aseemk commented on the diff
lib/GraphDatabase._coffee
@@ -241,6 +241,45 @@ module.exports = class GraphDatabase
catch error
throw adjustError error
+ # wrapper around the Gremlin plugin to execute scripts bundled with
+ # Neo4j. Pass in the Gremlin script as a string, and optionally script
+ # parameters as a map -- recommended for both perf and security!
+ # http://docs.neo4j.org/chunked/snapshot/gremlin-plugin.html
+ # returns...
+ execute: (script, params, _) ->
+ try
+ services = @getServices _
+ endpoint = services.gremlin or
+ services.extensions?.GremlinPlugin?['execute_script']
@aseemk Owner
aseemk added a note

There is no official gremlin endpoint baked in, so simplifying this to only check the plugin endpoint.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@aseemk aseemk commented on the diff
lib/GraphDatabase._coffee
@@ -259,6 +298,16 @@ module.exports = class GraphDatabase
actual.call @, query, params, callback
@aseemk Owner
aseemk added a note

Minor note: the overload helper for query() should remain just after the query() method itself, so I'll move this up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@aseemk aseemk commented on the diff
package.json
@@ -24,7 +24,8 @@
"clean": "rm lib/*.js",
"postinstall": "npm run build",
"profile": "_coffee test/profile",
- "test": "_coffee test"
+ "test": "_coffee test",
+ "gremlin": "_coffee test/gremlin"
@aseemk Owner
aseemk added a note

I assume this was for testing during development. =) I'll go ahead remove this, but I'd really like to port our tests to Mocha, which among other things will let you grep which test cases to run!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@aseemk aseemk commented on the diff
test/gremlin._coffee
((24 lines not shown))
+# convenience aliases
+user0 = users[0]
+user1 = users[1]
+user2 = users[2]
+user3 = users[3]
+user4 = users[4]
+user5 = users[5]
+user6 = users[6]
+
+# test: can query a single user
+results = db.execute """
+ g.v(#{user0.id})
+""", {}, _
+
+assert.ok typeof results, 'object'
+assert.ok typeof results.data.name, 'string' # dislike this because it will throw for the wrong reasons possibly
@aseemk Owner
aseemk added a note

You can use results.data?.name to guard against those. =)

@aseemk Owner
aseemk added a note

But in this case, you don't need to check the name's type since you're already checking its value (via strict equality from assert.equal).

@aseemk Owner
aseemk added a note

Oh also, watch out -- if you're checking typeof, you want to use assert.equal, not assert.ok. (Tricky because instanceof is the opposite.) I'd love to move to expect.js or similar when moving to Mocha.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@aseemk aseemk commented on the diff
test/gremlin._coffee
((21 lines not shown))
+futures = (user.save() for user in users)
+future _ for future in futures
+
+# convenience aliases
+user0 = users[0]
+user1 = users[1]
+user2 = users[2]
+user3 = users[3]
+user4 = users[4]
+user5 = users[5]
+user6 = users[6]
+
+# test: can query a single user
+results = db.execute """
+ g.v(#{user0.id})
+""", {}, _
@aseemk Owner
aseemk added a note

Given that you already test no params at the end, I'll change this to test params support since it doesn't seem like any other case tests that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@aseemk aseemk commented on the diff
test/gremlin._coffee
((48 lines not shown))
+ f1 = user.createRelationshipTo users[i1], 'gremlin_follows', {}
+ f2 = user.createRelationshipTo users[i2], 'gremlin_follows', {}
+ f3 = user.createRelationshipTo users[i3], 'gremlin_follows', {}
+ f1 _
+ f2 _
+ f3 _
+
+# create follow relationships for each user in parallel
+futures = (createFollowRelationships(i) for user, i in users)
+future _ for future in futures
+
+relationships = db.execute """
+ g.v(#{user0.id}).in('gremlin_follows')
+""", {} , _
+
+# above is working, but lib should support returning instances of Node and Relationship if possible
@aseemk Owner
aseemk added a note

It does now! ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@aseemk aseemk merged commit 318d68e into thingdom:develop
@aseemk
Owner

Merged! Thanks and great work again @sprjr. =) I'll post here when this gets published to npm.

@aseemk
Owner

Okay, published as 0.2.17! Woot!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Sep 14, 2012
  1. Added functionality to execture Gremlin Scripts on the graph services…

    Stephen Rivas JR authored
    …, was able to wire up the function and build a working starting point, am not getting callbacks to fire when results are complete
  2. Added basic Gremlin test to be expanded on, included it in index of d…

    Stephen Rivas JR authored
    …efault tests
  3. Changed the name of the gremlin function to execute after talking it …

    Stephen Rivas JR authored
    …over with Aseem. I also commented out the results mapping
    
    in order to get the functionality tested and working. I plan to write more tests and reapply this section in the near future.
  4. Included a few initial Gremlin tests and verified data with a local g…

    Stephen Rivas JR authored
    …raph. Results returned as expected. Need to rework a bit of graceful handling, but basic functionality is there
  5. Quick fix to allow inspecting passed in arguments to .execute() with …

    Stephen Rivas JR authored
    …params functionally being optional now
  6. Merged in new util.transform() features from thingdom/node-neo4j-deve…

    Stephen Rivas JR authored
    …lop, worked in
    
    to return functionality with .execute(), tests still passing although they may need flushed
    out a bit more thoroughly.
  7. Removed unecessary type check for gremlin results with new util.trans…

    Stephen Rivas JR authored
    …form() functionality
This page is out of date. Refresh to see the latest.
View
49 lib/GraphDatabase._coffee
@@ -241,6 +241,45 @@ module.exports = class GraphDatabase
catch error
throw adjustError error
+ # wrapper around the Gremlin plugin to execute scripts bundled with
+ # Neo4j. Pass in the Gremlin script as a string, and optionally script
+ # parameters as a map -- recommended for both perf and security!
+ # http://docs.neo4j.org/chunked/snapshot/gremlin-plugin.html
+ # returns...
+ execute: (script, params, _) ->
+ try
+ services = @getServices _
+ endpoint = services.gremlin or
+ services.extensions?.GremlinPlugin?['execute_script']
@aseemk Owner
aseemk added a note

There is no official gremlin endpoint baked in, so simplifying this to only check the plugin endpoint.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+
+ if not endpoint
+ throw new Error 'Gremlin plugin not installed'
+
+ response = @_request.post
+ uri: endpoint
+ json: if params then {script, params} else {script}
+ , _
+
+ # XXX workaround for neo4j silent failures for invalid queries:
+ if response.statusCode is status.NO_CONTENT
+ throw new Error """
+ Unknown Neo4j error for Gremlin script:
+
+ #{script}
+
+ """
+
+ if response.statusCode isnt status.OK
+ # Database error
+ throw response
+
+ # Success: transform nodes/relationships
+ results = util.transform response.body, this
+ return results
+
+ catch error
+ throw adjustError error
+
# XXX temporary backwards compatibility shim for query() argument order:
do (actual = @::query) =>
@::query = (query, params, callback) ->
@@ -259,6 +298,16 @@ module.exports = class GraphDatabase
actual.call @, query, params, callback
@aseemk Owner
aseemk added a note

Minor note: the overload helper for query() should remain just after the query() method itself, so I'll move this up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+
+ #
+ do (actual = @::execute) =>
+ @::execute = (script, params, callback) ->
+ if typeof params is 'function'
+ callback = params
+ params = null
+
+ actual.call @, script, params, callback
+
# executes a query against the given node index. lucene syntax reference:
# http://lucene.apache.org/java/3_1_0/queryparsersyntax.html
queryNodeIndex: (index, query, _) ->
View
3  package.json
@@ -24,7 +24,8 @@
"clean": "rm lib/*.js",
"postinstall": "npm run build",
"profile": "_coffee test/profile",
- "test": "_coffee test"
+ "test": "_coffee test",
+ "gremlin": "_coffee test/gremlin"
@aseemk Owner
aseemk added a note

I assume this was for testing during development. =) I'll go ahead remove this, but I'd really like to port our tests to Mocha, which among other things will let you grep which test cases to run!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
},
"keywords": [
"neo4j", "graph", "database", "driver", "rest", "api", "client"
View
93 test/gremlin._coffee
@@ -0,0 +1,93 @@
+# will be used for testing gremlin script executions
+# as well as validating the return results are as expected
+
+assert = require('assert')
+neo4j = require('..')
+
+db = new neo4j.GraphDatabase 'http://localhost:7474'
+
+# convenience wrapper
+createNode = (name) ->
+ node = db.createNode {name}
+ node.name = name
+ node.toString = -> name
+ node
+
+#create some nodes
+users = for i in [0..6]
+ createNode "gremlinTest#{i}"
+
+# save in parallel
+futures = (user.save() for user in users)
+future _ for future in futures
+
+# convenience aliases
+user0 = users[0]
+user1 = users[1]
+user2 = users[2]
+user3 = users[3]
+user4 = users[4]
+user5 = users[5]
+user6 = users[6]
+
+# test: can query a single user
+results = db.execute """
+ g.v(#{user0.id})
+""", {}, _
@aseemk Owner
aseemk added a note

Given that you already test no params at the end, I'll change this to test params support since it doesn't seem like any other case tests that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+
+assert.ok typeof results, 'object'
+assert.ok typeof results.data.name, 'string' # dislike this because it will throw for the wrong reasons possibly
@aseemk Owner
aseemk added a note

You can use results.data?.name to guard against those. =)

@aseemk Owner
aseemk added a note

But in this case, you don't need to check the name's type since you're already checking its value (via strict equality from assert.equal).

@aseemk Owner
aseemk added a note

Oh also, watch out -- if you're checking typeof, you want to use assert.equal, not assert.ok. (Tricky because instanceof is the opposite.) I'd love to move to expect.js or similar when moving to Mocha.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+assert.equal results.data.name, user0.name
+
+# test: create relationships between users (same as cypher tests), then query by relationships
+createFollowRelationships = (i, _) ->
+ user = users[i]
+ i1 = (i + 1) % users.length
+ i2 = (i + 2) % users.length
+ i3 = (i + 3) % users.length
+ f1 = user.createRelationshipTo users[i1], 'gremlin_follows', {}
+ f2 = user.createRelationshipTo users[i2], 'gremlin_follows', {}
+ f3 = user.createRelationshipTo users[i3], 'gremlin_follows', {}
+ f1 _
+ f2 _
+ f3 _
+
+# create follow relationships for each user in parallel
+futures = (createFollowRelationships(i) for user, i in users)
+future _ for future in futures
+
+relationships = db.execute """
+ g.v(#{user0.id}).in('gremlin_follows')
+""", {} , _
+
+# above is working, but lib should support returning instances of Node and Relationship if possible
@aseemk Owner
aseemk added a note

It does now! ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+
+
+# handle multiple types of data return
+traversals = db.execute """
+ g.v(#{user0.id}).transform{ [it, it.out.toList(), it.in.count()] }
+""", {}, _
+
+assert.ok traversals instanceof Array
+assert.equal traversals.length, 1
+
+assert.ok traversals[0] instanceof Array
+assert.equal traversals[0].length, 3
+
+assert.ok typeof traversals[0][0], 'object'
+assert.ok traversals[0][1] instanceof Array
+assert.ok typeof traversals[0][2], 'number'
+
+console.log 'Multiple data types appear to have worked with .execute() and util.transform()'
+
+# ensure you can call without params
+
+params_test = db.execute """
+ g.v(#{user0.id})
+""", _
+
+assert.ok typeof params_test, 'object'
+assert.equal params_test.data.name, user0.name
+
+# Should be relatively clear at this point the .execute() function is working with gremlin on some level
+console.log 'Passed initial Gremlin tests'
View
1  test/index._coffee
@@ -1,2 +1,3 @@
require './crud'
require './cypher'
+require './gremlin'
Something went wrong with that request. Please try again.