Support/Request - Batch Operations #54

Closed
fisnik opened this Issue Oct 4, 2012 · 12 comments

Comments

Projects
None yet
7 participants

fisnik commented Oct 4, 2012

i could not find a way to get bulk or batch operations working with node-neo4j. i am looking on creating a dozen nodes and relating them to an existing node all in one query.

Any ideas?

Contributor

freeeve commented Oct 4, 2012

You could use Cypher! Might be kind of messy (string building) if you're doing a variable amount of creating, but it can do a dozen pretty easily...

start existing=node(<lookup node>) create (n1 {props...}), (n2 {props}), etc., n1-[:KNOWS]-existing, etc.;

Hey, how come you're not signed up for the meetup in a couple of weeks? :)

Owner

aseemk commented Oct 4, 2012

Cypher is indeed great for this. Thanks @wfreeman.

Neo4j also has a batch API which would support this, but this library doesn't support that yet (issue #12). Hopefully in a future version it will; see the roadmap. =)

@aseemk aseemk closed this Oct 4, 2012

fisnik commented Oct 5, 2012

Great tips... I got this working thanks to the suggestions from wfreeman -
and the Cypher API call on node-neo4j.

On Thu, Oct 4, 2012 at 10:24 AM, Aseem Kishore notifications@github.comwrote:

Cypher is indeed great for this. Thanks @wfreemanhttps://github.com/wfreeman
.

Neo4j also has a batch API which would support this, but this library
doesn't support that yet (issue #6#6).
Hopefully in a future version it will; see the roadmaphttps://github.com/thingdom/node-neo4j/wiki/Roadmap.
=)


Reply to this email directly or view it on GitHubhttps://github.com/thingdom/node-neo4j/issues/54#issuecomment-9142968.

mypark commented Nov 9, 2012

If I wanted to create a node and index it, and put it in a transaction, would putting it both the save and index operation in a batch be the best choice? Should it instead be done in a single cyphher or gremlin query? Seems like a pretty common use case so it would be helpful to do a SaveWithIndex method. What do you think would be the best way to go about it?

Owner

aseemk commented Nov 9, 2012

Assuming you only want to index the node once, at creation-time, Neo4j actually provides an API for unique creation and/or indexing of nodes:

http://docs.neo4j.org/chunked/1.8/rest-api-unique-indexes.html

This library doesn't expose that yet, but a feature request is open for it as issue #14, and @flipside is working on it in pull request #55. =)

Sorry it's not there yet, but until then, hope this helps!

blevine commented Nov 9, 2012

You could also enable auto-indexing and not have to worry about managing the index yourself. See http://docs.neo4j.org/chunked/milestone/auto-indexing.html. This assumes that you always want Nodes that contain specific properties to be indexed.

Owner

aseemk commented Nov 10, 2012

Great point @blevine — we exclusively auto-index as well.

@aseemk aseemk reopened this Nov 10, 2012

@aseemk aseemk closed this Nov 10, 2012

Sorry for the long question but: with these different options what would be the reference implementation for a standard scenario, say creating a user node with a unique email. Is it to use an auto index, and if so does an auto index support case-insensitve exact match. What if I also need to do a full text index at the time of creation on the description property of the user, as well, should I just add full-text auto index for that as well. What about when different nodes share the same property name, say user.description and article.description. Is the only solution using an auto-index be to change the property name between the two so that they would be different?

Why is this issue closed? Surely someone out there other than I is using Node.js and requires ACID transactions. I'd love to help extend node-neo4j to cover this (and better indexes as well). Are there any thoughts on the direction of the API in this area?

Contributor

freeeve commented Apr 16, 2013

Yeah, batch is still useful. It also allows you to push more operations through a single request, which can improve performance a lot in some use cases.

Something that either takes an array of operations to perform, or an object that you can append operations to (with an internal array), and then submit to neo would probably be good ways.

Owner

aseemk commented Apr 22, 2013

Hey @Pictorious, I'm so sorry for the delay in responding to your question. I searched my email and didn't seem to ever get a notification for it!

The auto-index is case-sensitive exact match. What we do for things like email and usernames is to store two properties on our user nodes: the original value, and a normalized value, and we (auto-)index the normalized one. This also gives us flexibility in normalizing beyond lowercasing, e.g. stripping punctuation from usernames.

If you want an additional fulltext index, you currently do have to create and manage one yourself manually currently. I realize that this library unfortunately doesn't currently let you create indexes with custom configuration. But indexing is getting overhauled in Neo4j 2.0, and a big thing they want to do is have all indexes be auto-indexes. So if this isn't urgent, indexing in general should be much more flexible and useful in Neo4j 2.0 onwards.

Hope that answers your q's!

Owner

aseemk commented Apr 22, 2013

@kwanalyssa: sorry, I must have made a typo in my initial reply at the top — this issue is closed because it's a duplicate of #12, not #6 as I originally wrote. (Will go up there and edit.)

A batch API is still useful, but as #12 says, mutable Cypher now actually gets you ACID transactions over the REST API for most scenarios. The one exception is indexing, and that pull request is still ongoing. I've been meaning to finish it up and merge it.

Otherwise, let's continue the discussion on the batch API in #12. Cheers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment