findOne performance issue with depth >= 1

[clement94]

Fetching data with findOne method on an object of class C is impacted by incoming relation of type R whereas this kind of relation is not specified in the java class C (suppose there is a relation between class A and class C but the Set is only specified on class A)
I think that fetching related datas (thanks to depth parameter) should not get datas that java classes do not need. Running time should not be so impacted by this kind of relation since user do not want them.
I already added my problem on stackoverflow : http://stackoverflow.com/questions/32884984/sdn4-or-neo4j-ogm-performances-issue in which class A is Element, class C is Attribute and relation R is Value

Thanks for your help

[jexp]

I agree, the fetching of data should explicitely specify the relationship-types that are asked for.

And if there are two directions asked for we could collect them like this:

MATCH path = (m:Label)-[:FOO|:BAR]->() WHERE id(m) = {id}
WITH m, collect(path) as paths
MATCH path = (m:Label)<-[:ALPHA|:BETA]-()
RETURN m, collect(path) as paths, path

Am 02.10.2015 um 09:31 schrieb clement94 notifications@github.com:

Fetching data with findOne method on an object of class C is impacted by incoming relation of type R whereas this kind of relation is not specified in the java class C (suppose there is a relation between class A and class C but the Set is only specified on class A)
I think that fetching related datas (thanks to depth parameter) should not get datas that java classes do not need. Running time should not be so impacted by this kind of relation since user do not want them.
I already added my problem on stackoverflow : http://stackoverflow.com/questions/32884984/sdn4-or-neo4j-ogm-performances-issue http://stackoverflow.com/questions/32884984/sdn4-or-neo4j-ogm-performances-issue in which class A is Element, class C is Attribute and relation R is Value

Thanks for your help

—
Reply to this email directly or view it on GitHub #55.

[Leward]

I also agree, it would be a very welcomed improvement. I encounter the same kind of issue.

At the moment I manage to have the behavior I expected by using custom queries but it increases the complexity of the code.

[jexp]

Sure, we'll look into this for one of the next releases.

[atg103]

There are a few things we can talk about doing here. Maybe we allow users to ask for relationships, maybe we restrict by relationship direction or maybe we look for related nodes labelled in a specific way. Maybe we do all of these things!

That said, what I've just described is exactly what a Cypher query is for. We need to be careful that we don't just end up overloading findOne et al to provide the functionality already available through @Query. We will investigate using things like fetch depth and optimising the generated MATCH queries to potentially address this issue, but we need to be sure that we're investing our endeavour to tackle a true performance problem and not just focussing specific outlying use cases.

[nayish1]

This question seems to be relevant:
http://stackoverflow.com/questions/33486334/load-an-object-in-neo4j-2-3-ogm-depth-of-2-very-slow

[clement94]

@jexp do you have any idea about when this will be fix please ? Thanks a lot

[luanne]

@clement94 we'll be discussing this at our planning meeting next week- we'll have an update post that.

[nayish1]

Was this ever discussed? is there any plan to solve this problem?

My development team found it impossible to work with OGM and downgraded back down to Spring Data Neo4j 3.4. This bug causes it to be impossible to load entities with their relationships which is the whole reason we use Neo4j. If at least we could load the ids of the related entites then we can at least continue working with AspectJ to load the relationships (not the best solution but at least something).

Is there anyone out there using OGM in production? how do u guys solve the problem that you can't load entities like categories that are very well connected?

[luanne]

@nayish It was discussed and it will be addressed in a future release. Curious to know how many relationships your entity has

[nayish1]

We have a category entity which holds the category name, color and some more information about the category. The number of relationships varies between the different categories, but each is connected to arounnd 20,000 nodes (and this number is growing).
The problem arises when loading an entity that has a relationship to a category with a depth bigger than one (since one gets us to the category and the second level gets us to 20,000 nodes).
In our java code the category has no relationships, only the other entites have to it.

[luanne]

Just so that I understand your problem better- you're asking that the OGM supports a depth per relationship type? So when loading an entity that has a relationship to a category, the category is loaded to depth 1 but the rest of the related entities are loaded to depth n? Or are you asking that the OGM understands how your domain objects are wired and load entities that match that model?
When custom queries support mapping results back to entities, would this help in the above case?

[nayish1]

Both are good, each has their own advantages. The one I think best addresses my problem is the understanding of the object model since it is unwanted behavior that my category entity loads all its related nodes even though I will only ever be using the nodes mapped in my object. in this sense, the fact that it doesnt understand my object wiring is a performance bug while the choosing custom relationship depths is more of a feature.

Does this answer your question? Not quite sure about your last question.

[natioskar]

I encountered the same problem and also ended up not upgrading to ogm/spring data neo4j 4.

In my case I have song nodes that are connected to genre nodes but since most genres have thousands of songs I end up touching too much of the graph by selecting more than one level.

If OGM used my object wiring for the relationships there would be no problem since my genre node has no mapped relationships, Only the songs have.

Will this problem be solved soon?

[dhruvasa-zz]

I was also trying to upgrade to SDN4 but couldn't because of said issue. Also QueryResult with no support for object/Collection of objects mapping was also creating problems in terms of clean code.

Holding back till both of these get solved !!

[clement94]

@luanne and @nayish nayish said "My development team found it impossible to work with OGM and downgraded back down to Spring Data Neo4j 3.4". I am exactly in the same situation. For us it's a critical issue which lead us to not use SDN4. Maybe it would be great if it would be possible to add a parameter on @relation (something like autofetch=false|true) wich stop loadding deep entities when it is false (or load only the id)

[luanne]

Thanks everyone for your comments. At the moment, I can confirm that this is on our to-do list and we'll update this issue when it's been assigned to a release.

[pablo-quilez]

Hi! First I want to thank you all for your wonderful work. This feature is an important requirement for us. We want to retrieve all children at once (a tree) with depth = -1. Do you have an approximate date of release? Thanks!

[nayish1]

Did this make it in to the 2.0.1 version. If not, is there an ETA on a solution?

[ancisse]

Hello,
I have the same problem. Is that a correction will be made ​​on the bug clement94 ?

[mangrish]

FYI All: We are working on this issue actively at the moment. It is slated for release with OGM 3.0 and SDN 5.0.

[ancisse]

Thanks!
Do you have an idea of ​​the date of availability of the correction?

[mangrish]

@ancisse I would say late April for a release candidate for both the OGM and SDN. We are exploring various options at the moment.

[frant-hartm]

This has been addressed in 3.0.0 release by providing query load strategy based on schema, see #70 and related commits and is available in RC1 or latest snapshot.

We would love to hear your feedback.