Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lazy Load for DbRef [DATAMONGO-348] #1281

Closed
spring-projects-issues opened this issue Dec 11, 2011 · 16 comments
Closed

Lazy Load for DbRef [DATAMONGO-348] #1281

spring-projects-issues opened this issue Dec 11, 2011 · 16 comments
Assignees
Labels
has: votes-jira in: mapping Mapping and conversion infrastructure type: enhancement A general enhancement
Milestone

Comments

@spring-projects-issues
Copy link

Steve Mosley opened DATAMONGO-348 and commented

DbRef's appear to be loaded eagerly.

Would be nice if there was support for storing DbRef's on a document but being able to lazy (or manually) load them


Affects: 1.0 M5

Reference URL: http://forum.springsource.org/showthread.php?113321-Support-for-lazy-loading-in-Spring-Data-Document

Issue Links:

Referenced from: commits ea33e8b, 8fb390e, c3f8f1e, df1c449, 785a7ef, fd60d32, c63994e, 1bb29f3, 5e47df1, f33b4ec, 1b2f6a1, a66e762, 1055487, 88e7c8b, c22a74c, b70c60b, 1d299c3, b808fd3, badcf06, ead4733, 1e936da, c972524, cb615f1, 4b5dab3

26 votes, 23 watchers

@spring-projects-issues
Copy link
Author

Steven Sheehy commented

In addition to lazy loading, I would like to add my support for the manually loading of DBRefs. We are using DBRefs to point to multiple collections and using ObjectIds when the collection name cannot vary as recommended by Mongo (http://www.mongodb.org/display/DOCS/Database+References). As a result, we had to write a custom converter to bypass the automatic fetching of DBRefs that Spring Data defaults to. It would be nice if we didn't have to use a converter and could specify this behavior via the @DbRef annotation

@spring-projects-issues
Copy link
Author

Andrew Bethell commented

Is another way around this just to not have spring-data map certain fields? If using the MongoRepository - you could do this with the @Query annotation and specify which fields you want returned, and just omit the DBRefs field

@spring-projects-issues
Copy link
Author

Oliver Drotbohm commented

Until we eventually get to solve this issue in a nicer way, you can simply have DBRef properties in your domain objects (which is not ideal of course but get's the job done for now) and resolve them manually.

class Account {
  @DBRef
  DBRef owner;
}

Account account = …;
MongoConverter converter = …;
User user = converter.read(User.class, account.owner.fetch());

@spring-projects-issues
Copy link
Author

Tito George commented

Work around with DBRef did not work. It failed while saving.
My code is like this

class Request{
@Id
private ObjectId id;
@DBRef
private com.mongodb.DBRef who;
@DBRef
private com.mongodb.DBRef whom;
@DBRef
private com.mongodb.DBRef group;
}
AddRequest addRequest = new AddRequest();
addRequest.setWho(new DBRef(mongoTemplate.getDb(), "users" ,who.getId()));
addRequest.setWhom(new DBRef(mongoTemplate.getDb(), "users" ,whom.getId()));
addRequest = addRequestDataRepository.save(addRequest);

Failed with below ST

java.lang.StackOverflowError
	at sun.reflect.generics.reflectiveObjects.WildcardTypeImpl.hashCode(WildcardTypeImpl.java:212)
	at java.util.Arrays.hashCode(Arrays.java:3655)
	at sun.reflect.generics.reflectiveObjects.ParameterizedTypeImpl.hashCode(ParameterizedTypeImpl.java:190)
	at org.springframework.util.ObjectUtils.nullSafeHashCode(ObjectUtils.java:336)
	at org.springframework.data.util.TypeDiscoverer.hashCode(TypeDiscoverer.java:367)
	at org.springframework.data.util.ParentTypeAwareTypeInformation.hashCode(ParentTypeAwareTypeInformation.java:79)
	at org.springframework.util.ObjectUtils.nullSafeHashCode(ObjectUtils.java:336)
	at org.springframework.data.util.ParentTypeAwareTypeInformation.hashCode(ParentTypeAwareTypeInformation.java:79)
	at org.springframework.util.ObjectUtils.nullSafeHashCode(ObjectUtils.java:336)

From Eclipse when StackOverflowError is caught.

Thread [main] (Suspended (exception StackOverflowError))
	MongoMappingContext(AbstractMappingContext<E,P>).addPersistentEntity(TypeInformation<?>) line: 307
	AbstractMappingContext$PersistentPropertyCreator.doWith(Field) line: 413	                  
	ReflectionUtils.doWithFields(Class<?>, FieldCallback, FieldFilter) line: 570	                  
	MongoMappingContext(AbstractMappingContext<E,P>).addPersistentEntity(TypeInformation<?>) line: 283
	AbstractMappingContext$PersistentPropertyCreator.doWith(Field) line: 413                          
	ReflectionUtils.doWithFields(Class<?>, FieldCallback, FieldFilter) line: 570	                  
	MongoMappingContext(AbstractMappingContext<E,P>).addPersistentEntity(TypeInformation<?>) line: 283

@spring-projects-issues
Copy link
Author

David Anderson commented

Tito,

Oliver's Account class had an error in it, it should have been:

class Account {
  @DBRef
  User owner;
}

@spring-projects-issues
Copy link
Author

Tito George commented

David,
I have a class called User, belongs to collection users. User contains a list called requests(another collection called request).
Class request was like below initially.

class Request{
@Id
private ObjectId id;
@DBRef
private User who;
@DBRef
private User whom;
@DBRef
private User group;
}

User.java
User{
@DBRef
List<Requests> requests;
}

Say an instance of User1 can contain list requests and request object can contain User1 (in its fields who or whom). I am able to save User as well as Request. But while querying i am getting https://jira.springsource.org/browse/DATAMONGO-488.

So I was hoping Oliver's workaround would solve this. But its failing wile saving.

@spring-projects-issues
Copy link
Author

Sebastian Julius commented

Are there any plans when this feature will be integrated? We're currently creating 4-times more requests to the DB then required. Therefore it'd be brilliant to have lazy loading. Thanks in advance

@spring-projects-issues
Copy link
Author

regis commented

The morphia project also provided this feature.

https://github.com/mongodb/morphia/blob/master/morphia/src/main/java/org/mongodb/morphia/annotations/Reference.java

 
@Entity("employees")
class Employee {
  // auto-generated, if not set (see ObjectId)
  @Id ObjectId id;


  //refs are stored**, and loaded automatically
  @Reference(lazy = true)
  List<Employee> underlings = new ArrayList<Employee>();

}

Why the spring data team doesnt reply ?

Hey guys the ticket is 3 years OLD....

@spring-projects-issues
Copy link
Author

Mikkel Dan-Rognlie commented

I see that @thomasd added sprint "The Road to Codd M1" to this issue. According to https://github.com/spring-projects/spring-data-commons/wiki/Release-Train-Codd, it seems we will probable have this feature in the near future. Can you guys confirm that you will be working on it? That would be awesome!

@spring-projects-issues
Copy link
Author

Oliver Drotbohm commented

Yes

@spring-projects-issues
Copy link
Author

Thomas Darimont commented

How should the lazy loading behaviour be enabled and what should be the default?

I would propose something like this, with lazy=false as the default:

@DBRef(lazy = true) List<User> fans;

Should we support lazy loading for all possible concrete types or just interfaces types or even just for collection types?
If the raw type has to be an interface we could simply return a JDK Proxy with an appropriate InvocationHandler that checks an "initialised"-flag on every method call. If not yet initialised the proxy will resolve the DBRef, store the result in a field and return that on subsequent invocations.

If we have to support arbitrary types, then we have to enforce some restrictions, e.g. like the fields of the type needn't be accessed directly - just via accessor methods. We could then generate a custom subclass of the given type (via CGLIB / ASM + (maybe) Objenesis for object construction) where we inject a proper initialisation check into the relevant method bodies and perform the required lazy loading logic if necessary and then delegate to the base class implementation

@spring-projects-issues
Copy link
Author

@spring-projects-issues
Copy link
Author

regis commented

We should stick by default to eager loading (like you propose), we will still compatible with the old behavior. And not breaking apps when they are going to update the version of spring mongo.

@spring-projects-issues
Copy link
Author

Mikkel Dan-Rognlie commented

I also think eager loading should be the default. Maybe we could have the possibility to change it globally as a property on MappingMongoConverter. Like for instance the disable-validation attribute? Then one could override it with individual @DBRef(lazy = true|false) on entities.

If it's not too complicated I think it would be nice to support arbitrary types using CGLIB / ASM. But as you said, maybe it is not desirable with the restrictions on field access (could be perceived as 'magical' etc). If it makes the implementation easier it would be ok to require an interface in order to use JDK proxies.

I don´t think it should be limited to collection types. If you in a query gets a long list of documents, where each doc has deep nested documents, which at arbitrary levels of nesting has one @DBRef, it is still a lot of extra fetches even though the individual @DBRef fields are not collections

@spring-projects-issues
Copy link
Author

Thomas Darimont commented

Hi,

could you please give us some feedback on our current POC for lazy loading of MongoDB associations:
https://github.com/spring-projects/spring-data-mongodb/compare/DATAMONGO-348

At the moment we support lazy loading for interface, concrete types, as far as they provide an appropriate parameterless ctor.
The @PersistenceConstructor annotation is supported as well.
If there is a need to support arbitrary constructors, we could also try to back-port the Objenesis stuff from
Spring Framework 4.0 (spring-projects/spring-framework#327)
The actual loading is performed only once and is triggered when a method on the proxy is called.

If you want to give it a try, then please do the following:

  • Change the version of your mongodb dependency to: 1.4.0.DATAMONGO-348-SNAPSHOT
  • Add the spring source snapshot-repository
<repositories>
    <repository>
        <id>repository.springsource.snapshot</id>
        <name>SpringSource Snapshot Repository</name>
        <url>http://repo.springsource.org/snapshot</url>
    </repository>
</repositories>

Cheers,
Thomas

@spring-projects-issues
Copy link
Author

Oliver Drotbohm commented

This merged into master. I added an optional dependency to Objenesis to be able to create proxies for classes without a default constructor. Feedback appreciated

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
has: votes-jira in: mapping Mapping and conversion infrastructure type: enhancement A general enhancement
Projects
None yet
Development

No branches or pull requests

2 participants