Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Query Run Time Growing Each Time It Is Executed [DATAMONGO-923] #1847

Closed
spring-projects-issues opened this issue May 1, 2014 · 14 comments
Closed
Assignees
Labels
in: repository Repositories abstraction type: bug A general bug

Comments

@spring-projects-issues
Copy link

spring-projects-issues commented May 1, 2014

Tracy Robinson opened DATAMONGO-923 and commented

I am using the Spring Data Mongo API query methods to store and fetch POJOs in and out of MongoDB. While the query execution times seem slow for the number of documents I am fetching, the more troubling problem is the growth in the execution time with each query. For example, I call the Mongo repository method "pojoRepository.findAll()" and its returns 44 documents in roughly 2.5 seconds. When I run the same exact query again it takes ~7.2 seconds. The next execution increases by about the same about ending up a little less than 13 seconds. The next runs in ~18 seconds, and so on and so on. If there are only 22 documents to fetch, the growth factor is smaller and likewise, if there are 66 documents the growth factor is larger.
I have eliminated MongoDB as the mongo client runs the same query in sub-second time, every time. I have tried using the Cacheable annotation API but, there is no effect, I assume because I am fetching ALL documents where the Cacheable functionality seems to depend on and leverage a field (preferably indexed) as query filter criteria. I was going to try different versions of Java but, thought I would post this problem in the meantime to see if anyone has any ideas about what's going on.

I have attached two images; one showing the test code that calls the query method and the second one showing the console output containing the execution times of the query method. The output shows the test running on 22 documents as opposed to 44. You can see the delta in growth is small as indicated in the above explanation.


Affects: 1.3.3 (Babbage SR2)

Attachments:

@spring-projects-issues
Copy link
Author

spring-projects-issues commented May 1, 2014

Oliver Drotbohm commented

Any chance you provide a tiny sample project to reproduce the issue? I tried something similar (a 100-runs-loop over a findAll() call returning 4 results) and it shows almost identical query execution times (~4ms on my machine).

Are you sure the MongoDB you're talking to is not under load currently? Does running the code on a later release (Codd SR2 - 1.3.2.RELEASE) change things?

@spring-projects-issues
Copy link
Author

spring-projects-issues commented May 1, 2014

Tracy Robinson commented

I don't have time at this moment to produce a sample but, my MongoDB is on my machine in use only by me and the application running this code. I am running 1.3.3 and I haven't tried another release yet. One other thing, I my time units are not correct. It's much worse actually. Instead of ms, it running in seconds. Take a close look at the output attached

@spring-projects-issues
Copy link
Author

spring-projects-issues commented May 1, 2014

Oliver Drotbohm commented

I really have a hard time imagining what could cause 22 objects take almost a second to load. Here's the code I am running:

@Test
public void testname() {

  List<Person> people = new ArrayList<Person>();

  for (int i = 0; i < 100; i++) {
    people.add(new Person("first" + i, "last" + i));
  }

  repository.save(people);

  for (int i = 0; i < 100; i++) {

    long start = new Date().getTime();
    List<Person> result = repository.findAll();
    long end = new Date().getTime();

    System.out.println("Query " + i + " ran " + (end - start) + "ms and returned " + result.size() + " results");
  }
}

Here's the output I get:

Query 0 ran 169ms and returned 107 results
Query 1 ran 72ms and returned 107 results
Query 2 ran 59ms and returned 107 results
Query 3 ran 59ms and returned 107 results
Query 4 ran 44ms and returned 107 results
Query 5 ran 48ms and returned 107 results
…
Query 94 ran 8ms and returned 107 results
Query 95 ran 9ms and returned 107 results
Query 96 ran 17ms and returned 107 results
Query 97 ran 4ms and returned 107 results
Query 98 ran 7ms and returned 107 results
Query 99 ran 7ms and returned 107 results

See how the query times even decrease as MongoDB should be caching the results quite intensively

@spring-projects-issues
Copy link
Author

spring-projects-issues commented May 1, 2014

Tracy Robinson commented

Well that makes two of us that are having a hard time. One thing I am fairly certain of and that is that this problem has nothing to do with MongoDB. Also, to be fair, the object that I am fetching has a list of embedded objects which also have a list of embedded objects. So really my 22 objects are actually closer to 100. To me, that still doesn't justify the performance and it definitely doesn't explain the growth in execution time.

@spring-projects-issues
Copy link
Author

spring-projects-issues commented May 1, 2014

Oliver Drotbohm commented

I am totally with you but have a hard time to operate on something I cannot reproduce. Any chance you share a sample project that shows the degration? Seeing the relevant parts of your domain objects would probably be helpful as well

@spring-projects-issues
Copy link
Author

spring-projects-issues commented May 1, 2014

Tracy Robinson commented

I'm in a classified environment so I have to be careful what I send. It will take me a little bit of time to strip things down to the bare minimum needed to show you. Let me get back to you tomorrow. Right now I have to work on some other stuff. Thanks for working with me! Talk to you tomorrow

@spring-projects-issues
Copy link
Author

spring-projects-issues commented May 1, 2014

Oliver Drotbohm commented

No worries, Tracy. Take your time, just wanted to indicate I am patiently waiting for your next steps :)

@spring-projects-issues
Copy link
Author

spring-projects-issues commented May 1, 2014

Tracy Robinson commented

Greatly appreciated :)

@spring-projects-issues
Copy link
Author

spring-projects-issues commented May 2, 2014

Tracy Robinson commented

Oliver, I didn't get time to create a sample app but, I am trying to play with the spring data mongo version and the spring framework version. Having a hard time finding the match ups between the two. Kind of new to maven so its not coming naturally

@spring-projects-issues
Copy link
Author

spring-projects-issues commented May 5, 2014

Tracy Robinson commented

Oliver, I wanted to provide you with an update. In short, I solved my problem by changing the version of Spring Boot from 1.0.0.RC1 to 1.0.2.RELEASE. I still don't know what exactly the problem was but, I certainly don't have the problem anymore. I repeat execution times now get optimized just like your example demonstrated. Do you recommend ALWAYS using the "CURRENT" release? I would feel better about doing that if there were release notes or change logs for the builds (what changed, what bugs were fixed, what bugs still exist, etc.). Please let me know what you think?

@spring-projects-issues
Copy link
Author

spring-projects-issues commented May 6, 2014

Oliver Drotbohm commented

First of all, I am happy to hear you could get rid off the problem. Generally speaking, we definitely recommend using the latest bugfix version, as - as the name suggests - you're asserted to run on the latest bugfixes and might not run into bugs that have already been fixed.

The announcements (see a sample here) usually contain links to the changelog. They're generally available on our servers for static documentation. The latest one, e.g can always be found here. Another approach is skimming through the issues list for already released versions. Does that make sense?

@spring-projects-issues
Copy link
Author

spring-projects-issues commented May 6, 2014

Tracy Robinson commented

Makes perfect sense. I am going to look at the change log now. I only wonder if there is way to get notified when there is a new release. That would be great. For now I will just plan on periodically manually checking.

Thanks again for your support. By the way - I LOVE coding with the Spring Data Mongo API. So cool!

@spring-projects-issues
Copy link
Author

spring-projects-issues commented May 6, 2014

Oliver Drotbohm commented

Depends on your preferences: we publish announcements on the blog which can be subscribed to, tweet the announcements. If you want a more automated and professional approach, VersionEye might be worth a look. Closing this as cannot reproduce as the issue doesn't seem to present in recent versions

@spring-projects-issues
Copy link
Author

spring-projects-issues commented May 9, 2014

Thomas Darimont commented

According to the discussion this was not an issue of Spring Data Mongodb.
Closing as cannot reproduce

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in: repository Repositories abstraction type: bug A general bug
Projects
None yet
Development

No branches or pull requests

2 participants