JAVA-1125 Change way remove(query) on gridfs is performed to improve performances #171
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…d maxLifeTimeMS to MongoClientURI
…> max write batch size
Added a test to ensure that getMore triggers a MongoExecutionTimeoutException
…not generate the index name if none is provided
Small documentation clarification and testcase update JAVA-1105
…Ensure that key checking and _id creation occurs for all insert paths
…turns false if the server is unable to provide the count. BulkWriteResult.getModifiedCount() now throws if the count is unavailable.
…ting of both write errors and write concern errors
…h either wnote or jnote as write concern errors
…r by removing unnecessary synchronization in DefaultServer
…e value is not the default.
…d fit in to a single update command message.
…hunks inserted into a collection using power-of-two allocator do not waste a lot of space
…properly when the driver is running as an OSGI module.
…ases to 2.6 release.
…the write commands are exceeded. There is one limit for the number of items that are allowed in each command (maxWriteBatchSize from ismaster). There is a second limit for the number of bytes in the encoded message (maxBsonObjectSize from ismaster), with an exception for a write command containing just a single item, which is allowed to exceed that limit.
…d Javadoc annotation.
…direct connections, and in general only use the set of server selectors that make sense for the connection mode and cluster type
…recated method is the one to get the results.
|
Updated/improved by #192 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
While using a query to remove data in the GridFS (both [bucket].files and [bucket].chunks collections) the current driver first issues a select, and then loops over the results to run 2 removes (files and chunks) on each iteration.
On large resultsets, this behaviour can results in thousands of requests x2 (files and chunks).
I can understand that performing files and chunks removal, one after the other, is a way to limit data inconsistency. But there still is a risk.
Thus, as long as the linked removal between files and chunks isn't managed by the server itself, the client side is responsible for checking whether both files and chunks are consistent.
Solution
This pull request for a remove(query) only issues 3 requests :
Fields ids are remembered using a list.
On a single remove this won't be a great improvement, but on large sets of files it'll be worthwhile.
[Fixes : https://jira.mongodb.org/browse/JAVA-1125]