Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make ModelType.Count(filter_query) to perform server-side document count queries #24

Open
eshork opened this issue Dec 19, 2020 · 3 comments
Milestone

Comments

@eshork
Copy link
Owner

eshork commented Dec 19, 2020

Under current implementation, Result.Count() needs to retrieve and store all records within the query result set before it can provide the total count of query matching records. And because of this, it cannot operate when Result.IsStreaming() == true

@eshork
Copy link
Owner Author

eshork commented Dec 19, 2020

It would also be possible to add new functionality such that Result.Count() could operate under Result.Streaming() conditions, but it seems impractical.

One way to do this would be to retrieve all result records from the driver cursor, since the cursor does not provide any sort of count mechanism other than to simply read the records until no more are available. However, reading the records under Streaming conditions means we cannot store them for later access. The Result would only be usable once, to give the return value of Count(). This way also passed the object data across the network even though it is only being used for incremental tabulation, which is very inefficient.

Alternatively, invoking Result.Count could perform a separate out-of-band query that would perform the operation on the MongoDB server side. However, since this is a separate out-of-band query, the returned result may not match the number of records available within the previously generated Result.cursor. Making this value accessible via a Result method would imply a much stronger association than is feasible.

@eshork
Copy link
Owner Author

eshork commented Dec 19, 2020

We could add a Count(filter_query) method onto ModelType, which would perform a dedicated count query that could run on the Mongo server side.

Yes, this is still performing a 2nd query in order to acquire the value, but because it is not simply a method of a Result, it creates a more clear distinction that this action will perform a new query that does not have strong association with any existing Result object.

The downside here is that we don't have a typically volatile object with which to associate the ModelType.Count() query. So, we have no object to tie any cache state with, and so every invocate must be a query back to the db server. If a library consumer uses ModelType.Count() within a loop comparison, they're going to end up running a lot of queries against the server.

Example:

// this is the worst idea ever
for i := 0; i < myModelType.Count(); i++ {
  // anything at all
}

We should be able to note this problem within the documentation. It could be pitfall for hasty novices that don't read the function documentation, but to me it still seems a lot more obvious than if we wedged in a quasi-working Result.Count method

@eshork eshork added this to To do in road to v1.0 via automation Dec 19, 2020
@eshork eshork added this to the 1.0 milestone Dec 19, 2020
@eshork
Copy link
Owner Author

eshork commented Dec 19, 2020

relates to #23 #21

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
road to v1.0
  
To do
Development

No branches or pull requests

1 participant