Since #381 the rake task to bulk import all the records loads all the records into memory to count the number of records. The change was done because what looks like what was a bug in ActiveRecord when using default_scope and sql grouping.
I'd like to get that change reverted back, otherwise there is no point on paginating the records from the DB since the all the records have been previously loaded into memory. I've also tried to replicate the default_scope bug with ActiveRecord v3.2.8 (latest released version) and it looks like the issue with count and grouping has been fixed.
We'll be soon doing a bulk import and the current task with this behaviour of loading all the records into memory is not practical
Count records in SQL, avoids loading all the records into memory
Hi, yes, makes sense -- will it work with other storage engines such as MongoDB etc?
I don't have a setup with mongoDB and Tire, but I suspect that as long as the mongo client implements ActiveModel it should work.
I've tested Model.count with a Mongoid Model and it works so I see no reason why it wouldn't work.
Any updates on this PR @karmi ? I believe Mongo based ORM's should keep working as expected. Let me know if you want me to check anything else to get this merged in.
Hi, will test it agains a Mongoid based app, so we don't break it for them. Should get to it this week.
[FIX] Use `MyModel.count`, not `MyModel.all.count` in the Rake import…
Previously, all records had to be loaded into memory (errr),
better to use the `Model.count` method
Thanks!, verified, merged, pushed.