Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

ensure_unique_url very slow and memory consuming on large datasets #69

atomgas opened this Issue Nov 16, 2012 · 6 comments


None yet
3 participants

atomgas commented Nov 16, 2012

in https://github.com/rsl/stringex/blob/master/lib/stringex/acts_as_url.rb#L109 it does a find(:all) on the model, which in case of an empty strings or nil in url_attribute, will be all records (> 70k in my case).

I think you need a special strategy to handle empty strings or nil in url_attribute, for the other cases I would also recommend not using find(:all)


rsl commented Nov 16, 2012

already planning on using batched finds here. open to other ideas as well.

atomgas commented Nov 16, 2012

is would use SQL count on attr_name with "SOME%" this would give a good starting point,
after that I would generate the next say 100 possible results, try to find these all at once using SQL IN (possible_20, possible_21, ...) and then try to find the first not existing result with ruby, this you can repeat without to worry about preformance.


rsl commented Nov 16, 2012

got a patch idea?


rsl commented Nov 19, 2012

master uses find_each to go through smaller sets. short of a patch submission i don't have a clear idea here. feel free to submit a pull request or whatever. happy thanksgiving as well.

norbert commented Nov 23, 2012

@rsl we ran into another similar issue. Expect a patch shortly.


rsl commented Aug 27, 2013

closing this as it seems dead. reopen if needed.

@rsl rsl closed this Aug 27, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment