Skip to content
Ruby on Rails plugin to add PostgreSQL cursor support to Active Record PostgreSQL plugin for processing large result sets.
Ruby
Find file
Pull request Compare This branch is 3 commits behind afair:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.
lib
tasks
test
MIT-LICENSE
README.rdoc
Rakefile
init.rb
install.rb
uninstall.rb

README.rdoc

PostgreSQLCursor

PostgreSQL Cursor is an extension to the ActiveRecord PostgreSQLAdapter for very large result sets. It provides a cursor open/fetch/close interface to access data without loading all rows into memory, and instead loads the result rows in “chunks” (default of 10_000 rows), buffers them, and returns the rows one at a time.

For web pages, an application would not want to process a large amount of data, usually employing a Pagination scheme to present it to the users. Background processes sometimes need to generate a large

Previous solutions employ pagination to fetch each block, then re-running the query for the next “page”. This plugin avoids re-executing the query by using the PostgreSQL cursors.

Like the #find_by_sql method, #find_cursor returns each row as a hash instead of an instantiated model class. The rationale for this is performance, though an option to return instances is available. Julian's benchmarks showed returning instances was a factor of 4 slower than return the hash.

A Rails/ActiveRecord plugin for the PostgreSQL database adapter that will add cursors to a find_cursor() method to process very large result sets.

the find_cursor method uses cursors to pull in one data block (of x records) at a time, and return each record as a Hash to a procedural block. When each data block is exhausted, it will fetch the next one.

find_cursor_by_sql takes a custom SQL statement and returns each row.

Example

Account.find_cursor(:account_cursor, true, :all, :conditions=>["status = ?", 'good']) do |row|
  puts row.to_json
end

Account.find_cursor_by_sql(:account_cursor, "select ...", *args) do |row|
  puts row.to_json
end

Authors

Allen Fair, allen.fair@gmail.com, github.com/afair

Thank you to:

Copyright © 2010 Allen M. Fair, released under the MIT license

Something went wrong with that request. Please try again.