Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Ruby on Rails plugin to add PostgreSQL cursor support to Active Record PostgreSQL plugin for processing large result sets.
Ruby
branch: master

This branch is 3 commits behind afair:master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
lib
tasks
test
MIT-LICENSE
README.rdoc
Rakefile
init.rb
install.rb
uninstall.rb

README.rdoc

PostgreSQLCursor

PostgreSQL Cursor is an extension to the ActiveRecord PostgreSQLAdapter for very large result sets. It provides a cursor open/fetch/close interface to access data without loading all rows into memory, and instead loads the result rows in “chunks” (default of 10_000 rows), buffers them, and returns the rows one at a time.

For web pages, an application would not want to process a large amount of data, usually employing a Pagination scheme to present it to the users. Background processes sometimes need to generate a large

Previous solutions employ pagination to fetch each block, then re-running the query for the next “page”. This plugin avoids re-executing the query by using the PostgreSQL cursors.

Like the #find_by_sql method, #find_cursor returns each row as a hash instead of an instantiated model class. The rationale for this is performance, though an option to return instances is available. Julian's benchmarks showed returning instances was a factor of 4 slower than return the hash.

A Rails/ActiveRecord plugin for the PostgreSQL database adapter that will add cursors to a find_cursor() method to process very large result sets.

the find_cursor method uses cursors to pull in one data block (of x records) at a time, and return each record as a Hash to a procedural block. When each data block is exhausted, it will fetch the next one.

find_cursor_by_sql takes a custom SQL statement and returns each row.

Example

Account.find_cursor(:account_cursor, true, :all, :conditions=>["status = ?", 'good']) do |row|
  puts row.to_json
end

Account.find_cursor_by_sql(:account_cursor, "select ...", *args) do |row|
  puts row.to_json
end

Authors

Allen Fair, allen.fair@gmail.com, github.com/afair

Thank you to:

Copyright © 2010 Allen M. Fair, released under the MIT license

Something went wrong with that request. Please try again.