Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

Add a columns_introspection extension that attempts to skip database …

…queries by introspecting selected columns

Sequel's previous behavior has been to do a database query to
retrieve the columns for a dataset unless the dataset has already
cached the columns.  This commit adds an extension that makes
Sequel attempt to introspect the selected columns to guess at
the columns that would be returned.  This should work in most
cases, but as there is no guarantee that Sequel will guess
correctly, this is not being done by default.

To use this, after loading the extension, you can extend any
dataset with Sequel::ColumnIntrospection.  If you want to use
this for all datasets, run:

  Sequel::Dataset.introspect_all_columns

This adds some hooks to the specs so that all of Sequel's specs
can be run with this extension.  To do so, define the
SEQUEL_COLUMNS_INTROSPECTION environment variable when running
the specs.
  • Loading branch information...
commit 5e626da4e3309d366a30972168a5bfdbece2605f 1 parent ab73962
@jeremyevans authored
View
2  CHANGELOG
@@ -1,5 +1,7 @@
=== HEAD
+* Add a columns_introspection extension that attempts to skip database queries by introspecting selected columns (jeremyevans)
+
* When combining old integer migrations and new timestamp migrations, make sure old integer migrations are all applied first (jeremyevans)
* Support dynamic callbacks to customize regular association loading at query time (jeremyevans)
View
61 lib/sequel/extensions/columns_introspection.rb
@@ -0,0 +1,61 @@
+# The columns_introspection extension attempts to introspect the
+# selected columns for a dataset before issuing a query. If it
+# thinks it can guess correctly at the columns the query will use,
+# it will return the columns without issuing a database query.
+# This method is not fool-proof, it's possible that some databases
+# will use column names that Sequel does not expect.
+#
+# To enable this for a single dataset, extend the dataset with
+# Sequel::ColumnIntrospection. To enable this for all datasets, run:
+#
+# Sequel::Dataset.introspect_all_columns
+
+module Sequel
+ module ColumnsIntrospection
+ # Attempt to guess the columns that will be returned
+ # if there are columns selected, in order to skip a database
+ # query to retrieve the columns. This should work with
+ # Symbols, SQL::Identifiers, SQL::QualifiedIdentifiers, and
+ # SQL::AliasedExpressions.
+ def columns
+ return @columns if @columns
+ return columns_without_introspection unless cols = opts[:select] and !cols.empty?
+ probable_columns = cols.map{|c| probable_column_name(c)}
+ if probable_columns.all?
+ @columns = probable_columns
+ else
+ columns_without_introspection
+ end
+ end
+
+ private
+
+ # Return the probable name of the column, or nil if one
+ # cannot be determined.
+ def probable_column_name(c)
+ case c
+ when Symbol
+ _, c, a = split_symbol(c)
+ (a || c).to_sym
+ when SQL::Identifier
+ c.value.to_sym
+ when SQL::QualifiedIdentifier
+ col = c.column
+ col.is_a?(SQL::Identifier) ? col.value.to_sym : col.to_sym
+ when SQL::AliasedExpression
+ a = c.aliaz
+ a.is_a?(SQL::Identifier) ? a.value.to_sym : a.to_sym
+ end
+ end
+ end
+
+ class Dataset
+ alias columns_without_introspection columns
+
+ # Enable column introspection for every dataset.
+ def self.introspect_all_columns
+ include ColumnsIntrospection
+ remove_method(:columns) if instance_methods(false).map{|x| x.to_s}.include?('columns')
+ end
+ end
+end
View
5 spec/adapters/spec_helper.rb
@@ -10,6 +10,11 @@
rescue LoadError
end
+if ENV['SEQUEL_COLUMNS_INTROSPECTION']
+ Sequel.extension :columns_introspection
+ Sequel::Dataset.introspect_all_columns
+end
+
class Sequel::Database
def log_duration(duration, message)
log_info(message)
View
5 spec/core/spec_helper.rb
@@ -5,6 +5,11 @@
require 'sequel/core'
end
+if ENV['SEQUEL_COLUMNS_INTROSPECTION']
+ Sequel.extension :columns_introspection
+ Sequel::Dataset.introspect_all_columns
+end
+
class MockDataset < Sequel::Dataset
def insert(*args)
@db.execute insert_sql(*args)
View
91 spec/extensions/columns_introspection_spec.rb
@@ -0,0 +1,91 @@
+require File.join(File.dirname(File.expand_path(__FILE__)), 'spec_helper')
+
+describe "Sequel::Dataset.introspect_all_columns" do
+ before do
+ @db = MODEL_DB
+ @ds = @db[:a]
+ class Sequel::Dataset
+ # Handle case where introspect_all_columns has already been called
+ alias columns columns_without_introspection unless instance_methods(false).map{|x| x.to_s}.include?('columns')
+ end
+ Sequel::Dataset.introspect_all_columns
+ @db.reset
+ end
+ after do
+ class Sequel::Dataset
+ alias columns columns_without_introspection
+ end
+ end
+
+ specify "should turn on column introspection by default" do
+ @ds.select(:x).columns.should == [:x]
+ @db.sqls.length.should == 0
+ end
+end
+
+describe "columns_introspection extension" do
+ before do
+ @db = MODEL_DB
+ @ds = @db[:a]
+ @ds.extend(Sequel::ColumnsIntrospection.dup) # dup to allow multiple places in class hierarchy
+ @db.reset
+ end
+
+ specify "should not issue a database query if the columns are already loaded" do
+ @ds.instance_variable_set(:@columns, [:x])
+ @ds.columns.should == [:x]
+ @db.sqls.length.should == 0
+ end
+
+ specify "should handle plain symbols without a database query" do
+ @ds.select(:x).columns.should == [:x]
+ @db.sqls.length.should == 0
+ end
+
+ specify "should handle qualified symbols without a database query" do
+ @ds.select(:t__x).columns.should == [:x]
+ @db.sqls.length.should == 0
+ end
+
+ specify "should handle aliased symbols without a database query" do
+ @ds.select(:x___a).columns.should == [:a]
+ @db.sqls.length.should == 0
+ end
+
+ specify "should handle qualified and aliased symbols without a database query" do
+ @ds.select(:t__x___a).columns.should == [:a]
+ @db.sqls.length.should == 0
+ end
+
+ specify "should handle SQL::Identifiers " do
+ @ds.select(:x.identifier).columns.should == [:x]
+ @db.sqls.length.should == 0
+ end
+
+ specify "should handle SQL::QualifiedIdentifiers" do
+ @ds.select(:x.qualify(:t)).columns.should == [:x]
+ @ds.select(:x.identifier.qualify(:t)).columns.should == [:x]
+ @db.sqls.length.should == 0
+ end
+
+ specify "should handle SQL::AliasedExpressions" do
+ @ds.select(:x.as(:a)).columns.should == [:a]
+ @ds.select(:x.as(:a.identifier)).columns.should == [:a]
+ @db.sqls.length.should == 0
+ end
+
+ specify "should issue a database query if the wildcard is selected" do
+ @ds.columns
+ @db.sqls.length.should == 1
+ end
+
+ specify "should issue a database query if an unsupported type is used" do
+ @ds.select(1).columns
+ @db.sqls.length.should == 1
+ end
+
+ specify "should not have column introspection on by default" do
+ @db[:a].select(:x).columns
+ @db.sqls.length.should == 1
+ end
+end
View
4 spec/extensions/spec_helper.rb
@@ -8,9 +8,11 @@
require 'sequel/model'
end
-Sequel.extension(*%w'string_date_time inflector pagination query pretty_table blank migration schema_dumper looser_typecasting sql_expr thread_local_timezones to_dot')
+Sequel.extension(*%w'string_date_time inflector pagination query pretty_table blank migration schema_dumper looser_typecasting sql_expr thread_local_timezones to_dot columns_introspection')
{:hook_class_methods=>[], :schema=>[], :validation_class_methods=>[]}.each{|p, opts| Sequel::Model.plugin(p, *opts)}
+Sequel::Dataset.introspect_all_columns if ENV['SEQUEL_COLUMNS_INTROSPECTION']
+
def skip_warn(s)
warn "Skipping test of #{s}" if ENV["SKIPPED_TEST_WARN"]
end
View
5 spec/integration/spec_helper.rb
@@ -9,6 +9,11 @@
rescue LoadError
end
+if ENV['SEQUEL_COLUMNS_INTROSPECTION']
+ Sequel.extension :columns_introspection
+ Sequel::Dataset.introspect_all_columns
+end
+
Sequel::Model.use_transactions = false
$sqls = []
View
5 spec/model/spec_helper.rb
@@ -8,6 +8,11 @@
require 'sequel/model'
end
+if ENV['SEQUEL_COLUMNS_INTROSPECTION']
+ Sequel.extension :columns_introspection
+ Sequel::Dataset.introspect_all_columns
+end
+
class MockDataset < Sequel::Dataset
def insert(*args)
@db.execute insert_sql(*args)
View
1  www/pages/plugins
@@ -71,6 +71,7 @@
<ul>
<li><a href="rdoc-plugins/files/lib/sequel/extensions/blank_rb.html">blank</a>: Adds blank? instance methods to all objects.</li>
+<li><a href="rdoc-plugins/files/lib/sequel/extensions/columns_introspection_rb.html">columns_introspection</a>: Attemps to skip database queries by introspecting the selected columns if possible.</li>
<li><a href="rdoc-plugins/files/lib/sequel/extensions/inflector_rb.html">inflector</a>: Adds instance-level inflection methods to String.</li>
<li><a href="rdoc-plugins/files/lib/sequel/extensions/looser_typecasting_rb.html">looser_typecasting</a>: Uses .to_f and .to_i instead of Kernel.Float and Kernel.Integer when typecasting floats and integers.</li>
<li><a href="rdoc-plugins/files/lib/sequel/extensions/migration_rb.html">migration</a>: Adds Migration and Migrator classes for easily migrating the database schema forward or reverting to a previous version.</li>
Please sign in to comment.
Something went wrong with that request. Please try again.