Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

Massive changes: Identifier input/output converter methods, H2 JDBC s…

…ubadapter, many fixes

This is a huge commit, and I'm too lazy to break it down into a
smaller one, since a lot of the parts are interdependent.

The biggest change is the addition of identifier input and output
methods.  The identifier_input_method is used when when literalizing
identifiers going into the database.  For example:

  DB[:table].insert(:column=>'value')

It will affect table and column, but not value since that is not an
identifier.  The default value is :upcase, since the SQL standard is
that unquoted identifiers are folded to uppercase, so you get:

  INSERT INTO "TABLE" ("COLUMN") VALUES ('value')

If you set the identifier_input_method to nil, which the MySQL,
PostgreSQL, and SQLite adapters do by default, you get:

  INSERT INTO "table" ("column") VALUES ('value')

The identifier_output_method is used when retrieving information
from the database.  Since SQL identifier are generally uppercase
(since that's what unquoted identifiers are folded to), and we
generally want to deal with them in lowercase, the default is
:downcase, so that you get lowercase identifiers even on a database
with uppercase identifiers.

Like most Sequel settings, this can be set globally, per database,
or per dataset:

  # Global
  Sequel.identifier_input_method = :downcase
  Sequel.identifier_output_method = :upcase
  # Per Database
  DB.identifier_input_method = :camelize
  DB.identifier_output_method = :underscore
  # Per Dataset
  ds = DB[:table]
  ds.identifier_input_method = :underscore
  ds.identifier_output_method = :reverse

I could have added a downcase_identifiers setting similar to the
upcase_identifiers setting, but that is confusing since they
operate differently.  The identifier input/output method naming
makes more sense, and gives you the flexibility to do something
crazy like :camelize and :underscore, which you may need to do
if you use CamelCase names in your database.

The upcase_identifiers setting now just sets the
identifier_input_method to :upcase (or nil if it is set to false).

The identifier_output_method is used inside of fetch_rows (actually,
a method that fetch_rows calls), and therefore it requires
changes to all adapters (since that's where the database specific
fetching happens).  As I only test 5 of the adapters and Sequel
supports 13, I need everyone that uses an adapter other than
MySQL, PostgreSQL, SQLite, JDBC, and DataObjects to test this
commit against their database and make sure I didn't break anything.
Unlike usual, not all of the changes I made to adapters I don't have
access to are small.  The most significant changes that I did not
test were to the firebird and informix adapters.

This commit adds an H2 subadapter for JDBC.  H2 is an embedded
database similar to SQLite, but it is easier for JRuby users to use
as it doesn't rely on any native code.  It passes all integration
tests with the exception of the schema tests, because it doesn't
support schema parsing yet.

This commit also adds a Database#tables method to the JDBC adapter,
and hopefully full schema parsing for JDBC will be coming soon.

This commit also changes the JDBC adapter so that Java specific
types such as Java::JavaSQL::Timestamp, Java::JavaSQL::Time,
and Java::JavaIo::BufferedReader are converted into normal ruby
types before being returned to the user.  This fixed a few bugs
in the intergation tests, and should make using the JDBC adapter
easier.

This commit fixes the literalization of Times and DateTimes when
using the JDBC MySQL subadapter.  It also fixes the use of
SQL::Blobs when using the JDBC PostgreSQL subadapter.

This commit refactors the MySQL native adapter so as to not
modify the Mysql::Result class.  This was necessary in order to
implement the identifier_output_method support.

This commit makes Database#quote_identifiers= have affect on
future usages of the schema modification methods such as
create_table, by removing the cached schema_utility_dataset.
The identifier input/output methods have the same effect.

This commit changes the default quoted_identifier method to
double any double quotes in the identifier, in addition to
surrounding the identifier in double quotes.

This commit changes the MySQL specs to only run the socket
specs if the native mysql adapter is being used, since it
uses the native mysql adapter to run those specs.

Due to implementation details,
Database.identifier_(input|output)_method returns "" if no
method should be used, and nil if it should fall back to the
adapter support.  You should not rely on this, though I don't plan
on changing it.

This commit breaks backwards compatibility in the following ways:

Using a database other than MySQL, PostgreSQL, or SQLite, if you
used to get uppercase symbols when retrieving records, you will
now probably get lowercase symbols.  To continue to receive
uppercase symbols:

  DB.identifier_output_method = nil

The Database#lowercase method in the DBI adapter was removed, as
it isn't needed since the default is to lowercase identifiers.
  • Loading branch information...
commit cf4b1558649cda1880cc382ddef360e1a41ef339 1 parent 885f92a
@jeremyevans authored
Showing with 537 additions and 170 deletions.
  1. +34 −0 lib/sequel_core.rb
  2. +1 −1  lib/sequel_core/adapters/ado.rb
  3. +2 −2 lib/sequel_core/adapters/db2.rb
  4. +2 −11 lib/sequel_core/adapters/dbi.rb
  5. +1 −1  lib/sequel_core/adapters/do.rb
  6. +6 −9 lib/sequel_core/adapters/firebird.rb
  7. +10 −1 lib/sequel_core/adapters/informix.rb
  8. +45 −14 lib/sequel_core/adapters/jdbc.rb
  9. +54 −0 lib/sequel_core/adapters/jdbc/h2.rb
  10. +10 −0 lib/sequel_core/adapters/jdbc/mysql.rb
  11. +2 −0  lib/sequel_core/adapters/jdbc/postgresql.rb
  12. +53 −77 lib/sequel_core/adapters/mysql.rb
  13. +1 −1  lib/sequel_core/adapters/odbc.rb
  14. +1 −1  lib/sequel_core/adapters/openbase.rb
  15. +2 −2 lib/sequel_core/adapters/oracle.rb
  16. +1 −1  lib/sequel_core/adapters/postgres.rb
  17. +8 −3 lib/sequel_core/adapters/shared/mysql.rb
  18. +8 −3 lib/sequel_core/adapters/shared/postgres.rb
  19. +8 −3 lib/sequel_core/adapters/shared/sqlite.rb
  20. +1 −1  lib/sequel_core/adapters/sqlite.rb
  21. +100 −21 lib/sequel_core/database.rb
  22. +19 −5 lib/sequel_core/dataset.rb
  23. +2 −2 lib/sequel_core/dataset/sql.rb
  24. +6 −0 lib/sequel_core/schema/sql.rb
  25. +1 −1  spec/adapters/mysql_spec.rb
  26. +93 −8 spec/sequel_core/database_spec.rb
  27. +60 −0 spec/sequel_core/dataset_spec.rb
  28. +4 −1 spec/sequel_core/spec_helper.rb
  29. +2 −1  spec/sequel_model/spec_helper.rb
View
34 lib/sequel_core.rb
@@ -69,6 +69,37 @@ def self.connect(*args, &block)
end
metaalias :open, :connect
+ # Set the method to call on identifiers going into the database. This affects
+ # the literalization of identifiers by calling this method on them before they are input.
+ # Sequel upcases identifiers in all SQL strings for most databases, so to turn that off:
+ #
+ # Sequel.identifier_input_method = nil
+ #
+ # to downcase instead:
+ #
+ # Sequel.identifier_input_method = :downcase
+ #
+ # Other string methods work as well.
+ def self.identifier_input_method=(value)
+ Database.identifier_input_method = value
+ end
+
+ # Set the method to call on identifiers coming out of the database. This affects
+ # the literalization of identifiers by calling this method on them when they are
+ # retrieved from the database. Sequel downcases identifiers retrieved for most
+ # databases, so to turn that off:
+ #
+ # Sequel.identifier_output_method = nil
+ #
+ # to upcase instead:
+ #
+ # Sequel.identifier_output_method = :upcase
+ #
+ # Other string methods work as well.
+ def self.identifier_output_method=(value)
+ Database.identifier_output_method = value
+ end
+
# Set whether to quote identifiers for all databases by default. By default,
# Sequel quotes identifiers in all SQL strings, so to turn that off:
#
@@ -92,6 +123,9 @@ def self.single_threaded=(value)
# lower case (MySQL, PostgreSQL, and SQLite).
#
# Sequel.upcase_identifiers = false
+ #
+ # This will set the indentifier_input_method to :upcase if value is true
+ # or nil if value is false.
def self.upcase_identifiers=(value)
Database.upcase_identifiers = value
end
View
2  lib/sequel_core/adapters/ado.rb
@@ -58,7 +58,7 @@ def fetch_rows(sql)
execute(sql) do |s|
@columns = s.Fields.extend(Enumerable).map do |column|
name = column.Name.empty? ? '(no column name)' : column.Name
- name.to_sym
+ output_identifier(name)
end
unless s.eof
View
4 lib/sequel_core/adapters/db2.rb
@@ -89,7 +89,7 @@ def literal(v)
def fetch_rows(sql)
execute(sql) do |sth|
@column_info = get_column_info(sth)
- @columns = @column_info.map {|c| c[:name]}
+ @columns = @column_info.map {|c| output_identifier(c[:name])}
while (rc = SQLFetch(@handle)) != SQL_NO_DATA_FOUND
@db.check_error(rc, "Could not fetch row")
yield hash_row(sth)
@@ -118,7 +118,7 @@ def hash_row(sth)
rc, v = SQLGetData(sth, i+1, c[:db2_type], c[:precision])
@db.check_error(rc, "Could not get data")
- @row[c[:name]] = convert_type(v)
+ row[output_identifier(c[:name])] = convert_type(v)
end
row
end
View
13 lib/sequel_core/adapters/dbi.rb
@@ -3,8 +3,6 @@
module Sequel
module DBI
class Database < Sequel::Database
- attr_writer :lowercase
-
set_adapter_scheme :dbi
DBI_ADAPTERS = {
@@ -69,11 +67,6 @@ def do(sql, opts={})
synchronize(opts[:server]){|conn| conn.do(sql)}
end
alias_method :execute_dui, :do
-
- # Converts all column names to lowercase
- def lowercase
- @lowercase ||= false
- end
private
@@ -97,10 +90,8 @@ def literal(v)
def fetch_rows(sql, &block)
execute(sql) do |s|
begin
- @columns = s.column_names.map do |c|
- @db.lowercase ? c.downcase.to_sym : c.to_sym
- end
- s.fetch {|r| yield hash_row(s, r)}
+ @columns = s.column_names.map{|c| output_identifier(c)}
+ s.fetch{|r| yield hash_row(s, r)}
ensure
s.finish rescue nil
end
View
2  lib/sequel_core/adapters/do.rb
@@ -191,7 +191,7 @@ def literal(v)
# with symbol keys.
def fetch_rows(sql)
execute(sql) do |reader|
- cols = @columns = reader.fields.map{|f| f.to_sym}
+ cols = @columns = reader.fields.map{|f| output_identifier(f)}
while(reader.next!) do
h = {}
cols.zip(reader.values).each{|k, v| h[k] = v}
View
15 lib/sequel_core/adapters/firebird.rb
@@ -216,10 +216,6 @@ def disconnect_connection(c)
def quote_identifiers_default
false
end
-
- def upcase_identifiers_default
- true
- end
end
# Dataset class for Firebird datasets
@@ -237,10 +233,12 @@ class Dataset < Sequel::Dataset
def fetch_rows(sql, &block)
execute(sql) do |s|
begin
- @columns = s.fields.map do |c|
- c.name.to_sym
+ @columns = s.fields.map{|c| output_identifier(c)}
+ s.fetchall(:symbols_hash).each do |r|
+ h = {}
+ r.each{|k,v| h[output_identifier(k)] = v}
+ yield h
end
- s.fetchall(:symbols_hash).each{ |r| yield r}
ensure
s.close
end
@@ -288,8 +286,7 @@ def literal(v)
end
def quote_identifier(name)
- name = super
- Fb::Global::reserved_keyword?(name) ? quoted_identifier(name.upcase) : name
+ Fb::Global::reserved_keyword?(name) ? quoted_identifier(name.upcase) : super
end
# The order of clauses in the SELECT SQL statement
View
11 lib/sequel_core/adapters/informix.rb
@@ -53,7 +53,16 @@ def literal(v)
def fetch_rows(sql, &block)
execute(sql) do |cursor|
begin
- cursor.open.each_hash(&block)
+ col_map = nil
+ cursor.open.each_hash do |h|
+ unless col_map
+ col_map = {}
+ @columns = h.keys.map{|k| col_map[k] = output_identifier(k)}
+ end
+ h2 = {}
+ h.each{|k,v| h2[col_map[k]||k] = v}
+ yield h2
+ end
ensure
cursor.drop
end
View
59 lib/sequel_core/adapters/jdbc.rb
@@ -61,6 +61,12 @@ module JavaSQL
require 'sequel_core/adapters/shared/mssql'
db.extend(Sequel::MSSQL::DatabaseMethods)
com.microsoft.sqlserver.jdbc.SQLServerDriver
+ end,
+ :h2=>proc do |db|
+ require 'sequel_core/adapters/jdbc/h2'
+ db.extend(Sequel::JDBC::H2::DatabaseMethods)
+ JDBC.load_gem('h2')
+ org.h2.Driver
end
}
@@ -179,6 +185,13 @@ def execute_insert(sql, opts={})
execute(sql, {:type=>:insert}.merge(opts))
end
+ # All tables in this database
+ def tables(server=nil)
+ ts = []
+ synchronize(server){|c| dataset.send(:process_result_set, c.getMetaData.getTables(nil, nil, nil, ['TABLE'].to_java(:string))){|h| ts << dataset.send(:output_identifier, h[:table_name])}}
+ ts
+ end
+
# Default transaction method that should work on most JDBC
# databases. Does not use the JDBC transaction methods, uses
# SQL BEGIN/ROLLBACK/COMMIT statements instead.
@@ -373,20 +386,7 @@ def execute_insert(sql, opts={}, &block)
# Correctly return rows from the database and return them as hashes.
def fetch_rows(sql, &block)
- execute(sql) do |result|
- # get column names
- meta = result.getMetaData
- column_count = meta.getColumnCount
- @columns = []
- column_count.times {|i| @columns << meta.getColumnName(i+1).to_sym}
-
- # get rows
- while result.next
- row = {}
- @columns.each_with_index {|v, i| row[v] = result.getObject(i+1)}
- yield row
- end
- end
+ execute(sql){|result| process_result_set(result, &block)}
self
end
@@ -416,10 +416,41 @@ def prepare(type, name=nil, values=nil)
private
+ # Convert the type. Used for converting Java types to ruby types.
+ def convert_type(v)
+ case v
+ when Java::JavaSQL::Timestamp, Java::JavaSQL::Time
+ v.to_string.to_sequel_time
+ when Java::JavaIo::BufferedReader
+ lines = []
+ while(line = v.read_line) do lines << line end
+ lines.join("\n")
+ else
+ v
+ end
+ end
+
# Extend the dataset with the JDBC stored procedure methods.
def prepare_extend_sproc(ds)
ds.extend(StoredProcedureMethods)
end
+
+ # Split out from fetch rows to allow processing of JDBC result sets
+ # that don't come from issuing an SQL string.
+ def process_result_set(result)
+ # get column names
+ meta = result.getMetaData
+ column_count = meta.getColumnCount
+ @columns = []
+ column_count.times {|i| @columns << output_identifier(meta.getColumnName(i+1))}
+
+ # get rows
+ while result.next
+ row = {}
+ @columns.each_with_index {|v, i| row[v] = convert_type(result.getObject(i+1))}
+ yield row
+ end
+ end
end
end
end
View
54 lib/sequel_core/adapters/jdbc/h2.rb
@@ -0,0 +1,54 @@
+module Sequel
+ module JDBC
+ # Database and Dataset support for H2 databases accessed via JDBC.
+ module H2
+ # Instance methods for H2 Database objects accessed via JDBC.
+ module DatabaseMethods
+ # Return Sequel::JDBC::H2::Dataset object with the given opts.
+ def dataset(opts=nil)
+ Sequel::JDBC::H2::Dataset.new(self, opts)
+ end
+
+ # H2 uses an IDENTITY type
+ def serial_primary_key_options
+ {:primary_key => true, :type => :identity}
+ end
+
+ private
+
+ # Use IDENTITY() to get the last inserted id.
+ def last_insert_id(conn, opts={})
+ stmt = conn.createStatement
+ begin
+ rs = stmt.executeQuery('SELECT IDENTITY();')
+ rs.next
+ rs.getInt(1)
+ ensure
+ stmt.close
+ end
+ end
+
+ # Default to a single connection for a memory database.
+ def connection_pool_default_options
+ o = super
+ uri == 'jdbc:h2:mem:' ? o.merge(:max_connections=>1) : o
+ end
+ end
+
+ # Dataset class for H2 datasets accessed via JDBC.
+ class Dataset < JDBC::Dataset
+ # Use H2 syntax for Date, DateTime, and Time types.
+ def literal(v)
+ case v
+ when Date
+ v.strftime("DATE '%Y-%m-%d'")
+ when DateTime, Time
+ v.strftime("TIMESTAMP '%Y-%m-%d %H:%M:%S'")
+ else
+ super
+ end
+ end
+ end
+ end
+ end
+end
View
10 lib/sequel_core/adapters/jdbc/mysql.rb
@@ -46,6 +46,16 @@ def insert(*values)
execute_insert(insert_sql(*values))
end
+ # Handle time types correctly
+ def literal(v)
+ case v
+ when Time, DateTime
+ v.strftime("'%Y-%m-%d %H:%M:%S'")
+ else
+ super
+ end
+ end
+
# Use execute_insert to execute the replace_sql.
def replace(*args)
execute_insert(replace_sql(*args))
View
2  lib/sequel_core/adapters/jdbc/postgresql.rb
@@ -97,6 +97,8 @@ def literal(v)
case v
when LiteralString
v
+ when SQL::Blob
+ super
when String
db.synchronize{|c| "'#{c.escape_string(v)}'"}
when Java::JavaSql::Timestamp
View
130 lib/sequel_core/adapters/mysql.rb
@@ -2,84 +2,40 @@
require 'sequel_core/adapters/shared/mysql'
require 'sequel_core/dataset/stored_procedures'
-# Add methods to get columns, yield hashes with symbol keys, and do
-# type conversion.
-class Mysql::Result
- # Mapping of type numbers to conversion methods.
- MYSQL_TYPES = {
- 0 => :to_d, # MYSQL_TYPE_DECIMAL
- 1 => :to_i, # MYSQL_TYPE_TINY
- 2 => :to_i, # MYSQL_TYPE_SHORT
- 3 => :to_i, # MYSQL_TYPE_LONG
- 4 => :to_f, # MYSQL_TYPE_FLOAT
- 5 => :to_f, # MYSQL_TYPE_DOUBLE
- # 6 => ??, # MYSQL_TYPE_NULL
- 7 => :to_sequel_time, # MYSQL_TYPE_TIMESTAMP
- 8 => :to_i, # MYSQL_TYPE_LONGLONG
- 9 => :to_i, # MYSQL_TYPE_INT24
- 10 => :to_date, # MYSQL_TYPE_DATE
- 11 => :to_time, # MYSQL_TYPE_TIME
- 12 => :to_sequel_time, # MYSQL_TYPE_DATETIME
- 13 => :to_i, # MYSQL_TYPE_YEAR
- 14 => :to_date, # MYSQL_TYPE_NEWDATE
- # 15 => :to_s # MYSQL_TYPE_VARCHAR
- # 16 => :to_s, # MYSQL_TYPE_BIT
- 246 => :to_d, # MYSQL_TYPE_NEWDECIMAL
- 247 => :to_i, # MYSQL_TYPE_ENUM
- 248 => :to_i, # MYSQL_TYPE_SET
- 249 => :to_blob, # MYSQL_TYPE_TINY_BLOB
- 250 => :to_blob, # MYSQL_TYPE_MEDIUM_BLOB
- 251 => :to_blob, # MYSQL_TYPE_LONG_BLOB
- 252 => :to_blob, # MYSQL_TYPE_BLOB
- # 253 => :to_s, # MYSQL_TYPE_VAR_STRING
- # 254 => :to_s, # MYSQL_TYPE_STRING
- # 255 => :to_s # MYSQL_TYPE_GEOMETRY
- }
-
- # Return an array of column name symbols for this result set.
- def columns(with_table = nil)
- unless @columns
- @column_types = []
- @columns = fetch_fields.map do |f|
- @column_types << f.type
- (with_table ? "#{f.table}.#{f.name}" : f.name).to_sym
- end
- end
- @columns
- end
-
- # yield a hash with symbol keys and type converted values.
- def sequel_each_hash(with_table = nil)
- c = columns
- while row = fetch_row
- h = {}
- c.each_with_index {|f, i| h[f] = convert_type(row[i], @column_types[i])}
- yield h
- end
- end
-
- private
-
- # Convert the type of v using the method in MYSQL_TYPES[type].
- def convert_type(v, type)
- if v
- if type == 1 && Sequel.convert_tinyint_to_bool
- # We special case tinyint here to avoid adding
- # a method to an ancestor of Fixnum
- v.to_i == 0 ? false : true
- else
- (t = MYSQL_TYPES[type]) ? v.send(t) : v
- end
- else
- nil
- end
- end
-
-end
-
module Sequel
# Module for holding all MySQL-related classes and modules for Sequel.
module MySQL
+ # Mapping of type numbers to conversion methods.
+ MYSQL_TYPES = {
+ 0 => :to_d, # MYSQL_TYPE_DECIMAL
+ 1 => :to_i, # MYSQL_TYPE_TINY
+ 2 => :to_i, # MYSQL_TYPE_SHORT
+ 3 => :to_i, # MYSQL_TYPE_LONG
+ 4 => :to_f, # MYSQL_TYPE_FLOAT
+ 5 => :to_f, # MYSQL_TYPE_DOUBLE
+ # 6 => ??, # MYSQL_TYPE_NULL
+ 7 => :to_sequel_time, # MYSQL_TYPE_TIMESTAMP
+ 8 => :to_i, # MYSQL_TYPE_LONGLONG
+ 9 => :to_i, # MYSQL_TYPE_INT24
+ 10 => :to_date, # MYSQL_TYPE_DATE
+ 11 => :to_time, # MYSQL_TYPE_TIME
+ 12 => :to_sequel_time, # MYSQL_TYPE_DATETIME
+ 13 => :to_i, # MYSQL_TYPE_YEAR
+ 14 => :to_date, # MYSQL_TYPE_NEWDATE
+ # 15 => :to_s # MYSQL_TYPE_VARCHAR
+ # 16 => :to_s, # MYSQL_TYPE_BIT
+ 246 => :to_d, # MYSQL_TYPE_NEWDECIMAL
+ 247 => :to_i, # MYSQL_TYPE_ENUM
+ 248 => :to_i, # MYSQL_TYPE_SET
+ 249 => :to_blob, # MYSQL_TYPE_TINY_BLOB
+ 250 => :to_blob, # MYSQL_TYPE_MEDIUM_BLOB
+ 251 => :to_blob, # MYSQL_TYPE_LONG_BLOB
+ 252 => :to_blob, # MYSQL_TYPE_BLOB
+ # 253 => :to_s, # MYSQL_TYPE_VAR_STRING
+ # 254 => :to_s, # MYSQL_TYPE_STRING
+ # 255 => :to_s # MYSQL_TYPE_GEOMETRY
+ }
+
# Database class for MySQL databases used with Sequel.
class Database < Sequel::Database
include Sequel::MySQL::DatabaseMethods
@@ -316,8 +272,13 @@ def delete(opts = nil)
# Yield all rows matching this dataset
def fetch_rows(sql)
execute(sql) do |r|
- @columns = r.columns
- r.sequel_each_hash {|row| yield row}
+ column_types = []
+ @columns = r.fetch_fields.map{|f| column_types << f.type; output_identifier(f.name)}
+ while row = r.fetch_row
+ h = {}
+ @columns.each_with_index {|f, i| h[f] = convert_type(row[i], column_types[i])}
+ yield h
+ end
end
self
end
@@ -363,6 +324,21 @@ def update(*args)
private
+ # Convert the type of v using the method in MYSQL_TYPES[type].
+ def convert_type(v, type)
+ if v
+ if type == 1 && Sequel.convert_tinyint_to_bool
+ # We special case tinyint here to avoid adding
+ # a method to an ancestor of Fixnum
+ v.to_i == 0 ? false : true
+ else
+ (t = MYSQL_TYPES[type]) ? v.send(t) : v
+ end
+ else
+ nil
+ end
+ end
+
# Set the :type option to :select if it hasn't been set.
def execute(sql, opts={}, &block)
super(sql, {:type=>:select}.merge(opts), &block)
View
2  lib/sequel_core/adapters/odbc.rb
@@ -129,7 +129,7 @@ def fetch_rows(sql, &block)
if (n = c.name).empty?
n = UNTITLED_COLUMN % (untitled_count += 1)
end
- n.to_sym
+ output_identifier(n)
end
rows = s.fetch_all
rows.each {|row| yield hash_row(row)} if rows
View
2  lib/sequel_core/adapters/openbase.rb
@@ -51,7 +51,7 @@ def literal(v)
def fetch_rows(sql)
execute(sql) do |result|
begin
- @columns = result.column_infos.map {|c| c.name.to_sym}
+ @columns = result.column_infos.map{|c| output_identifier(c.name)}
result.each do |r|
row = {}
r.each_with_index {|v, i| row[@columns[i]] = v}
View
4 lib/sequel_core/adapters/oracle.rb
@@ -90,10 +90,10 @@ def literal(v)
def fetch_rows(sql, &block)
execute(sql) do |cursor|
begin
- @columns = cursor.get_col_names.map {|c| c.downcase.to_sym}
+ @columns = cursor.get_col_names.map{|c| output_identifier(c)}
while r = cursor.fetch
row = {}
- r.each_with_index {|v, i| row[columns[i]] = v unless columns[i] == :raw_rnum_}
+ r.each_with_index {|v, i| row[@columns[i]] = v unless @columns[i] == :raw_rnum_}
yield row
end
ensure
View
2  lib/sequel_core/adapters/postgres.rb
@@ -312,7 +312,7 @@ def fetch_rows(sql)
cols = []
execute(sql) do |res|
res.nfields.times do |fieldnum|
- cols << [fieldnum, PG_TYPES[res.ftype(fieldnum)], res.fname(fieldnum).to_sym]
+ cols << [fieldnum, PG_TYPES[res.ftype(fieldnum)], output_identifier(res.fname(fieldnum))]
end
@columns = cols.map{|c| c.at(2)}
res.ntuples.times do |recnum|
View
11 lib/sequel_core/adapters/shared/mysql.rb
@@ -100,9 +100,14 @@ def use(db_name)
private
- # MySQL folds unquoted identifiers to lowercase, so it shouldn't need to upcase identifiers by default.
- def upcase_identifiers_default
- false
+ # MySQL folds unquoted identifiers to lowercase, so it shouldn't need to upcase identifiers on input.
+ def identifier_input_method_default
+ nil
+ end
+
+ # MySQL folds unquoted identifiers to lowercase, so it shouldn't need to upcase identifiers on output.
+ def identifier_output_method_default
+ nil
end
# Use the MySQL specific DESCRIBE syntax to get a table description.
View
11 lib/sequel_core/adapters/shared/postgres.rb
@@ -437,9 +437,14 @@ def transaction(server=nil)
private
- # PostgreSQL folds unquoted identifiers to lowercase, so it shouldn't need to upcase identifiers by default.
- def upcase_identifiers_default
- false
+ # PostgreSQL folds unquoted identifiers to lowercase, so it shouldn't need to upcase identifiers on input.
+ def identifier_input_method_default
+ nil
+ end
+
+ # PostgreSQL folds unquoted identifiers to lowercase, so it shouldn't need to upcase identifiers on output.
+ def identifier_output_method_default
+ nil
end
# The result of the insert for the given table and values. If values
View
11 lib/sequel_core/adapters/shared/sqlite.rb
@@ -85,9 +85,14 @@ def temp_store=(value)
private
- # SQLite folds unquoted identifiers to lowercase, so it shouldn't need to upcase identifiers by default.
- def upcase_identifiers_default
- false
+ # SQLite folds unquoted identifiers to lowercase, so it shouldn't need to upcase identifiers on input.
+ def identifier_input_method_default
+ nil
+ end
+
+ # SQLite folds unquoted identifiers to lowercase, so it shouldn't need to upcase identifiers on output.
+ def identifier_output_method_default
+ nil
end
# SQLite supports schema parsing using the table_info PRAGMA, so
View
2  lib/sequel_core/adapters/sqlite.rb
@@ -195,7 +195,7 @@ def explain
# Yield a hash for each row in the dataset.
def fetch_rows(sql)
execute(sql) do |result|
- @columns = result.columns.map {|c| c.to_sym}
+ @columns = result.columns.map{|c| output_identifier(c)}
column_count = @columns.size
result.each do |values|
row = {}
View
121 lib/sequel_core/database.rb
@@ -22,6 +22,12 @@ class Database
# Hash of adapters that have been used
@@adapters = Hash.new
+
+ # The identifier input method to use by default
+ @@identifier_input_method = nil
+
+ # The identifier output method to use by default
+ @@identifier_output_method = nil
# Whether to use the single threaded connection pool by default
@@single_threaded = false
@@ -29,15 +35,12 @@ class Database
# Whether to quote identifiers (columns and tables) by default
@@quote_identifiers = nil
- # Whether to upcase identifiers (columns and tables) by default
- @@upcase_identifiers = nil
-
# The default schema to use
attr_accessor :default_schema
# Array of SQL loggers to use for this database
attr_accessor :loggers
-
+
# The options for this database
attr_reader :opts
@@ -46,12 +49,6 @@ class Database
# The prepared statement objects for this database, keyed by name
attr_reader :prepared_statements
-
- # Whether to quote identifiers (columns and tables) for this database
- attr_writer :quote_identifiers
-
- # Whether to upcase identifiers (columns and tables) for this database
- attr_writer :upcase_identifiers
# Constructs a new instance of a database connection with the specified
# options hash.
@@ -61,9 +58,12 @@ class Database
# Takes the following options:
# * :default_schema : The default schema to use, should generally be nil
# * :disconnection_proc: A proc used to disconnect the connection.
+ # * :identifier_input_method: A string method symbol to call on identifiers going into the database
+ # * :identifier_output_method: A string method symbol to call on identifiers coming from the database
# * :loggers : An array of loggers to use.
# * :quote_identifiers : Whether to quote identifiers
# * :single_threaded : Whether to use a single-threaded connection pool
+ # * :upcase_identifiers : Whether to upcase identifiers going into the database
#
# All options given are also passed to the ConnectionPool. If a block
# is given, it is used as the connection_proc for the ConnectionPool.
@@ -75,6 +75,9 @@ def initialize(opts = {}, &block)
@default_schema = opts[:default_schema]
@prepared_statements = {}
@transactions = []
+ if opts.include?(:upcase_identifiers)
+ @identifier_input_method = opts[:upcase_identifiers] ? :upcase : ""
+ end
@pool = (@single_threaded ? SingleThreadedPool : ConnectionPool).new(connection_pool_default_options.merge(opts), &block)
@pool.connection_proc = proc{|server| connect(server)} unless block
@pool.disconnection_proc = proc{|conn| disconnect_connection(conn)} unless opts[:disconnection_proc]
@@ -152,6 +155,26 @@ def self.connect(conn_string, opts = {}, &block)
c.new(opts)
end
end
+
+ # The method to call on identifiers going into the database
+ def self.identifier_input_method
+ @@identifier_input_method
+ end
+
+ # Set the method to call on identifiers going into the database
+ def self.identifier_input_method=(v)
+ @@identifier_input_method = v || ""
+ end
+
+ # The method to call on identifiers coming from the database
+ def self.identifier_output_method
+ @@identifier_output_method
+ end
+
+ # Set the method to call on identifiers coming from the database
+ def self.identifier_output_method=(v)
+ @@identifier_output_method = v || ""
+ end
# Sets the default quote_identifiers mode for new databases.
# See Sequel.quote_identifiers=.
@@ -168,7 +191,7 @@ def self.single_threaded=(value)
# Sets the default quote_identifiers mode for new databases.
# See Sequel.quote_identifiers=.
def self.upcase_identifiers=(value)
- @@upcase_identifiers = value
+ self.identifier_input_method = value ? :upcase : nil
end
### Private Class Methods ###
@@ -309,6 +332,44 @@ def get(expr)
dataset.get(expr)
end
+ # The method to call on identifiers going into the database
+ def identifier_input_method
+ case @identifier_input_method
+ when nil
+ @identifier_input_method = @opts.include?(:identifier_input_method) ? @opts[:identifier_input_method] : (@@identifier_input_method.nil? ? identifier_input_method_default : @@identifier_input_method)
+ @identifier_input_method == "" ? nil : @identifier_input_method
+ when ""
+ nil
+ else
+ @identifier_input_method
+ end
+ end
+
+ # Set the method to call on identifiers going into the database
+ def identifier_input_method=(v)
+ reset_schema_utility_dataset
+ @identifier_input_method = v || ""
+ end
+
+ # The method to call on identifiers coming from the database
+ def identifier_output_method
+ case @identifier_output_method
+ when nil
+ @identifier_output_method = @opts.include?(:identifier_output_method) ? @opts[:identifier_output_method] : (@@identifier_output_method.nil? ? identifier_output_method_default : @@identifier_output_method)
+ @identifier_output_method == "" ? nil : @identifier_output_method
+ when ""
+ nil
+ else
+ @identifier_output_method
+ end
+ end
+
+ # Set the method to call on identifiers coming from the database
+ def identifier_output_method=(v)
+ reset_schema_utility_dataset
+ @identifier_output_method = v || ""
+ end
+
# Returns a string representation of the database object including the
# class name and the connection URI (or the opts if the URI
# cannot be constructed).
@@ -344,6 +405,12 @@ def query(&block)
dataset.query(&block)
end
+ # Whether to quote identifiers (columns and tables) for this database
+ def quote_identifiers=(v)
+ reset_schema_utility_dataset
+ @quote_identifiers = v
+ end
+
# Returns true if the database quotes identifiers.
def quote_identifiers?
return @quote_identifiers unless @quote_identifiers.nil?
@@ -494,11 +561,15 @@ def typecast_value(column_type, value)
value
end
end
+
+ # Set whether to upcase identifiers going into the database.
+ def upcase_identifiers=(v)
+ self.identifier_input_method = v ? :upcase : nil
+ end
# Returns true if the database upcases identifiers.
def upcase_identifiers?
- return @upcase_identifiers unless @upcase_identifiers.nil?
- @upcase_identifiers = @opts.include?(:upcase_identifiers) ? @opts[:upcase_identifiers] : (@@upcase_identifiers.nil? ? upcase_identifiers_default : @@upcase_identifiers)
+ identifier_input_method == :upcase
end
# Returns the URI identifying the database.
@@ -543,6 +614,22 @@ def connection_pool_default_options
{}
end
+ # The method to apply to identifiers going into the database by default.
+ # Should be overridden in subclasses for databases that fold unquoted
+ # identifiers to lower case instead of uppercase, such as
+ # MySQL, PostgreSQL, and SQLite.
+ def identifier_input_method_default
+ :upcase
+ end
+
+ # The method to apply to identifiers coming the database by default.
+ # Should be overridden in subclasses for databases that fold unquoted
+ # identifiers to lower case instead of uppercase, such as
+ # MySQL, PostgreSQL, and SQLite.
+ def identifier_output_method_default
+ :downcase
+ end
+
def quote_identifiers_default
true
end
@@ -593,14 +680,6 @@ def server_opts(server)
def transaction_error(e, *classes)
raise_error(e, :classes=>classes) unless Error::Rollback === e
end
-
- # Sets whether to upcase identifiers by default. Should be
- # overridden in subclasses for databases that fold unquoted
- # identifiers to lower case instead of uppercase, such as
- # MySQL, PostgreSQL, and SQLite.
- def upcase_identifiers_default
- true
- end
end
end
View
24 lib/sequel_core/dataset.rb
@@ -71,6 +71,12 @@ class Dataset
# The database that corresponds to this dataset
attr_accessor :db
+
+ # Set the method to call on identifiers going into the database for this dataset
+ attr_accessor :identifier_input_method
+
+ # Set the method to call on identifiers coming the database for this dataset
+ attr_accessor :identifier_output_method
# The hash of options for this dataset, keys are symbols.
attr_accessor :opts
@@ -83,9 +89,6 @@ class Dataset
# fetch_rows to return.
attr_accessor :row_proc
- # Whether to upcase identifiers for this dataset
- attr_writer :upcase_identifiers
-
# Constructs a new instance of a dataset with an associated database and
# options. Datasets are usually constructed by invoking Database methods:
#
@@ -100,7 +103,8 @@ class Dataset
def initialize(db, opts = nil)
@db = db
@quote_identifiers = db.quote_identifiers? if db.respond_to?(:quote_identifiers?)
- @upcase_identifiers = db.upcase_identifiers? if db.respond_to?(:upcase_identifiers?)
+ @identifier_input_method = db.identifier_input_method if db.respond_to?(:identifier_input_method)
+ @identifier_output_method = db.identifier_output_method if db.respond_to?(:identifier_output_method)
@opts = opts || {}
@row_proc = nil
@transform = nil
@@ -419,9 +423,13 @@ def transform_save(r)
end
end
+ def upcase_identifiers=(v)
+ @identifier_input_method = v ? :upcase : nil
+ end
+
# Whether this dataset upcases identifiers.
def upcase_identifiers?
- @upcase_identifiers
+ @identifier_input_method == :upcase
end
# Updates values for the dataset. The returned value is generally the
@@ -482,5 +490,11 @@ def mutation_method(meth, *args, &block)
@opts.merge!(copy.opts)
self
end
+
+ # Modify the identifier returned from the database based on the
+ # identifier_output_method.
+ def output_identifier(v)
+ (i = identifier_output_method) ? v.to_s.send(i).to_sym : v.to_sym
+ end
end
end
View
4 lib/sequel_core/dataset/sql.rb
@@ -573,7 +573,7 @@ def qualified_identifier_sql(qcr)
# quote the name with quoted_identifier.
def quote_identifier(name)
name = name.to_s
- name = name.upcase if upcase_identifiers?
+ name = name.send(@identifier_input_method) if @identifier_input_method
name = quoted_identifier(name) if quote_identifiers?
name
end
@@ -590,7 +590,7 @@ def quote_schema_table(table)
# should be overridden by subclasses to provide quoting not matching the
# SQL standard, such as backtick (used by MySQL and SQLite).
def quoted_identifier(name)
- "\"#{name}\""
+ "\"#{name.to_s.gsub('"', '""')}\""
end
# Returns a copy of the dataset with the order reversed. If no order is
View
6 lib/sequel_core/schema/sql.rb
@@ -273,6 +273,12 @@ def schema_utility_dataset
def remove_cached_schema(table)
@schemas.delete(quote_schema_table(table)) if @schemas
end
+
+ # Remove the cached schema_utility_dataset, because the identifier
+ # quoting has changed.
+ def reset_schema_utility_dataset
+ @schema_utility_dataset = nil
+ end
# Match the database's column type to a ruby type via a
# regular expression. The following ruby types are supported:
View
2  spec/adapters/mysql_spec.rb
@@ -564,7 +564,7 @@ def MYSQL_DB.ret_commit
end
# Socket tests should only be run if the MySQL server is on localhost
-if %w'localhost 127.0.0.1 ::1'.include? MYSQL_URI.host
+if %w'localhost 127.0.0.1 ::1'.include?(MYSQL_URI.host) and MYSQL_DB.class.adapter_scheme == :mysql
context "A MySQL database" do
specify "should accept a socket option" do
db = Sequel.mysql(MYSQL_DB.opts[:database], :host => 'localhost', :user => MYSQL_DB.opts[:user], :password => MYSQL_DB.opts[:password], :socket => MYSQL_SOCKET_FILE)
View
101 spec/sequel_core/database_spec.rb
@@ -6,7 +6,8 @@
end
teardown do
Sequel.quote_identifiers = false
- Sequel.upcase_identifiers = false
+ Sequel.identifier_input_method = nil
+ Sequel.identifier_output_method = nil
end
specify "should receive options" do
@@ -74,6 +75,48 @@
db.upcase_identifiers = false
db.upcase_identifiers?.should == false
end
+
+ specify "should respect the :identifier_input_method option" do
+ Sequel.identifier_input_method = nil
+ db = Sequel::Database.new(:identifier_input_method=>nil)
+ db.identifier_input_method.should == nil
+ db.identifier_input_method = :downcase
+ db.identifier_input_method.should == :downcase
+ db = Sequel::Database.new(:identifier_input_method=>:upcase)
+ db.identifier_input_method.should == :upcase
+ db.identifier_input_method = nil
+ db.identifier_input_method.should == nil
+ Sequel.identifier_input_method = :downcase
+ db = Sequel::Database.new(:identifier_input_method=>nil)
+ db.identifier_input_method.should == nil
+ db.identifier_input_method = :upcase
+ db.identifier_input_method.should == :upcase
+ db = Sequel::Database.new(:identifier_input_method=>:upcase)
+ db.identifier_input_method.should == :upcase
+ db.identifier_input_method = nil
+ db.identifier_input_method.should == nil
+ end
+
+ specify "should respect the :identifier_output_method option" do
+ Sequel.identifier_output_method = nil
+ db = Sequel::Database.new(:identifier_output_method=>nil)
+ db.identifier_output_method.should == nil
+ db.identifier_output_method = :downcase
+ db.identifier_output_method.should == :downcase
+ db = Sequel::Database.new(:identifier_output_method=>:upcase)
+ db.identifier_output_method.should == :upcase
+ db.identifier_output_method = nil
+ db.identifier_output_method.should == nil
+ Sequel.identifier_output_method = :downcase
+ db = Sequel::Database.new(:identifier_output_method=>nil)
+ db.identifier_output_method.should == nil
+ db.identifier_output_method = :upcase
+ db.identifier_output_method.should == :upcase
+ db = Sequel::Database.new(:identifier_output_method=>:upcase)
+ db.identifier_output_method.should == :upcase
+ db.identifier_output_method = nil
+ db.identifier_output_method.should == nil
+ end
specify "should use the default Sequel.quote_identifiers value" do
Sequel.quote_identifiers = true
@@ -96,14 +139,56 @@
Sequel::Database.upcase_identifiers = false
Sequel::Database.new({}).upcase_identifiers?.should == false
end
+
+ specify "should use the default Sequel.identifier_input_method value" do
+ Sequel.identifier_input_method = :downcase
+ Sequel::Database.new({}).identifier_input_method.should == :downcase
+ Sequel.identifier_input_method = :upcase
+ Sequel::Database.new({}).identifier_input_method.should == :upcase
+ Sequel::Database.identifier_input_method = :downcase
+ Sequel::Database.new({}).identifier_input_method.should == :downcase
+ Sequel::Database.identifier_input_method = :upcase
+ Sequel::Database.new({}).identifier_input_method.should == :upcase
+ end
+
+ specify "should use the default Sequel.identifier_output_method value" do
+ Sequel.identifier_output_method = :downcase
+ Sequel::Database.new({}).identifier_output_method.should == :downcase
+ Sequel.identifier_output_method = :upcase
+ Sequel::Database.new({}).identifier_output_method.should == :upcase
+ Sequel::Database.identifier_output_method = :downcase
+ Sequel::Database.new({}).identifier_output_method.should == :downcase
+ Sequel::Database.identifier_output_method = :upcase
+ Sequel::Database.new({}).identifier_output_method.should == :upcase
+ end
- specify "should respect the upcase_indentifiers_default method if Sequel.upcase_identifiers = nil" do
- Sequel.upcase_identifiers = nil
- Sequel::Database.new({}).upcase_identifiers?.should == true
- x = Class.new(Sequel::Database){def upcase_identifiers_default; false end}
- x.new({}).upcase_identifiers?.should == false
- y = Class.new(Sequel::Database){def upcase_identifiers_default; true end}
- y.new({}).upcase_identifiers?.should == true
+ specify "should respect the quote_indentifiers_default method if Sequel.quote_identifiers = nil" do
+ Sequel.quote_identifiers = nil
+ Sequel::Database.new({}).quote_identifiers?.should == true
+ x = Class.new(Sequel::Database){def quote_identifiers_default; false end}
+ x.new({}).quote_identifiers?.should == false
+ y = Class.new(Sequel::Database){def quote_identifiers_default; true end}
+ y.new({}).quote_identifiers?.should == true
+ end
+
+ specify "should respect the identifier_input_method_default method" do
+ class Sequel::Database
+ @@identifier_input_method = nil
+ end
+ x = Class.new(Sequel::Database){def identifier_input_method_default; :downcase end}
+ x.new({}).identifier_input_method.should == :downcase
+ y = Class.new(Sequel::Database){def identifier_input_method_default; :camelize end}
+ y.new({}).identifier_input_method.should == :camelize
+ end
+
+ specify "should respect the identifier_output_method_default method if Sequel.upcase_identifiers = nil" do
+ class Sequel::Database
+ @@identifier_output_method = nil
+ end
+ x = Class.new(Sequel::Database){def identifier_output_method_default; :upcase end}
+ x.new({}).identifier_output_method.should == :upcase
+ y = Class.new(Sequel::Database){def identifier_output_method_default; :underscore end}
+ y.new({}).identifier_output_method.should == :underscore
end
specify "should just use a :uri option for jdbc with the full connection string" do
View
60 spec/sequel_core/dataset_spec.rb
@@ -39,6 +39,66 @@
specify "should include Enumerable" do
Sequel::Dataset.included_modules.should include(Enumerable)
end
+
+ specify "should get quote_identifiers default from database" do
+ db = Sequel::Database.new(:quote_identifiers=>true)
+ db[:a].quote_identifiers?.should == true
+ db = Sequel::Database.new(:quote_identifiers=>false)
+ db[:a].quote_identifiers?.should == false
+ end
+
+ specify "should get identifier_input_method default from database" do
+ db = Sequel::Database.new(:identifier_input_method=>:upcase)
+ db[:a].identifier_input_method.should == :upcase
+ db = Sequel::Database.new(:identifier_input_method=>:downcase)
+ db[:a].identifier_input_method.should == :downcase
+ end
+
+ specify "should get identifier_output_method default from database" do
+ db = Sequel::Database.new(:identifier_output_method=>:upcase)
+ db[:a].identifier_output_method.should == :upcase
+ db = Sequel::Database.new(:identifier_output_method=>:downcase)
+ db[:a].identifier_output_method.should == :downcase
+ end
+end
+
+context "Dataset" do
+ setup do
+ @dataset = Sequel::Dataset.new("db")
+ end
+
+ specify "should have quote_identifiers= method which changes literalization of identifiers" do
+ @dataset.quote_identifiers = true
+ @dataset.literal(:a).should == '"a"'
+ @dataset.quote_identifiers = false
+ @dataset.literal(:a).should == 'a'
+ end
+
+ specify "should have upcase_identifiers= method which changes literalization of identifiers" do
+ @dataset.upcase_identifiers = true
+ @dataset.literal(:a).should == 'A'
+ @dataset.upcase_identifiers = false
+ @dataset.literal(:a).should == 'a'
+ end
+
+ specify "should have identifier_input_method= method which changes literalization of identifiers" do
+ @dataset.identifier_input_method = :upcase
+ @dataset.literal(:a).should == 'A'
+ @dataset.identifier_input_method = :downcase
+ @dataset.literal(:A).should == 'a'
+ @dataset.identifier_input_method = :reverse
+ @dataset.literal(:at_b).should == 'b_ta'
+ end
+
+ specify "should have identifier_output_method= method which changes identifiers returned from the database" do
+ @dataset.send(:output_identifier, "at_b_C").should == :at_b_C
+ @dataset.identifier_output_method = :upcase
+ @dataset.send(:output_identifier, "at_b_C").should == :AT_B_C
+ @dataset.identifier_output_method = :downcase
+ @dataset.send(:output_identifier, "at_b_C").should == :at_b_c
+ @dataset.identifier_output_method = :reverse
+ @dataset.send(:output_identifier, "at_b_C").should == :C_b_ta
+ end
end
context "Dataset#clone" do
View
5 spec/sequel_core/spec_helper.rb
@@ -25,7 +25,8 @@ def quoted_identifier(c)
class MockDatabase < Sequel::Database
@@quote_identifiers = false
- @@upcase_identifiers = false
+ self.identifier_input_method = nil
+ self.identifier_output_method = nil
attr_reader :sqls
def execute(sql, opts={})
@@ -44,6 +45,8 @@ def dataset; MockDataset.new(self); end
class SchemaDummyDatabase < Sequel::Database
attr_reader :sqls
+ self.identifier_input_method = nil
+ self.identifier_output_method = nil
def execute(sql, opts={})
@sqls ||= []
View
3  spec/sequel_model/spec_helper.rb
@@ -34,7 +34,8 @@ def quoted_identifier(c)
class MockDatabase < Sequel::Database
@@quote_identifiers = false
- @@upcase_identifiers = false
+ self.identifier_input_method = nil
+ self.identifier_output_method = nil
attr_reader :sqls
def execute(sql, opts={})
Please sign in to comment.
Something went wrong with that request. Please try again.