Permalink
Browse files

Massive changes: Identifier input/output converter methods, H2 JDBC s…

…ubadapter, many fixes

This is a huge commit, and I'm too lazy to break it down into a
smaller one, since a lot of the parts are interdependent.

The biggest change is the addition of identifier input and output
methods.  The identifier_input_method is used when when literalizing
identifiers going into the database.  For example:

  DB[:table].insert(:column=>'value')

It will affect table and column, but not value since that is not an
identifier.  The default value is :upcase, since the SQL standard is
that unquoted identifiers are folded to uppercase, so you get:

  INSERT INTO "TABLE" ("COLUMN") VALUES ('value')

If you set the identifier_input_method to nil, which the MySQL,
PostgreSQL, and SQLite adapters do by default, you get:

  INSERT INTO "table" ("column") VALUES ('value')

The identifier_output_method is used when retrieving information
from the database.  Since SQL identifier are generally uppercase
(since that's what unquoted identifiers are folded to), and we
generally want to deal with them in lowercase, the default is
:downcase, so that you get lowercase identifiers even on a database
with uppercase identifiers.

Like most Sequel settings, this can be set globally, per database,
or per dataset:

  # Global
  Sequel.identifier_input_method = :downcase
  Sequel.identifier_output_method = :upcase
  # Per Database
  DB.identifier_input_method = :camelize
  DB.identifier_output_method = :underscore
  # Per Dataset
  ds = DB[:table]
  ds.identifier_input_method = :underscore
  ds.identifier_output_method = :reverse

I could have added a downcase_identifiers setting similar to the
upcase_identifiers setting, but that is confusing since they
operate differently.  The identifier input/output method naming
makes more sense, and gives you the flexibility to do something
crazy like :camelize and :underscore, which you may need to do
if you use CamelCase names in your database.

The upcase_identifiers setting now just sets the
identifier_input_method to :upcase (or nil if it is set to false).

The identifier_output_method is used inside of fetch_rows (actually,
a method that fetch_rows calls), and therefore it requires
changes to all adapters (since that's where the database specific
fetching happens).  As I only test 5 of the adapters and Sequel
supports 13, I need everyone that uses an adapter other than
MySQL, PostgreSQL, SQLite, JDBC, and DataObjects to test this
commit against their database and make sure I didn't break anything.
Unlike usual, not all of the changes I made to adapters I don't have
access to are small.  The most significant changes that I did not
test were to the firebird and informix adapters.

This commit adds an H2 subadapter for JDBC.  H2 is an embedded
database similar to SQLite, but it is easier for JRuby users to use
as it doesn't rely on any native code.  It passes all integration
tests with the exception of the schema tests, because it doesn't
support schema parsing yet.

This commit also adds a Database#tables method to the JDBC adapter,
and hopefully full schema parsing for JDBC will be coming soon.

This commit also changes the JDBC adapter so that Java specific
types such as Java::JavaSQL::Timestamp, Java::JavaSQL::Time,
and Java::JavaIo::BufferedReader are converted into normal ruby
types before being returned to the user.  This fixed a few bugs
in the intergation tests, and should make using the JDBC adapter
easier.

This commit fixes the literalization of Times and DateTimes when
using the JDBC MySQL subadapter.  It also fixes the use of
SQL::Blobs when using the JDBC PostgreSQL subadapter.

This commit refactors the MySQL native adapter so as to not
modify the Mysql::Result class.  This was necessary in order to
implement the identifier_output_method support.

This commit makes Database#quote_identifiers= have affect on
future usages of the schema modification methods such as
create_table, by removing the cached schema_utility_dataset.
The identifier input/output methods have the same effect.

This commit changes the default quoted_identifier method to
double any double quotes in the identifier, in addition to
surrounding the identifier in double quotes.

This commit changes the MySQL specs to only run the socket
specs if the native mysql adapter is being used, since it
uses the native mysql adapter to run those specs.

Due to implementation details,
Database.identifier_(input|output)_method returns "" if no
method should be used, and nil if it should fall back to the
adapter support.  You should not rely on this, though I don't plan
on changing it.

This commit breaks backwards compatibility in the following ways:

Using a database other than MySQL, PostgreSQL, or SQLite, if you
used to get uppercase symbols when retrieving records, you will
now probably get lowercase symbols.  To continue to receive
uppercase symbols:

  DB.identifier_output_method = nil

The Database#lowercase method in the DBI adapter was removed, as
it isn't needed since the default is to lowercase identifiers.
  • Loading branch information...
jeremyevans committed Jan 24, 2009
1 parent 885f92a commit cf4b1558649cda1880cc382ddef360e1a41ef339
View
@@ -69,6 +69,37 @@ def self.connect(*args, &block)
end
metaalias :open, :connect
+ # Set the method to call on identifiers going into the database. This affects
+ # the literalization of identifiers by calling this method on them before they are input.
+ # Sequel upcases identifiers in all SQL strings for most databases, so to turn that off:
+ #
+ # Sequel.identifier_input_method = nil
+ #
+ # to downcase instead:
+ #
+ # Sequel.identifier_input_method = :downcase
+ #
+ # Other string methods work as well.
+ def self.identifier_input_method=(value)
+ Database.identifier_input_method = value
+ end
+
+ # Set the method to call on identifiers coming out of the database. This affects
+ # the literalization of identifiers by calling this method on them when they are
+ # retrieved from the database. Sequel downcases identifiers retrieved for most
+ # databases, so to turn that off:
+ #
+ # Sequel.identifier_output_method = nil
+ #
+ # to upcase instead:
+ #
+ # Sequel.identifier_output_method = :upcase
+ #
+ # Other string methods work as well.
+ def self.identifier_output_method=(value)
+ Database.identifier_output_method = value
+ end
+
# Set whether to quote identifiers for all databases by default. By default,
# Sequel quotes identifiers in all SQL strings, so to turn that off:
#
@@ -92,6 +123,9 @@ def self.single_threaded=(value)
# lower case (MySQL, PostgreSQL, and SQLite).
#
# Sequel.upcase_identifiers = false
+ #
+ # This will set the indentifier_input_method to :upcase if value is true
+ # or nil if value is false.
def self.upcase_identifiers=(value)
Database.upcase_identifiers = value
end
@@ -58,7 +58,7 @@ def fetch_rows(sql)
execute(sql) do |s|
@columns = s.Fields.extend(Enumerable).map do |column|
name = column.Name.empty? ? '(no column name)' : column.Name
- name.to_sym
+ output_identifier(name)
end
unless s.eof
@@ -89,7 +89,7 @@ def literal(v)
def fetch_rows(sql)
execute(sql) do |sth|
@column_info = get_column_info(sth)
- @columns = @column_info.map {|c| c[:name]}
+ @columns = @column_info.map {|c| output_identifier(c[:name])}
while (rc = SQLFetch(@handle)) != SQL_NO_DATA_FOUND
@db.check_error(rc, "Could not fetch row")
yield hash_row(sth)
@@ -118,7 +118,7 @@ def hash_row(sth)
rc, v = SQLGetData(sth, i+1, c[:db2_type], c[:precision])
@db.check_error(rc, "Could not get data")
- @row[c[:name]] = convert_type(v)
+ row[output_identifier(c[:name])] = convert_type(v)
end
row
end
@@ -3,8 +3,6 @@
module Sequel
module DBI
class Database < Sequel::Database
- attr_writer :lowercase
-
set_adapter_scheme :dbi
DBI_ADAPTERS = {
@@ -69,11 +67,6 @@ def do(sql, opts={})
synchronize(opts[:server]){|conn| conn.do(sql)}
end
alias_method :execute_dui, :do
-
- # Converts all column names to lowercase
- def lowercase
- @lowercase ||= false
- end
private
@@ -97,10 +90,8 @@ def literal(v)
def fetch_rows(sql, &block)
execute(sql) do |s|
begin
- @columns = s.column_names.map do |c|
- @db.lowercase ? c.downcase.to_sym : c.to_sym
- end
- s.fetch {|r| yield hash_row(s, r)}
+ @columns = s.column_names.map{|c| output_identifier(c)}
+ s.fetch{|r| yield hash_row(s, r)}
ensure
s.finish rescue nil
end
@@ -191,7 +191,7 @@ def literal(v)
# with symbol keys.
def fetch_rows(sql)
execute(sql) do |reader|
- cols = @columns = reader.fields.map{|f| f.to_sym}
+ cols = @columns = reader.fields.map{|f| output_identifier(f)}
while(reader.next!) do
h = {}
cols.zip(reader.values).each{|k, v| h[k] = v}
@@ -216,10 +216,6 @@ def disconnect_connection(c)
def quote_identifiers_default
false
end
-
- def upcase_identifiers_default
- true
- end
end
# Dataset class for Firebird datasets
@@ -237,10 +233,12 @@ class Dataset < Sequel::Dataset
def fetch_rows(sql, &block)
execute(sql) do |s|
begin
- @columns = s.fields.map do |c|
- c.name.to_sym
+ @columns = s.fields.map{|c| output_identifier(c)}
+ s.fetchall(:symbols_hash).each do |r|
+ h = {}
+ r.each{|k,v| h[output_identifier(k)] = v}
+ yield h
end
- s.fetchall(:symbols_hash).each{ |r| yield r}
ensure
s.close
end
@@ -288,8 +286,7 @@ def literal(v)
end
def quote_identifier(name)
- name = super
- Fb::Global::reserved_keyword?(name) ? quoted_identifier(name.upcase) : name
+ Fb::Global::reserved_keyword?(name) ? quoted_identifier(name.upcase) : super
end
# The order of clauses in the SELECT SQL statement
@@ -53,7 +53,16 @@ def literal(v)
def fetch_rows(sql, &block)
execute(sql) do |cursor|
begin
- cursor.open.each_hash(&block)
+ col_map = nil
+ cursor.open.each_hash do |h|
+ unless col_map
+ col_map = {}
+ @columns = h.keys.map{|k| col_map[k] = output_identifier(k)}
+ end
+ h2 = {}
+ h.each{|k,v| h2[col_map[k]||k] = v}
+ yield h2
+ end
ensure
cursor.drop
end
@@ -61,6 +61,12 @@ module JavaSQL
require 'sequel_core/adapters/shared/mssql'
db.extend(Sequel::MSSQL::DatabaseMethods)
com.microsoft.sqlserver.jdbc.SQLServerDriver
+ end,
+ :h2=>proc do |db|
+ require 'sequel_core/adapters/jdbc/h2'
+ db.extend(Sequel::JDBC::H2::DatabaseMethods)
+ JDBC.load_gem('h2')
+ org.h2.Driver
end
}
@@ -179,6 +185,13 @@ def execute_insert(sql, opts={})
execute(sql, {:type=>:insert}.merge(opts))
end
+ # All tables in this database
+ def tables(server=nil)
+ ts = []
+ synchronize(server){|c| dataset.send(:process_result_set, c.getMetaData.getTables(nil, nil, nil, ['TABLE'].to_java(:string))){|h| ts << dataset.send(:output_identifier, h[:table_name])}}
+ ts
+ end
+
# Default transaction method that should work on most JDBC
# databases. Does not use the JDBC transaction methods, uses
# SQL BEGIN/ROLLBACK/COMMIT statements instead.
@@ -373,20 +386,7 @@ def execute_insert(sql, opts={}, &block)
# Correctly return rows from the database and return them as hashes.
def fetch_rows(sql, &block)
- execute(sql) do |result|
- # get column names
- meta = result.getMetaData
- column_count = meta.getColumnCount
- @columns = []
- column_count.times {|i| @columns << meta.getColumnName(i+1).to_sym}
-
- # get rows
- while result.next
- row = {}
- @columns.each_with_index {|v, i| row[v] = result.getObject(i+1)}
- yield row
- end
- end
+ execute(sql){|result| process_result_set(result, &block)}
self
end
@@ -416,10 +416,41 @@ def prepare(type, name=nil, values=nil)
private
+ # Convert the type. Used for converting Java types to ruby types.
+ def convert_type(v)
+ case v
+ when Java::JavaSQL::Timestamp, Java::JavaSQL::Time
+ v.to_string.to_sequel_time
+ when Java::JavaIo::BufferedReader
+ lines = []
+ while(line = v.read_line) do lines << line end
+ lines.join("\n")
+ else
+ v
+ end
+ end
+
# Extend the dataset with the JDBC stored procedure methods.
def prepare_extend_sproc(ds)
ds.extend(StoredProcedureMethods)
end
+
+ # Split out from fetch rows to allow processing of JDBC result sets
+ # that don't come from issuing an SQL string.
+ def process_result_set(result)
+ # get column names
+ meta = result.getMetaData
+ column_count = meta.getColumnCount
+ @columns = []
+ column_count.times {|i| @columns << output_identifier(meta.getColumnName(i+1))}
+
+ # get rows
+ while result.next
+ row = {}
+ @columns.each_with_index {|v, i| row[v] = convert_type(result.getObject(i+1))}
+ yield row
+ end
+ end
end
end
end
@@ -0,0 +1,54 @@
+module Sequel
+ module JDBC
+ # Database and Dataset support for H2 databases accessed via JDBC.
+ module H2
+ # Instance methods for H2 Database objects accessed via JDBC.
+ module DatabaseMethods
+ # Return Sequel::JDBC::H2::Dataset object with the given opts.
+ def dataset(opts=nil)
+ Sequel::JDBC::H2::Dataset.new(self, opts)
+ end
+
+ # H2 uses an IDENTITY type
+ def serial_primary_key_options
+ {:primary_key => true, :type => :identity}
+ end
+
+ private
+
+ # Use IDENTITY() to get the last inserted id.
+ def last_insert_id(conn, opts={})
+ stmt = conn.createStatement
+ begin
+ rs = stmt.executeQuery('SELECT IDENTITY();')
+ rs.next
+ rs.getInt(1)
+ ensure
+ stmt.close
+ end
+ end
+
+ # Default to a single connection for a memory database.
+ def connection_pool_default_options
+ o = super
+ uri == 'jdbc:h2:mem:' ? o.merge(:max_connections=>1) : o
+ end
+ end
+
+ # Dataset class for H2 datasets accessed via JDBC.
+ class Dataset < JDBC::Dataset
+ # Use H2 syntax for Date, DateTime, and Time types.
+ def literal(v)
+ case v
+ when Date
+ v.strftime("DATE '%Y-%m-%d'")
+ when DateTime, Time
+ v.strftime("TIMESTAMP '%Y-%m-%d %H:%M:%S'")
+ else
+ super
+ end
+ end
+ end
+ end
+ end
+end
@@ -46,6 +46,16 @@ def insert(*values)
execute_insert(insert_sql(*values))
end
+ # Handle time types correctly
+ def literal(v)
+ case v
+ when Time, DateTime
+ v.strftime("'%Y-%m-%d %H:%M:%S'")
+ else
+ super
+ end
+ end
+
# Use execute_insert to execute the replace_sql.
def replace(*args)
execute_insert(replace_sql(*args))
@@ -97,6 +97,8 @@ def literal(v)
case v
when LiteralString
v
+ when SQL::Blob
+ super
when String
db.synchronize{|c| "'#{c.escape_string(v)}'"}
when Java::JavaSql::Timestamp
Oops, something went wrong.

0 comments on commit cf4b155

Please sign in to comment.