Skip to content
This repository

Fix handling of timestamp and datetime columns on PostgreSQL #12012

Open
wants to merge 1 commit into from

5 participants

Roderick van Domburg Jon Rowe Robin Dupret Paul Nikitochkin James Sanders
Roderick van Domburg

Timestamp columns are defined as timestamp with time zone, datetime columns as timestamp without time zone. PostgreSQL converts timestamp values from the local time zone to UTC for storage and back; datetime values are treated as-is. This is in line with MySQL.

While there:

  • Push superfluous column type casting down to super.
  • Make a number of regular expressions a little more efficient.
  • Improve documentation spelling and completeness.
Roderick van Domburg

Build should be restarted because of minitest problems on travis. Test suite passes fine here.

Jon Rowe

Has anyone had a chance to look at this, seems useful!

activerecord/lib/active_record/connection_adapters/postgresql/schema_statements.rb
((6 lines not shown))
460 461
             return super unless precision
461 462
 
462 463
             case precision
463  
-              when 0..6; "timestamp(#{precision})"
464  
-              else raise(ActiveRecordError, "No timestamp type has precision of #{precision}. The allowed range of precision is from 0 to 6")
  464
+              when 0..6; type_string == "datetime" ? "timestamp(#{precision}) without time zone" : "timestamp(#{precision}) with time zone"
  465
+              else raise(ActiveRecordError, "No datetime or timestamp type has precision of #{precision}. The allowed range of precision is from 0 to 6.")
1
Robin Dupret Collaborator
robin850 added a note October 16, 2013

Could you please change this to something which reads better like:

when 0..6
  ...
else
  raise(ActiveRecordError, ...)
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Robin Dupret
Collaborator

Thanks for your patch! :heart:

I've added a small note and actually, I'm wondering if you should not add a setup method to the test which would define a @connection instance variable storing the ActiveRecord::Base.connection, what do you think ?

Also, could you please rebase your pull request ? It doesn't merge cleanly.

Roderick van Domburg
Roderick van Domburg

@robin850, I just updated my fork with your suggestions.

Paul Nikitochkin

Please squash your commits!

Robin Dupret
Collaborator

Also, please rebase your pull request, this doesn't merge cleanly. Thank you!

Roderick van Domburg

Squashed and rebased!

Roderick van Domburg Fix handling of timestamp and datetime columns on PostgreSQL.
Timestamp columns are defined as `timestamp with time zone`,
datetime columns as `timestamp without time zone`. PostgreSQL
converts timestamp values from the local time zone to UTC for
storage and back; datetime values are treated as-is. This is in
line with MySQL.
f7c2b00
James Sanders

@robin850: Any progress on getting this merged in? I'd really like to be able to use timestamps with timezone.

Robin Dupret
Collaborator

@jsanders : Sorry, I should have mentioned it but I'm not a merger. I've snuck it under the 4.2.0 milestone.

James Sanders

@robin850: Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 1 unique commit by 1 author.

Jan 05, 2014
Roderick van Domburg Fix handling of timestamp and datetime columns on PostgreSQL.
Timestamp columns are defined as `timestamp with time zone`,
datetime columns as `timestamp without time zone`. PostgreSQL
converts timestamp values from the local time zone to UTC for
storage and back; datetime values are treated as-is. This is in
line with MySQL.
f7c2b00
This page is out of date. Refresh to see the latest.
10  activerecord/CHANGELOG.md
Source Rendered
... ...
@@ -1,3 +1,13 @@
  1
+*   Fix handling of timestamp and datetime columns on PostgreSQL.
  2
+
  3
+    Timestamp columns are defined as `timestamp with time zone`,
  4
+    datetime columns as `timestamp without time zone`. PostgreSQL
  5
+    converts timestamp values from the local time zone to UTC for
  6
+    storage and back; datetime values are treated as-is. This is in
  7
+    line with MySQL.
  8
+
  9
+    *Roderick van Domburg*
  10
+
1 11
 *   Deprecate unused `ActiveRecord::Base.symbolized_base_class`
2 12
     and `ActiveRecord::Base.symbolized_sti_name` without replacement.
3 13
 
21  activerecord/lib/active_record/connection_adapters/postgresql/schema_statements.rb
@@ -432,20 +432,21 @@ def index_name_length
432 432
 
433 433
         # Maps logical Rails types to PostgreSQL-specific data types.
434 434
         def type_to_sql(type, limit = nil, precision = nil, scale = nil)
435  
-          case type.to_s
  435
+          type_string = type.to_s
  436
+          case type_string
436 437
           when 'binary'
437 438
             # PostgreSQL doesn't support limits on binary (bytea) columns.
438  
-            # The hard limit is 1Gb, because of a 32-bit size field, and TOAST.
  439
+            # The hard limit is 1 GB, because of a 32-bit size field, and TOAST.
439 440
             case limit
440 441
             when nil, 0..0x3fffffff; super(type)
441 442
             else raise(ActiveRecordError, "No binary type has byte size #{limit}.")
442 443
             end
443 444
           when 'text'
444 445
             # PostgreSQL doesn't support limits on text columns.
445  
-            # The hard limit is 1Gb, according to section 8.3 in the manual.
  446
+            # The hard limit is 1 GB, according to section 8.3 in the manual.
446 447
             case limit
447 448
             when nil, 0..0x3fffffff; super(type)
448  
-            else raise(ActiveRecordError, "The limit on text can be at most 1GB - 1byte.")
  449
+            else raise(ActiveRecordError, "The limit on text can be at most 1 GB - 1 byte.")
449 450
             end
450 451
           when 'integer'
451 452
             return 'integer' unless limit
@@ -456,12 +457,18 @@ def type_to_sql(type, limit = nil, precision = nil, scale = nil)
456 457
               when 5..8; 'bigint'
457 458
               else raise(ActiveRecordError, "No integer type has byte size #{limit}. Use a numeric with precision 0 instead.")
458 459
             end
459  
-          when 'datetime'
  460
+          when /^(?:datetime|timestamp)$/
460 461
             return super unless precision
461 462
 
462 463
             case precision
463  
-              when 0..6; "timestamp(#{precision})"
464  
-              else raise(ActiveRecordError, "No timestamp type has precision of #{precision}. The allowed range of precision is from 0 to 6")
  464
+            when 0..6
  465
+              if type_string == 'datetime'
  466
+                "timestamp(#{precision}) without time zone"
  467
+              else
  468
+                "timestamp(#{precision}) with time zone"
  469
+              end
  470
+            else
  471
+              raise(ActiveRecordError, "No datetime or timestamp type has precision of #{precision}. The allowed range of precision is from 0 to 6.")
465 472
             end
466 473
           else
467 474
             super
37  activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb
@@ -168,9 +168,9 @@ def has_default_function?(default_value, default)
168 168
 
169 169
         def extract_limit(sql_type)
170 170
           case sql_type
171  
-          when /^bigint/i;    8
172  
-          when /^smallint/i;  2
173  
-          when /^timestamp/i; nil
  171
+          when /^bigint/;    8
  172
+          when /^smallint/;  2
  173
+          when /^timestamp/; nil
174 174
           else super
175 175
           end
176 176
         end
@@ -178,14 +178,14 @@ def extract_limit(sql_type)
178 178
         # Extracts the scale from PostgreSQL-specific data types.
179 179
         def extract_scale(sql_type)
180 180
           # Money type has a fixed scale of 2.
181  
-          sql_type =~ /^money/ ? 2 : super
  181
+          sql_type == 'money' ? 2 : super
182 182
         end
183 183
 
184 184
         # Extracts the precision from PostgreSQL-specific data types.
185 185
         def extract_precision(sql_type)
186 186
           if sql_type == 'money'
187 187
             self.class.money_precision
188  
-          elsif sql_type =~ /timestamp/i
  188
+          elsif sql_type =~ /^timestamp/
189 189
             $1.to_i if sql_type =~ /\((\d+)\)/
190 190
           else
191 191
             super
@@ -195,14 +195,16 @@ def extract_precision(sql_type)
195 195
         # Maps PostgreSQL-specific data types to logical Rails types.
196 196
         def simplified_type(field_type)
197 197
           case field_type
198  
-          # Numeric and monetary types
199  
-          when /^(?:real|double precision)$/
  198
+          # Numeric type
  199
+          when 'real'
200 200
             :float
201  
-          # Monetary types
  201
+          # Monetary type
202 202
           when 'money'
203 203
             :decimal
  204
+          # Key/value pair type
204 205
           when 'hstore'
205 206
             :hstore
  207
+          # Tree-like type
206 208
           when 'ltree'
207 209
             :ltree
208 210
           # Network address types
@@ -212,14 +214,13 @@ def simplified_type(field_type)
212 214
             :cidr
213 215
           when 'macaddr'
214 216
             :macaddr
215  
-          # Character types
216  
-          when /^(?:character varying|bpchar)(?:\(\d+\))?$/
217  
-            :string
218  
-          # Binary data types
  217
+          # Binary data type
219 218
           when 'bytea'
220 219
             :binary
221 220
           # Date/time types
222  
-          when /^timestamp with(?:out)? time zone$/
  221
+          # Only timestamps *without* time zone are of type :datetime.
  222
+          # Timestamps *with* time zone are of type :timestamp and are handled by super.
  223
+          when /^timestamp(?:\(\d+\))? without time zone$/
223 224
             :datetime
224 225
           when /^interval(?:|\(\d+\))$/
225 226
             :string
@@ -238,7 +239,7 @@ def simplified_type(field_type)
238 239
           # Arrays
239 240
           when /^\D+\[\]$/
240 241
             :string
241  
-          # Object identifier types
  242
+          # Object identifier type
242 243
           when 'oid'
243 244
             :integer
244 245
           # UUID type
@@ -247,9 +248,7 @@ def simplified_type(field_type)
247 248
           # JSON type
248 249
           when 'json'
249 250
             :json
250  
-          # Small and big integer types
251  
-          when /^(?:small|big)int$/
252  
-            :integer
  251
+          # Range types
253 252
           when /(num|date|tstz|ts|int4|int8)range$/
254 253
             field_type.to_sym
255 254
           # Pass through all types that are not specific to PostgreSQL.
@@ -421,8 +420,8 @@ class Table < ActiveRecord::ConnectionAdapters::Table
421 420
         integer:     { name: "integer" },
422 421
         float:       { name: "float" },
423 422
         decimal:     { name: "decimal" },
424  
-        datetime:    { name: "timestamp" },
425  
-        timestamp:   { name: "timestamp" },
  423
+        datetime:    { name: "timestamp without time zone" },
  424
+        timestamp:   { name: "timestamp with time zone" },
426 425
         time:        { name: "time" },
427 426
         date:        { name: "date" },
428 427
         daterange:   { name: "daterange" },
69  activerecord/test/cases/adapters/postgresql/timestamp_test.rb
@@ -2,9 +2,13 @@
2 2
 require 'models/developer'
3 3
 require 'models/topic'
4 4
 
5  
-class TimestampTest < ActiveRecord::TestCase
  5
+class PostgresqlTimestampTest < ActiveRecord::TestCase
6 6
   fixtures :topics
7 7
 
  8
+  def setup
  9
+    @connection = ActiveRecord::Base.connection
  10
+  end
  11
+
8 12
   def test_group_by_date
9 13
     keys = Topic.group("date_trunc('month', created_at)").count.keys
10 14
     assert_operator keys.length, :>, 0
@@ -29,23 +33,28 @@ def test_save_infinity_and_beyond
29 33
     assert_equal(-1.0 / 0.0, d.updated_at)
30 34
   end
31 35
 
32  
-  def test_default_datetime_precision
33  
-    ActiveRecord::Base.connection.create_table(:foos)
34  
-    ActiveRecord::Base.connection.add_column :foos, :created_at, :datetime
35  
-    ActiveRecord::Base.connection.add_column :foos, :updated_at, :datetime
  36
+  def test_default_datetime_and_timestamp_precision
  37
+    @connection.create_table(:foos)
  38
+    @connection.add_column :foos, :created_at, :timestamp
  39
+    @connection.add_column :foos, :birth_date, :datetime
36 40
     assert_nil activerecord_column_option('foos', 'created_at', 'precision')
  41
+    assert_nil activerecord_column_option('foos', 'birth_date', 'precision')
37 42
   end
38 43
 
39  
-  def test_timestamp_data_type_with_precision
40  
-    ActiveRecord::Base.connection.create_table(:foos)
41  
-    ActiveRecord::Base.connection.add_column :foos, :created_at, :datetime, :precision => 0
42  
-    ActiveRecord::Base.connection.add_column :foos, :updated_at, :datetime, :precision => 5
43  
-    assert_equal 0, activerecord_column_option('foos', 'created_at', 'precision')
44  
-    assert_equal 5, activerecord_column_option('foos', 'updated_at', 'precision')
  44
+  def test_datetime_and_timestamp_with_precision
  45
+    @connection.create_table(:foos)
  46
+    @connection.add_column :foos, :created_at,  :timestamp, :precision => 0
  47
+    @connection.add_column :foos, :updated_at,  :timestamp, :precision => 5
  48
+    @connection.add_column :foos, :birth_date,  :datetime,  :precision => 0
  49
+    @connection.add_column :foos, :anniversary, :datetime,  :precision => 5
  50
+    assert_equal 0, activerecord_column_option('foos', 'created_at',  'precision')
  51
+    assert_equal 5, activerecord_column_option('foos', 'updated_at',  'precision')
  52
+    assert_equal 0, activerecord_column_option('foos', 'birth_date',  'precision')
  53
+    assert_equal 5, activerecord_column_option('foos', 'anniversary', 'precision')
45 54
   end
46 55
 
47 56
   def test_timestamps_helper_with_custom_precision
48  
-    ActiveRecord::Base.connection.create_table(:foos) do |t|
  57
+    @connection.create_table(:foos) do |t|
49 58
       t.timestamps :precision => 4
50 59
     end
51 60
     assert_equal 4, activerecord_column_option('foos', 'created_at', 'precision')
@@ -53,27 +62,35 @@ def test_timestamps_helper_with_custom_precision
53 62
   end
54 63
 
55 64
   def test_passing_precision_to_timestamp_does_not_set_limit
56  
-    ActiveRecord::Base.connection.create_table(:foos) do |t|
  65
+    @connection.create_table(:foos) do |t|
57 66
       t.timestamps :precision => 4
58 67
     end
59  
-    assert_nil activerecord_column_option("foos", "created_at", "limit")
60  
-    assert_nil activerecord_column_option("foos", "updated_at", "limit")
  68
+    assert_nil activerecord_column_option('foos', 'created_at', 'limit')
  69
+    assert_nil activerecord_column_option('foos', 'updated_at', 'limit')
61 70
   end
62 71
 
63 72
   def test_invalid_timestamp_precision_raises_error
64 73
     assert_raises ActiveRecord::ActiveRecordError do
65  
-      ActiveRecord::Base.connection.create_table(:foos) do |t|
66  
-        t.timestamps :precision => 7
  74
+      @connection.create_table(:foos) do |t|
  75
+        t.timestamp :created_at, :precision => 7
  76
+      end
  77
+    end
  78
+  end
  79
+
  80
+  def test_invalid_datetime_precision_raises_error
  81
+    assert_raises ActiveRecord::ActiveRecordError do
  82
+      @connection.create_table(:foos) do |t|
  83
+        t.datetime :birth_date, :precision => 7
67 84
       end
68 85
     end
69 86
   end
70 87
 
71 88
   def test_postgres_agrees_with_activerecord_about_precision
72  
-    ActiveRecord::Base.connection.create_table(:foos) do |t|
  89
+    @connection.create_table(:foos) do |t|
73 90
       t.timestamps :precision => 4
74 91
     end
75  
-    assert_equal '4', pg_datetime_precision('foos', 'created_at')
76  
-    assert_equal '4', pg_datetime_precision('foos', 'updated_at')
  92
+    assert_equal '4', pg_datetime_precision("foos", "created_at")
  93
+    assert_equal '4', pg_datetime_precision("foos", "updated_at")
77 94
   end
78 95
 
79 96
   def test_bc_timestamp
@@ -82,10 +99,18 @@ def test_bc_timestamp
82 99
     assert_equal date, Developer.find_by_name("aaron").updated_at
83 100
   end
84 101
 
  102
+  def test_datetime_and_timestamp_types
  103
+    @connection.create_table(:foos)
  104
+    @connection.add_column :foos, :birth_date, :datetime
  105
+    @connection.add_column :foos, :created_at, :timestamp
  106
+    assert_equal :datetime,  activerecord_column_option('foos', 'birth_date', 'type')
  107
+    assert_equal :timestamp, activerecord_column_option('foos', 'created_at', 'type')
  108
+  end
  109
+
85 110
   private
86 111
 
87 112
     def pg_datetime_precision(table_name, column_name)
88  
-      results = ActiveRecord::Base.connection.execute("SELECT column_name, datetime_precision FROM information_schema.columns WHERE table_name ='#{table_name}'")
  113
+      results = @connection.execute("SELECT column_name, datetime_precision FROM information_schema.columns WHERE table_name ='#{table_name}'")
89 114
       result = results.find do |result_hash|
90 115
         result_hash["column_name"] == column_name
91 116
       end
@@ -93,7 +118,7 @@ def pg_datetime_precision(table_name, column_name)
93 118
     end
94 119
 
95 120
     def activerecord_column_option(tablename, column_name, option)
96  
-      result = ActiveRecord::Base.connection.columns(tablename).find do |column|
  121
+      result = @connection.columns(tablename).find do |column|
97 122
         column.name == column_name
98 123
       end
99 124
       result && result.send(option)
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.