Skip to content
This repository

schema_format :sql should behave like schema_format :ruby #2948

Merged
merged 1 commit into from over 2 years ago
Alex Tambellini

This commit adds a db:structure:load task that is run instead of
db:schema:load when schema_format is set to :sql. This patch also removes
the prefixing of the structure.sql files to mimic the use of a single
schema.rb file. The patch originates from github issue #715.

Oleg Dashevskii

+1

Tony Collen

+1 here as well, although it looks like the patch doesn't apply cleanly against master. Any chance to get a more up-to-date patch?

Alex Tambellini

@tonyc, just updated the patch, 50c2ca7 should apply against master now.

Tony Collen
Tony Collen

Playing around with this patch a bit, I don't think it's quite right. When I run "rake," I get errors about duplicate constraints and tables. Rails.env within the tasks is set to "development", so it's trying to pipe structure.sql into my development database for tests, which is wrong, since we're running tests. We can't just hardcode the task to use abcs['test']['database'] though, because then we lose the ability to manually run "rake db:structure:load" and have the structure sucked into the development DB.

Tony Collen

@atambo, I've made some changes that seem to work for me so far, at least while running 'rake' . I'm going to test it with other various tasks to see if it's working correctly.

Edit: Here's a squashed changeset with mine applied atop of @atambo's:

tonyc/rails@e40a0f3

Tony Collen

FWIW, a coworker and I have been using my squashed changeset in a fork of Rails for a number of weeks now without issues, and I'm fairly happy with how it's working so far.

Alex Tambellini schema_format :sql should behave like schema_format :ruby
This commit adds a db:structure:load task that is run instead of
db:schema:load when schema_format is set to :sql. This patch also removes
the prefixing of the structure.sql files to mimic the use of a single
schema.rb file. The patch originates from github issue #715.
15fb430
Alex Tambellini

I've cherry-picked @tonyc's changes.

Aaron Patterson tenderlove merged commit 43821bf into from November 16, 2011
Aaron Patterson tenderlove closed this November 16, 2011
Tony Collen

Thanks! <3 <3

Nick Gauthier

OMG!

Mike Subelsky

this is awesome, will definitely save me time/pain in a constraint-heavy app I'm working on

Gopal Patel

Will this be backported to a 3.1.x release?

Juan M. Cuello

I think this commit introduces some issues.

Previously to these changes, when I executed rake db:test:clone_structure
the structure was dumped (to structure.sql) and then loaded again no matter whether
the config.active_record.schema_format was set to :sql or :ruby. That was OK
because I'm explicitly saying that I want the structure to be cloned.

With these changes, when I call now the same task, although the structure is
dumped, the loaded schema depends on the value of the
config.active_record.schema_format. So If I have it set to :ruby and then I
call rake db:test:clone_structure the loaded file is schema.rb and not the
structure.sql how it is supposed to be according to the name of the task and
its description.

The same thing happens when I call rake db:test:clone. When called,
a new schema.rb is created but the file used to load the schema depends on
config.active_record.schema_format. If I have it set to :sql, a new schema.rb
is created but the structure.sql file is used to create the database. So it is
not consistent.

I didn't create an issue/pull-request because I wanted to discuss it here first,
but in case we decide to make some changes, I think we could do something like this:

Juanmcuello@88a4aea

Derek Prior

If I submitted a pull request that applied cleanly to 3.1-stable, would it be accepted there? I'd love to see these changes make it into the next 3.1 patch while we await 3.2.0 final.

Juan M. Cuello

@derekprior, this was already fixed in #4036. If you want the fix to be included in next release I think you will have to create a pull request to backport it.

Ankur Sethi

@atambo I am on 3.2.3 and this doesn't work.

This is in my application.rb: config.active_record.schema_format = :sql

This is the rake trace:

** Invoke db:setup (first_time)
** Invoke db:schema:load_if_ruby (first_time)
** Invoke db:create (first_time)
** Invoke db:load_config 
** Execute db:create
** Execute db:schema:load_if_ruby
** Invoke db:schema:load (first_time)

In the console:

ActiveRecord::Base.schema_format
=> :sql

I also confirmed that the application.rb is loaded by the rake task.

Alex Tambellini
atambo commented May 21, 2012

@amalagaura, it looks like db:setup is broken when using schema_format :sql. For now you should be able to use db:create db:migrate db:seed db:reset db:drop etc, just not db:setup.

Morgan Christiansson morganchristiansson commented on the diff June 20, 2012
activerecord/lib/active_record/railties/databases.rake
((31 lines not shown))
383 397
       else
384 398
         raise "Task not supported by '#{abcs[Rails.env]["adapter"]}'"
385 399
       end
386 400
 
387 401
       if ActiveRecord::Base.connection.supports_migrations?
388  
-        File.open("#{Rails.root}/db/#{Rails.env}_structure.sql", "a") { |f| f << ActiveRecord::Base.connection.dump_schema_information }
1

postgres pg_dump doesn't reset it's schema so these INSERT statements will fail if the search_path has not been set to include public at end of migration

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Juan M. Cuello

Not sure if it's what you mean, but I think pull request #4132 already fixed it.

See also this line in master.

Morgan Christiansson

Awesome, yes that should have fixed it.

Can't wait for the next rails release for this change - as currently migrating 12 databases with parallel tests in our CI takes 75 seconds (67% of the total setup cost) and this should reduce that quite a lot.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 1 unique commit by 1 author.

Nov 04, 2011
Alex Tambellini schema_format :sql should behave like schema_format :ruby
This commit adds a db:structure:load task that is run instead of
db:schema:load when schema_format is set to :sql. This patch also removes
the prefixing of the structure.sql files to mimic the use of a single
schema.rb file. The patch originates from github issue #715.
15fb430
This page is out of date. Refresh to see the latest.
104  activerecord/lib/active_record/railties/databases.rake
@@ -151,6 +151,7 @@ db_namespace = namespace :db do
151 151
     ActiveRecord::Migration.verbose = ENV["VERBOSE"] ? ENV["VERBOSE"] == "true" : true
152 152
     ActiveRecord::Migrator.migrate(ActiveRecord::Migrator.migrations_paths, ENV["VERSION"] ? ENV["VERSION"].to_i : nil)
153 153
     db_namespace["schema:dump"].invoke if ActiveRecord::Base.schema_format == :ruby
  154
+    db_namespace["structure:dump"].invoke if ActiveRecord::Base.schema_format == :sql
154 155
   end
155 156
 
156 157
   namespace :migrate do
@@ -174,6 +175,7 @@ db_namespace = namespace :db do
174 175
       raise 'VERSION is required' unless version
175 176
       ActiveRecord::Migrator.run(:up, ActiveRecord::Migrator.migrations_paths, version)
176 177
       db_namespace['schema:dump'].invoke if ActiveRecord::Base.schema_format == :ruby
  178
+      db_namespace['structure:dump'].invoke if ActiveRecord::Base.schema_format == :sql
177 179
     end
178 180
 
179 181
     # desc 'Runs the "down" for a given migration VERSION.'
@@ -182,6 +184,7 @@ db_namespace = namespace :db do
182 184
       raise 'VERSION is required' unless version
183 185
       ActiveRecord::Migrator.run(:down, ActiveRecord::Migrator.migrations_paths, version)
184 186
       db_namespace['schema:dump'].invoke if ActiveRecord::Base.schema_format == :ruby
  187
+      db_namespace['structure:dump'].invoke if ActiveRecord::Base.schema_format == :sql
185 188
     end
186 189
 
187 190
     desc 'Display status of migrations'
@@ -222,6 +225,7 @@ db_namespace = namespace :db do
222 225
     step = ENV['STEP'] ? ENV['STEP'].to_i : 1
223 226
     ActiveRecord::Migrator.rollback(ActiveRecord::Migrator.migrations_paths, step)
224 227
     db_namespace['schema:dump'].invoke if ActiveRecord::Base.schema_format == :ruby
  228
+    db_namespace['structure:dump'].invoke if ActiveRecord::Base.schema_format == :sql
225 229
   end
226 230
 
227 231
   # desc 'Pushes the schema to the next version (specify steps w/ STEP=n).'
@@ -229,10 +233,14 @@ db_namespace = namespace :db do
229 233
     step = ENV['STEP'] ? ENV['STEP'].to_i : 1
230 234
     ActiveRecord::Migrator.forward(ActiveRecord::Migrator.migrations_paths, step)
231 235
     db_namespace['schema:dump'].invoke if ActiveRecord::Base.schema_format == :ruby
  236
+    db_namespace['structure:dump'].invoke if ActiveRecord::Base.schema_format == :sql
232 237
   end
233 238
 
234 239
   # desc 'Drops and recreates the database from db/schema.rb for the current environment and loads the seeds.'
235  
-  task :reset => [ 'db:drop', 'db:setup' ]
  240
+  task :reset => :environment do
  241
+    db_namespace["drop"].invoke
  242
+    db_namespace["setup"].invoke
  243
+  end
236 244
 
237 245
   # desc "Retrieves the charset for the current environment's database"
238 246
   task :charset => :environment do
@@ -285,7 +293,12 @@ db_namespace = namespace :db do
285 293
   end
286 294
 
287 295
   desc 'Create the database, load the schema, and initialize with the seed data (use db:reset to also drop the db first)'
288  
-  task :setup => [ 'db:create', 'db:schema:load', 'db:seed' ]
  296
+  task :setup => :environment do
  297
+    db_namespace["create"].invoke
  298
+    db_namespace["schema:load"].invoke if ActiveRecord::Base.schema_format == :ruby
  299
+    db_namespace["structure:load"].invoke if ActiveRecord::Base.schema_format == :sql
  300
+    db_namespace["seed"].invoke
  301
+  end
289 302
 
290 303
   desc 'Load the seed data from db/seeds.rb'
291 304
   task :seed => 'db:abort_if_pending_migrations' do
@@ -360,81 +373,96 @@ db_namespace = namespace :db do
360 373
       case abcs[Rails.env]['adapter']
361 374
       when /mysql/, 'oci', 'oracle'
362 375
         ActiveRecord::Base.establish_connection(abcs[Rails.env])
363  
-        File.open("#{Rails.root}/db/#{Rails.env}_structure.sql", "w+") { |f| f << ActiveRecord::Base.connection.structure_dump }
  376
+        File.open("#{Rails.root}/db/structure.sql", "w:utf-8") { |f| f << ActiveRecord::Base.connection.structure_dump }
364 377
       when /postgresql/
365 378
         ENV['PGHOST']     = abcs[Rails.env]['host'] if abcs[Rails.env]['host']
366  
-        ENV['PGPORT']     = abcs[Rails.env]["port"].to_s if abcs[Rails.env]['port']
  379
+        ENV['PGPORT']     = abcs[Rails.env]['port'].to_s if abcs[Rails.env]['port']
367 380
         ENV['PGPASSWORD'] = abcs[Rails.env]['password'].to_s if abcs[Rails.env]['password']
  381
+        ENV['PGUSER']     = abcs[Rails.env]['username'].to_s if abcs[Rails.env]['username']
368 382
         search_path = abcs[Rails.env]['schema_search_path']
369 383
         unless search_path.blank?
370 384
           search_path = search_path.split(",").map{|search_path_part| "--schema=#{search_path_part.strip}" }.join(" ")
371 385
         end
372  
-        `pg_dump -i -U "#{abcs[Rails.env]['username']}" -s -x -O -f db/#{Rails.env}_structure.sql #{search_path} #{abcs[Rails.env]['database']}`
  386
+        `pg_dump -i -s -x -O -f db/structure.sql #{search_path} #{abcs[Rails.env]['database']}`
373 387
         raise 'Error dumping database' if $?.exitstatus == 1
374 388
       when /sqlite/
375 389
         dbfile = abcs[Rails.env]['database'] || abcs[Rails.env]['dbfile']
376  
-        `sqlite3 #{dbfile} .schema > db/#{Rails.env}_structure.sql`
  390
+        `sqlite3 #{dbfile} .schema > db/structure.sql`
377 391
       when 'sqlserver'
378  
-        `smoscript -s #{abcs[Rails.env]['host']} -d #{abcs[Rails.env]['database']} -u #{abcs[Rails.env]['username']} -p #{abcs[Rails.env]['password']} -f db\\#{Rails.env}_structure.sql -A -U`
  392
+        `smoscript -s #{abcs[Rails.env]['host']} -d #{abcs[Rails.env]['database']} -u #{abcs[Rails.env]['username']} -p #{abcs[Rails.env]['password']} -f db\\structure.sql -A -U`
379 393
       when "firebird"
380 394
         set_firebird_env(abcs[Rails.env])
381 395
         db_string = firebird_db_string(abcs[Rails.env])
382  
-        sh "isql -a #{db_string} > #{Rails.root}/db/#{Rails.env}_structure.sql"
  396
+        sh "isql -a #{db_string} > #{Rails.root}/db/structure.sql"
383 397
       else
384 398
         raise "Task not supported by '#{abcs[Rails.env]["adapter"]}'"
385 399
       end
386 400
 
387 401
       if ActiveRecord::Base.connection.supports_migrations?
388  
-        File.open("#{Rails.root}/db/#{Rails.env}_structure.sql", "a") { |f| f << ActiveRecord::Base.connection.dump_schema_information }
  402
+        File.open("#{Rails.root}/db/structure.sql", "a") { |f| f << ActiveRecord::Base.connection.dump_schema_information }
389 403
       end
390 404
     end
391  
-  end
392  
-
393  
-  namespace :test do
394  
-    # desc "Recreate the test database from the current schema.rb"
395  
-    task :load => 'db:test:purge' do
396  
-      ActiveRecord::Base.establish_connection(ActiveRecord::Base.configurations['test'])
397  
-      ActiveRecord::Schema.verbose = false
398  
-      db_namespace['schema:load'].invoke
399  
-    end
400 405
 
401  
-    # desc "Recreate the test database from the current environment's database schema"
402  
-    task :clone => %w(db:schema:dump db:test:load)
  406
+    # desc "Recreate the databases from the structure.sql file"
  407
+    task :load => [:environment, :load_config] do
  408
+      env = ENV['RAILS_ENV'] || 'test'
403 409
 
404  
-    # desc "Recreate the test databases from the development structure"
405  
-    task :clone_structure => [ 'db:structure:dump', 'db:test:purge' ] do
406 410
       abcs = ActiveRecord::Base.configurations
407  
-      case abcs['test']['adapter']
  411
+      case abcs[env]['adapter']
408 412
       when /mysql/
409  
-        ActiveRecord::Base.establish_connection(:test)
  413
+        ActiveRecord::Base.establish_connection(abcs[env])
410 414
         ActiveRecord::Base.connection.execute('SET foreign_key_checks = 0')
411  
-        IO.readlines("#{Rails.root}/db/#{Rails.env}_structure.sql").join.split("\n\n").each do |table|
  415
+        IO.readlines("#{Rails.root}/db/structure.sql").join.split("\n\n").each do |table|
412 416
           ActiveRecord::Base.connection.execute(table)
413 417
         end
414 418
       when /postgresql/
415  
-        ENV['PGHOST']     = abcs['test']['host'] if abcs['test']['host']
416  
-        ENV['PGPORT']     = abcs['test']['port'].to_s if abcs['test']['port']
417  
-        ENV['PGPASSWORD'] = abcs['test']['password'].to_s if abcs['test']['password']
418  
-        `psql -U "#{abcs['test']['username']}" -f "#{Rails.root}/db/#{Rails.env}_structure.sql" #{abcs['test']['database']} #{abcs['test']['template']}`
  419
+        ENV['PGHOST']     = abcs[env]['host'] if abcs[env]['host']
  420
+        ENV['PGPORT']     = abcs[env]['port'].to_s if abcs[env]['port']
  421
+        ENV['PGPASSWORD'] = abcs[env]['password'].to_s if abcs[env]['password']
  422
+        ENV['PGUSER']     = abcs[env]['username'].to_s if abcs[env]['username']
  423
+        `psql -f "#{Rails.root}/db/structure.sql" #{abcs[env]['database']} #{abcs[env]['template']}`
419 424
       when /sqlite/
420  
-        dbfile = abcs['test']['database'] || abcs['test']['dbfile']
421  
-        `sqlite3 #{dbfile} < "#{Rails.root}/db/#{Rails.env}_structure.sql"`
  425
+        dbfile = abcs[env]['database'] || abcs[env]['dbfile']
  426
+        `sqlite3 #{dbfile} < "#{Rails.root}/db/structure.sql"`
422 427
       when 'sqlserver'
423  
-        `sqlcmd -S #{abcs['test']['host']} -d #{abcs['test']['database']} -U #{abcs['test']['username']} -P #{abcs['test']['password']} -i db\\#{Rails.env}_structure.sql`
  428
+        `sqlcmd -S #{abcs[env]['host']} -d #{abcs[env]['database']} -U #{abcs[env]['username']} -P #{abcs[env]['password']} -i db\\structure.sql`
424 429
       when 'oci', 'oracle'
425  
-        ActiveRecord::Base.establish_connection(:test)
426  
-        IO.readlines("#{Rails.root}/db/#{Rails.env}_structure.sql").join.split(";\n\n").each do |ddl|
  430
+        ActiveRecord::Base.establish_connection(abcs[env])
  431
+        IO.readlines("#{Rails.root}/db/structure.sql").join.split(";\n\n").each do |ddl|
427 432
           ActiveRecord::Base.connection.execute(ddl)
428 433
         end
429 434
       when 'firebird'
430  
-        set_firebird_env(abcs['test'])
431  
-        db_string = firebird_db_string(abcs['test'])
432  
-        sh "isql -i #{Rails.root}/db/#{Rails.env}_structure.sql #{db_string}"
  435
+        set_firebird_env(abcs[env])
  436
+        db_string = firebird_db_string(abcs[env])
  437
+        sh "isql -i #{Rails.root}/db/structure.sql #{db_string}"
433 438
       else
434  
-        raise "Task not supported by '#{abcs['test']['adapter']}'"
  439
+        raise "Task not supported by '#{abcs[env]['adapter']}'"
  440
+      end
  441
+    end
  442
+  end
  443
+
  444
+  namespace :test do
  445
+    # desc "Recreate the test database from the current schema.rb"
  446
+    task :load => 'db:test:purge' do
  447
+      ActiveRecord::Base.establish_connection(ActiveRecord::Base.configurations['test'])
  448
+      ActiveRecord::Schema.verbose = false
  449
+      db_namespace["schema:load"].invoke if ActiveRecord::Base.schema_format == :ruby
  450
+
  451
+      begin
  452
+        old_env, ENV['RAILS_ENV'] = ENV['RAILS_ENV'], 'test'
  453
+        db_namespace["structure:load"].invoke if ActiveRecord::Base.schema_format == :sql
  454
+      ensure
  455
+        ENV['RAILS_ENV'] = old_env
435 456
       end
  457
+
436 458
     end
437 459
 
  460
+    # desc "Recreate the test database from the current environment's database schema"
  461
+    task :clone => %w(db:schema:dump db:test:load)
  462
+
  463
+    # desc "Recreate the test databases from the structure.sql file"
  464
+    task :clone_structure => [ "db:structure:dump", "db:test:load" ]
  465
+
438 466
     # desc "Empty the test database"
439 467
     task :purge => :environment do
440 468
       abcs = ActiveRecord::Base.configurations
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.